Automatic firmware testing method and device based on fuzzy input and medium

Abstract

The invention discloses a firmware automatic testing method and device based on fuzzy input and a medium. Relates to the field of computer security, and solves the problems of low efficiency and incompleteness of manual tampering in firmware testing. According to the method, the part to be modified in the firmware is tampered in a fuzzy input mode, unpredictable tampering behaviors of attacks can be simulated better, the test cases are generated by creating the parameters of the multiple groups of attribute permutation and combination and used for testing different tampering modes and degrees, a large number of test cases can be generated and executed without manual intervention, and the test efficiency is improved. Therefore, the test efficiency is remarkably improved, a plurality of potential vulnerabilities and weaknesses in the firmware can be covered, and the test comprehensiveness is improved.

Description

Technical Field

[0001] The present application relates to the field of computer security, and in particular to a firmware automated testing method, device and medium based on fuzzy input. Background Art

[0002] In the automotive industry, firmware tampering testing is an important part of ensuring the stability and security of vehicle systems. It can simulate hacker attacks or unexpected failures to help automakers identify and fix potential security vulnerabilities. This is crucial to protecting vehicles from malware attacks and ensuring the safety of drivers and passengers.

[0003] At present, when it is necessary to test the firmware tampering case, human intervention is required. The tester opens the file to manually modify the firmware and inputs the faulty firmware file into the test system for testing. Manually modify the byte data of a certain part of the firmware to a fixed value. If you want to change the parameters during the test, you need to re-create a faulty firmware file. Even if different values ​​are used for each test, these values ​​are simple and cannot simulate unpredictable tampering behavior. When testing different samples and different firmware structures, manual tampering is required again.

[0004] It can be seen that how to solve the problem of low efficiency and incompleteness of manual tampering in firmware testing is a technical problem that needs to be urgently solved by people in this field. Summary of the invention

[0005] The purpose of this application is to provide a firmware automated testing method, device and medium based on fuzzy input to solve the problem of low efficiency and incompleteness of manual tampering.

[0006] In order to solve the above technical problems, the present application provides a firmware automatic testing method based on fuzzy input, comprising:

[0007] Obtain and parse the firmware file to be analyzed;

[0008] Generate fuzzy data based on a preset fuzzy input algorithm and a preset random value;

[0009] According to different permutations and combinations of various states in the preset tampering block type, the preset tampering area, the preset tampering starting position, the preset tampering method, the preset fuzzy input method, the fuzzy data, and the preset tampering length, a plurality of test items are obtained, and a parameter structure list is generated;

[0010] Constructing a parameter-driven firmware tampering test case according to the parameter structure list and generating a corresponding executable file;

[0011] The executable file is imported into the automated testing framework for execution to generate test results.

[0012] As an optional solution, in the above-mentioned firmware automatic testing method based on fuzzy input, the step of obtaining and parsing the firmware file to be analyzed includes:

[0013] Configure the preset firmware full path parameter list, flash service custom process parameters, and firmware block information parameter table;

[0014] Calling a routine service to flash the firmware file to be analyzed according to the preset firmware full path parameter list and the flash service custom process parameters;

[0015] The data block, key metadata block, signature block and corresponding attribute start byte in the firmware file to be analyzed are identified according to the firmware block information parameter table.

[0016] As an optional solution, in the above-mentioned firmware automatic testing method based on fuzzy input, the generating fuzzy data based on a preset fuzzy input algorithm and a preset random value includes:

[0017] Receiving the preset random value input manually;

[0018] Or generate the preset random value according to an autonomous algorithm;

[0019] Or generate the preset random value according to a standard function library algorithm;

[0020] Fuzzy data is obtained according to the preset random value and the preset fuzzy input algorithm.

[0021] As an optional solution, in the above-mentioned firmware automated testing method based on fuzzy input, the preset tampering block types include: data block, key metadata block, signature block;

[0022] The preset tampering area includes: ID area, length area, firmware ID area, firmware version area, timestamp area, and MAC area;

[0023] The preset tampering starting position is the starting tampering byte number of the preset tampering area;

[0024] The preset fuzzy input modes include: fuzzy data generation, random bit flipping, byte injection, and no input.

[0025] As an optional solution, in the above-mentioned firmware automatic testing method based on fuzzy input, when the preset tampering block type is a data block, the corresponding tampering method includes: deleting data, adding data, and constructing data errors;

[0026] When the preset tampering block type is a key metadata block, the corresponding tampering methods include: deleting data, adding data, constructing data errors, key metadata ID area and length area data errors, key metadata ID area and version area data errors;

[0027] When the preset tampering block type is a signature block, the corresponding tampering methods include: deleting data, adding data, constructing data errors, signature data length information errors, and signature data area data errors.

[0028] As an optional solution, in the above-mentioned firmware automated testing method based on fuzzy input, the step of constructing a parameter-driven firmware tampering test case according to the parameter structure list and generating a corresponding executable file includes:

[0029] Calling a preconfigured test definition to perform corresponding tampering on the firmware file to be tested according to each group of test items in the parameter structure list, and generating a corresponding tampered firmware test case;

[0030] An executable file is compiled and generated according to the firmware tampering test case.

[0031] As an optional solution, the above firmware automatic testing method based on fuzzy input further includes:

[0032] According to the test function type, pre-condition test cases, forward flash / skip file / skip service test cases, parameter error / block error test cases, error condition injection test cases, download stability test cases, and robustness test cases are generated.

[0033] In order to solve the above technical problems, the present application provides a firmware automatic testing device based on fuzzy input, comprising:

[0034] The acquisition module is used to acquire and parse the firmware file to be analyzed;

[0035] A fuzzy module, used for generating fuzzy data based on a preset fuzzy input algorithm and a preset random value;

[0036] A construction module is used to obtain multiple test items according to different permutations and combinations of various states in a preset tampering block type, a preset tampering area, a preset tampering starting position, a preset tampering method, a preset fuzzy input method, the fuzzy data, and a preset tampering length, and generate a parameter structure list;

[0037] A tampering module, used to construct a parameter-driven tampering firmware test case according to the parameter structure list and generate a corresponding executable file;

[0038] The test module is used to import the executable file into the automated test framework for execution and generate test results.

[0039] In order to solve the above technical problems, the present application provides a firmware automatic testing device based on fuzzy input, comprising:

[0040] Memory for storing computer programs;

[0041] The processor is used to implement the steps of the above-mentioned firmware automatic testing method based on fuzzy input when executing the computer program.

[0042] In order to solve the above technical problems, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned firmware automatic testing method based on fuzzy input are implemented.

[0043] The firmware automation testing method based on fuzzy input provided by the present application obtains and parses the firmware file to be analyzed; generates fuzzy data based on a preset fuzzy input algorithm and a preset random value; obtains multiple test items according to different permutations and combinations of various states in the preset tampering block type, preset tampering area, preset tampering starting position, preset tampering method, preset fuzzy input method, fuzzy data, and preset tampering length, and generates a parameter structure list; constructs a parameter-driven tampering firmware test case according to the parameter structure list and generates a corresponding executable file; imports the executable file into the automated testing framework for execution, and generates test results. The present application uses fuzzy input to achieve the part to be modified in the tampering firmware, which can better simulate the unpredictable tampering behavior of the attack, and generates test cases by creating multiple sets of parameters of attribute permutations and combinations, which are used to test different tampering methods and degrees. A large number of test cases can be generated and executed without human intervention, thereby significantly improving the test efficiency, covering multiple potential vulnerabilities and weaknesses in the firmware, and improving the comprehensiveness of the test.

[0044] In addition, the present application also provides a device and a medium, which correspond to the above-mentioned firmware automatic testing method based on fuzzy input, and have the same effect as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 A flowchart of another firmware automated testing method based on fuzzy input provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of a firmware tampering test provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of firmware tampering test parameters provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of a category of test cases provided in an embodiment of the present application;

[0050] Figure 5 A structural diagram of a firmware automatic testing device based on fuzzy input provided in an embodiment of the present application;

[0051] Figure 6 A structural diagram of another firmware automation testing device based on fuzzy input provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] The core of this application is to provide a firmware automation testing method, device and medium based on fuzzy input.

[0054] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0055] The Secure Flash automated test solution is a type of automated test technology, which is specifically used in the test field of Secure Flash storage technology or related secure storage solutions. For the firmware tampering part of the Secure Flash automated test solution, the traditional solution: ① For the analysis of firmware of different formats, it is processed separately, and it cannot automatically parse and process firmware of multiple formats; ② When it is necessary to test the tampered firmware case, human intervention is required. The tester opens the file to manually modify the firmware and inputs the faulty firmware file into the test system for testing. ③ Manually modify the byte data of a certain part of the firmware to a fixed value. If you want to change the parameters during the test, you need to re-create a faulty firmware file. Even if different values ​​are used for each test, these values ​​are simple and cannot simulate unpredictable tampering behaviors. ④ The tampering location is single and cannot be overwritten. The tampering length is not flexible. ⑤ When the sample and firmware structure of the test are different, manual tampering is required again.

[0056] The automatic processing of firmware is not complete, and different project samples need to be re-adapted. Human intervention is required to manually create errors. Ordinary firmware tampering is often based on known vulnerabilities or specific attack paths, and lacks comprehensive coverage of unknown vulnerabilities and potential security risks. This method may not be able to discover hidden and undiscovered weaknesses in the firmware. Ordinary firmware tampering usually requires manual analysis of the firmware, locating the specific tampering point, and manually modifying it. This process is time-consuming and error-prone, especially when facing large or complex firmware. If you want to construct multiple error conditions, you need to create multiple files. Attackers often use automated and randomized means to tamper with the firmware to bypass the security detection mechanism. Ordinary firmware tampering methods are difficult to simulate this attack mode, so it is impossible to fully evaluate the security of the firmware in real attack scenarios. Ordinary firmware tampering methods are usually limited to specific tampering methods and content, and cannot flexibly respond to different attack strategies and scenarios. This limits the effectiveness and comprehensiveness of the test. The data may be single and simple, not covering all possibilities, and depends on the personal level of the tester. This application is not limited to what specific fields the firmware in the firmware test is used for.

[0057] In order to solve the above problems, the present application provides a firmware automation testing method based on fuzzy input, such as Figure 1 As shown, including:

[0058] S11: Obtain and parse the firmware file to be analyzed;

[0059] S12: Generate fuzzy data based on a preset fuzzy input algorithm and a preset random value;

[0060] S13: obtaining multiple test items according to different permutations and combinations of various states of a preset tampered block type, a preset tampered area, a preset tampered starting position, a preset tampered mode, a preset fuzzy input mode, fuzzy data, and a preset tampered length, and generating a parameter structure list;

[0061] S14: construct a parameter-driven firmware tampering test case according to the parameter structure list and generate a corresponding executable file;

[0062] S15: Import the executable file into the automated testing framework for execution and generate test results.

[0063] Step S11 involves obtaining the firmware file from a predetermined storage location, which can be achieved through an automated script or a dedicated firmware parsing tool to identify its structure and content. This includes performing targeted parsing based on the format of the firmware file, extracting key information such as firmware version, firmware size, firmware ID number, etc., reading the firmware file, analyzing the file header information, and identifying different blocks in the file, such as the data area, key metadata area, and signature area.

[0064] By flashing, you can identify whether the firmware has been updated. If so, you only need to modify the corresponding configuration without modifying the code.

[0065] Step S12 uses a predefined fuzzy input algorithm and random value generation technology to create fuzzy data for testing. By generating a large amount of random, abnormal or invalid data to test the firmware, a wider input space can be covered, making it more likely to discover hidden and unnoticed vulnerabilities in the firmware. Compared with traditional testing methods, fuzzy input is better at exploring the boundary conditions and abnormal situations of the firmware, which helps to discover deep-level vulnerabilities that are difficult to reveal through conventional testing methods.

[0066] In this embodiment, the Secure Flash automatic testing solution is based on vTESTStudio (a test case writing software). Users create test cases in the software and add and configure test steps. Users can adapt and accumulate the reuse of use cases by updating parameter sets. The test cases edited in vTESTstudio need to be compiled to generate executable files and then loaded into the automated testing framework for execution.

[0067] Step S13 obtains multiple test items according to different permutations and combinations of various states in the preset tampering block type, preset tampering area, preset tampering starting position, preset tampering method, preset fuzzy input method, fuzzy data, and preset tampering length, and generates a parameter structure list. These parameters define the target area, position, method and data of the tampering, and are used to determine the specific tampering position and how to tamper. Each combination represents a tampering method. Different attribute permutations and combinations are usually performed automatically, and a large number of test cases can be generated and executed without human intervention, thereby significantly improving the test efficiency.

[0068] Step S14 constructs a parameter-driven tampered firmware test case based on the parameter structure list and generates a corresponding executable file. The "parameter structure list" refers to a set of all test parameters that will be used to construct the test case. The test case can include different tampering scenarios and input data to simulate different attacks and failure modes. By constructing parameter-driven test cases and generating executable files, the test process can be automated and standardized, and the test efficiency and repeatability can be improved.

[0069] Step S15 imports the executable file into the automated test framework (CANoe) for execution and generates test results. After completing the automated test, the user can generate a detailed test report in CANoe to facilitate problem location and repair.

[0070] Through the firmware automation testing method based on fuzzy input provided by the embodiment of the present application, the firmware file to be analyzed is obtained and parsed; fuzzy data is generated based on a preset fuzzy input algorithm and a preset random value; multiple test items are obtained according to different permutations and combinations of various states in the preset tampering block type, preset tampering area, preset tampering starting position, preset tampering method, preset fuzzy input method, fuzzy data, and preset tampering length, and a parameter structure list is generated; a parameter-driven tampering firmware test case is constructed according to the parameter structure list and a corresponding executable file is generated; the executable file is imported into the automated testing framework for execution to generate a test result. This application uses fuzzy input to achieve the part to be modified in the tampering firmware, which can better simulate the unpredictable tampering behavior of the attack, and generates test cases by creating multiple groups of parameters of attribute permutations and combinations for testing different tampering methods and degrees. A large number of test cases can be generated and executed without human intervention, thereby significantly improving the test efficiency, covering multiple potential vulnerabilities and weaknesses in the firmware, and improving the comprehensiveness of the test.

[0071] Further, according to the above embodiment, in a specific embodiment, obtaining and parsing the firmware file to be analyzed includes:

[0072] Configure the preset firmware full path parameter list, flash service custom process parameters, and firmware block information parameter table;

[0073] Call the routine service to flash the firmware file to be analyzed according to the preset firmware full path parameter list and the flash service custom process parameters;

[0074] According to the firmware block information parameter table, the data block, key metadata block, signature block and corresponding attribute start byte in the firmware file to be analyzed are identified.

[0075] The preset firmware full path parameter list refers to a parameter list containing the detailed path of the firmware file storage location. This list enables the automated test system to accurately locate and access the firmware file to be tested.

[0076] Flashing service custom process parameters, this set of parameters defines the specific behavior in the firmware flashing process, including flashing commands, address range, data block size, etc., to adapt to different test requirements and firmware characteristics. In the specific operation process, you can choose which routine service to use according to actual needs, and make corresponding service settings. For example: automatically fill in the parameters to be used by the service, such as the service erase address and size of the 31 routine service, and automatically flash according to the file order and the order of the file blocks in it.

[0077] Call a specific routine service to perform the firmware flash operation. This service operates according to the preset firmware full path parameter list and the flash service custom process parameters.

[0078] The firmware block information parameter table can be used to identify different functional blocks in the firmware file, including data blocks, key metadata blocks and signature blocks.

[0079] Through the firmware block information parameter table, the system can not only identify these blocks, but also determine the properties of these blocks, such as the starting byte position of the ID area byte, length area byte, version area byte, Media Access Control Address (MAC) area byte, etc. After determining the starting byte, it is convenient to accurately find the modification position. For example, a test case is to tamper with the 2nd byte to the 8th byte of the ID area byte of the key metadata block.

[0080] By accurately configuring and identifying each part of the firmware file, the accuracy and reliability of the test are improved. The automated firmware flashing and parsing process reduces the need for manual operations and improves the efficiency and repeatability of the test. By identifying all key parts of the firmware, it ensures that the test can cover all important features of the firmware and improves the comprehensiveness of the test.

[0081] Further, according to the above embodiment, in a specific embodiment, generating fuzzy data based on a preset fuzzy input algorithm and a preset random value includes:

[0082] Receive a preset random value input manually;

[0083] Or generate a preset random value based on an autonomous algorithm;

[0084] Or generate preset random values ​​according to the standard function library algorithm;

[0085] Fuzzy data is obtained according to a preset random value and a preset fuzzy input algorithm.

[0086] In this embodiment, there are three sources of random values ​​for fuzzy input:

[0087] 1. Manual input, including: input by testers, or directly filling in values ​​in the parameter table according to customer requirements;

[0088] 2. Automatically generate random values ​​using autonomous algorithms: ① Based on the mutation strategy, use the General Purpose Timer (GPTTimer) as the entropy for generating random values ​​to perform a series of different operations to obtain the final random value, which is used to randomly replace the tampered data; ② Based on the rule strategy, use random bit flipping, format destruction and other methods to tamper with the data;

[0089] 3. Use the algorithm of the standard function library: ① Random number generation (Random) algorithm based on object-oriented high-level programming language (C#); ② Use random value generator (Vector CANoe).

[0090] The preset random value and the preset fuzzy input algorithm are combined to generate fuzzy data for testing. Specifically, the preset fuzzy algorithm can be a custom algorithm, or the random value can be used directly for tampering without being processed by the preset fuzzy algorithm.

[0091] By providing multiple methods for generating random values, the flexibility and diversity of test data are improved, and more attack and failure scenarios can be simulated. Automated random value generation reduces the need for manual operations and improves the efficiency and repeatability of testing. By generating fuzzy data, the security of firmware can be tested more comprehensively, revealing potential security vulnerabilities.

[0092] Further, according to the above embodiment, in a specific embodiment, the preset tampering block types include: data block, key metadata block, signature block;

[0093] The preset tampering areas include: ID area, length area, firmware ID area, firmware version area, timestamp area, and MAC area;

[0094] The preset tampering start position is the starting tampering byte number of the preset tampering area;

[0095] The preset fuzzy input methods include: fuzzy data generation, random bit flipping, byte injection, and no input.

[0096] The embodiment of the present application provides a specific tampering implementation method, and the preset tampering block types include: data block, key metadata block, and signature block.

[0097] Data block: refers to the part of the firmware that contains the actual functional data, which is a collection of codes or instructions required for the device to run.

[0098] Key metadata block: Contains information describing the data attributes and configuration, such as the source of the data, creation time, size, etc.

[0099] Signature Block: A digital signature used to verify the integrity and origin of the firmware, ensuring that the firmware has not been tampered with.

[0100] This embodiment mainly tampering with the above three types of data.

[0101] The preset tampering areas include: ID area: unique identifier of the firmware or data block. Length area: specifies the size of the firmware or data block. Firmware ID area: identifies the identity of the firmware. Firmware version area: identifies the version number of the firmware. Timestamp area: records the time when the firmware is created or modified. MAC area: contains the physical address or media access control address of the firmware.

[0102] According to the specific area included in the tampering block type, targeted tampering is performed. For example, the key metadata block includes the ID area; the signature block includes the length area. The corresponding tampering area is set according to its specific data type.

[0103] The preset tampering start position refers to the byte position where tampering starts in the firmware, which is calculated based on the starting point of the preset tampering area, for example, starting tampering at the third byte of the ID area of ​​the key metadata block.

[0104] The preset fuzzy input methods include: Fuzzy data generation: refers to the process of inputting pre-generated fuzzy data. Random bit flipping: simulates tampering by randomly changing bits in the firmware. Byte injection: inserts additional bytes at specific locations in the firmware to test the firmware's ability to handle abnormal data. No input: does not perform additional input, and tests the firmware's tampering behavior when data is missing (some bytes are deleted).

[0105] By covering different tamper block types and areas, the test can fully evaluate the security and stability of the firmware. The preset tampering start position ensures that the tampering can be accurately located in the key parts of the firmware, increasing the pertinence of the test. The preset fuzzy input method provides a variety of means to simulate attacks and faults, enhancing the diversity and effectiveness of the test.

[0106] Further, when the preset tampering block type is a data block, the corresponding tampering methods include: deleting data, adding data, and constructing data errors;

[0107] When the preset tampering block type is a key metadata block, the corresponding tampering methods include: deleting data, adding data, constructing data errors, key metadata ID area and length area data errors, key metadata ID area and version area data errors;

[0108] When the preset tampering block type is a signature block, the corresponding tampering methods include: deleting data, adding data, constructing data errors, signature data length information errors, and signature data area data errors.

[0109] The ways to tamper with a data block include: Deleting data: removing some or all of the data from a data block to simulate data loss. Adding data: inserting extra data into a data block to test the firmware's ability to handle extra data. Constructing data errors: intentionally introducing erroneous data, such as illegal values ​​or malformed data.

[0110] The tampering methods of the key metadata block include: Deleting data: removing key metadata. Adding data: adding additional metadata to the key metadata block. Constructing data errors: introducing metadata with incorrect format or value. Data errors in the key metadata ID area and length area: modifying the key ID or length information in the metadata. Data errors in the key metadata ID area and version area: changing the firmware ID or version information in the metadata.

[0111] The tampering methods of the signature block include: Deleting data: removing the signature data. Adding data: adding extra data to the signature block. Constructing data errors: introducing incorrect signature data. Signature data length information errors: changing the length information of the signature data. Signature data area data errors: modifying the signature data itself.

[0112] By using different tampering methods for different types of firmware blocks, it is ensured that the test can fully cover the key parts of the firmware. According to the characteristics of different firmware blocks, the corresponding tampering methods are selected to improve the pertinence and effectiveness of the test. By simulating various possible attack and failure scenarios, the security and stability test of the firmware is enhanced.

[0113] Table 1 is a parameter structure list of a firmware tampering test provided by this application. As shown in Table 1, each test item (TestCase) corresponds to multiple setting condition states.

[0114] Table 1 Parameter structure list of firmware tampering test

[0115]

[0116] Figure 2 A schematic diagram of a firmware tampering test provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, for the three types of data blocks, different tampering methods are selected, and the data of the tampering area of ​​the data block is determined in advance, and the injection position and data length are determined to facilitate specific and accurate tampering.

[0117] Figure 3 A schematic diagram of a firmware tampering test parameter provided in an embodiment of the present application, according to Figure 3 As shown, determine the tampered block type, tampered area, tampered starting position, tampering method, fuzzy input method, fuzzy data, tampered length, and the corresponding specific generation method or specific type.

[0118] Further, according to the above embodiment, in a specific embodiment, constructing a parameter-driven tampering firmware test case according to the parameter structure list and generating a corresponding executable file includes:

[0119] Call the preconfigured test definition to perform corresponding tampering on the firmware file to be tested according to each group of test items in the parameter structure list, and generate corresponding tampered firmware test cases;

[0120] Compile and generate executable files according to the tampered firmware test cases.

[0121] Pre-configure the test definition (Test Definition) during the development phase. This is usually achieved in vTESTstudio. By configuring its properties, including adding the necessary parameters. These parameters should correspond to the parameters in the parameter structure list created previously. When the tester uses it, there is no need to repeat the configuration.

[0122] When configuring the Test Definition, the user also needs to specify the commands or logic to be executed in the test case. These commands may include sending specific messages, verifying message content, waiting for a specific time, etc. vTESTstudio provides some basic methods and commands that can be used. For special functions that require additional coding implementation, some function libraries can be developed (function libraries are often .can or .cin files) and called. The Secure Flash automated test solution can use a custom written function library to call the basic library file, including firmware parsing, diagnostic message sending, fault injection functions, etc.

[0123] With the parameter structure list and the configured test definition, you can generate test cases by calling the test definition. This usually involves referencing the test definition through specific syntax or interface operations in a certain editing interface of vTESTstudio (such as Test Table or Test Sequence Diagram) and passing in the corresponding parameter values.

[0124] In vTESTstudio, this may involve adding one or more rows to the Test Table and specifying the name and parameter values ​​of the Test Definition in those rows; or in the Test Sequence Diagram, referencing the Test Definition by adding specific nodes and connections and configuring parameter values ​​in the nodes.

[0125] After completing the generation of TestCaseList, users need to compile these test cases into executable files, such as .vtuexe files (a test case file generated by vTESTstudio software), and then load them into the CANoe environment for execution. In CANoe, users can configure and execute these test cases through Test Configuration (a CANoe test window), and view test results and reports. Based on the vTESTstudio software mechanism, when there is one more set of TestCase data in the parameter structure list, one more set of test cases will be automatically generated.

[0126] In the Secure Flash automatic test, the firmware tampering test is only one of the items. In a specific embodiment, it also includes:

[0127] According to the test function type, pre-condition test cases, forward flash / skip file / skip service test cases, parameter error / block error test cases, error condition injection test cases, download stability test cases, and robustness test cases are generated.

[0128] Figure 4 A schematic diagram of a test case category provided in an embodiment of the present application, such as Figure 4 As shown in the figure, the test cases implemented by all the solutions are grouped according to their functions and divided into seven test case groups (Test Case Group), namely, pre-condition test cases, forward flash / skip file / skip service test cases, parameter error / block error test cases, error condition injection test cases, download stability test cases, robustness test cases, and firmware tampering test cases; each test case group corresponds to a parameter structure list (Struct List), and the test items (TestCase) in each test case group correspond to an item in the test group, that is, a Struct.

[0129] Table 2 is a parameter structure list of a precondition test provided in an embodiment of the present application. As shown in Table 2, according to specific test requirements, the corresponding parameters are configured to generate a parameter structure list, and further generate corresponding test cases to perform firmware testing.

[0130] Table 2 Parameter structure list of precondition test

[0131]

[0132] In the above embodiments, the firmware automation test method based on fuzzy input is described in detail, and the present application also provides an embodiment corresponding to the firmware automation test device based on fuzzy input. It should be noted that the present application describes the embodiments of the device part from two perspectives, one is based on the functional module perspective, and the other is based on the hardware perspective.

[0133] From the perspective of functional modules, Figure 5 A structural diagram of a firmware automatic testing device based on fuzzy input provided in an embodiment of the present application, such as Figure 5 As shown, a firmware automatic testing device based on fuzzy input includes:

[0134] An acquisition module 21 is used to acquire and parse the firmware file to be analyzed;

[0135] A fuzzy module 22, used to generate fuzzy data based on a preset fuzzy input algorithm and a preset random value;

[0136] A construction module 23 is used to obtain multiple test items according to different permutations and combinations of various states of a preset tampering block type, a preset tampering area, a preset tampering starting position, a preset tampering method, a preset fuzzy input method, fuzzy data, and a preset tampering length, and generate a parameter structure list;

[0137] The tampering module 24 is used to construct a parameter-driven tampering firmware test case according to the parameter structure list and generate a corresponding executable file;

[0138] The test module 25 is used to import the executable file into the automated test framework for execution and generate test results.

[0139] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.

[0140] Figure 6 A structural diagram of another firmware automatic testing device based on fuzzy input provided in an embodiment of the present application, such as Figure 6 As shown, the firmware automatic testing device based on fuzzy input includes: a memory 30 for storing a computer program;

[0141] The processor 31 is used to implement the steps of the method for obtaining user operation habit information in the above embodiment (automatic firmware testing method based on fuzzy input) when executing a computer program.

[0142] The firmware automatic testing device based on fuzzy input provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0143] Among them, the processor 31 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 31 may be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 31 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 31 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 31 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0144] The memory 30 may include one or more computer-readable storage media, which may be non-transitory. The memory 30 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 30 is at least used to store the following computer program 301, wherein, after the computer program is loaded and executed by the processor 31, it can implement the relevant steps of the firmware automation testing method based on fuzzy input disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 30 may also include an operating system 302 and data 303, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 302 may include Windows, Linux, etc. Data 303 may include but is not limited to data involved in implementing the firmware automation testing method based on fuzzy input, etc.

[0145] In some embodiments, the firmware automatic testing device based on fuzzy input may further include a display screen 32 , an input and output interface 33 , a communication interface 34 , a power supply 35 , and a communication bus 36 .

[0146] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the firmware automatic testing device based on fuzzy input, and may include more or less components than those shown in the figure.

[0147] The firmware automatic testing device based on fuzzy input provided in the embodiment of the present application includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the following method: a firmware automatic testing method based on fuzzy input.

[0148] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps described in the embodiment of the firmware automatic testing method based on fuzzy input are implemented.

[0149] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0150] The computer-readable storage medium provided in this embodiment stores a computer program, and when a processor executes the program, the following method can be implemented: a firmware automatic testing method based on fuzzy input.

[0151] The above is a detailed introduction to the firmware automated testing method, device and medium based on fuzzy input provided by the present application. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

[0152] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A firmware automated testing method based on fuzzy input, characterized in that: include: Obtain and parse the firmware file to be analyzed; Generate fuzzy data based on a preset fuzzy input algorithm and a preset random value; According to different permutations and combinations of various states in the preset tampering block type, the preset tampering area, the preset tampering starting position, the preset tampering method, the preset fuzzy input method, the fuzzy data, and the preset tampering length, a plurality of test items are obtained, and a parameter structure list is generated; Constructing a parameter-driven firmware tampering test case according to the parameter structure list and generating a corresponding executable file; The executable file is imported into the automated testing framework for execution to generate test results.

2. The firmware automatic testing method based on fuzzy input according to claim 1 is characterized in that: The obtaining and parsing of the firmware file to be analyzed includes: Configure the preset firmware full path parameter list, flash service custom process parameters, and firmware block information parameter table; Calling a routine service to flash the firmware file to be analyzed according to the preset firmware full path parameter list and the flash service custom process parameters; The data block, key metadata block, signature block and corresponding attribute start byte in the firmware file to be analyzed are identified according to the firmware block information parameter table.

3. The firmware automatic testing method based on fuzzy input according to claim 1 is characterized in that: The generating of fuzzy data based on a preset fuzzy input algorithm and a preset random value includes: Receiving the preset random value input manually; Or generate the preset random value according to an autonomous algorithm; Or generate the preset random value according to a standard function library algorithm; Fuzzy data is obtained according to the preset random value and the preset fuzzy input algorithm.

4. The firmware automatic testing method based on fuzzy input according to claim 3 is characterized in that: The preset tampering block types include: data block, key metadata block, signature block; The preset tampering area includes: ID area, length area, firmware ID area, firmware version area, timestamp area, and MAC area; The preset tampering starting position is the starting tampering byte number of the preset tampering area; The preset fuzzy input modes include: fuzzy data generation, random bit flipping, byte injection, and no input.

5. The firmware automatic testing method based on fuzzy input according to claim 3 is characterized in that: When the preset tampering block type is a data block, the corresponding tampering methods include: deleting data, adding data, and constructing data errors; When the preset tampering block type is a key metadata block, the corresponding tampering methods include: deleting data, adding data, constructing data errors, key metadata ID area and length area data errors, key metadata ID area and version area data errors; When the preset tampering block type is a signature block, the corresponding tampering methods include: deleting data, adding data, constructing data errors, signature data length information errors, and signature data area data errors.

6. The firmware automatic testing method based on fuzzy input according to claim 5 is characterized in that: The step of constructing a parameter-driven firmware tampering test case according to the parameter structure list and generating a corresponding executable file includes: Calling a preconfigured test definition to perform corresponding tampering on the firmware file to be tested according to each group of test items in the parameter structure list, and generating a corresponding tampered firmware test case; An executable file is compiled and generated according to the firmware tampering test case.

7. The firmware automatic testing method based on fuzzy input according to claim 5 is characterized in that: Also includes: According to the test function type, pre-condition test cases, forward flash / skip file / skip service test cases, parameter error / block error test cases, error condition injection test cases, download stability test cases, and robustness test cases are generated.

8. A firmware automatic testing device based on fuzzy input, characterized in that: include: The acquisition module is used to acquire and parse the firmware file to be analyzed; A fuzzy module, used for generating fuzzy data based on a preset fuzzy input algorithm and a preset random value; A construction module is used to obtain multiple test items according to different permutations and combinations of various states in a preset tampering block type, a preset tampering area, a preset tampering starting position, a preset tampering method, a preset fuzzy input method, the fuzzy data, and a preset tampering length, and generate a parameter structure list; A tampering module, used to construct a parameter-driven tampering firmware test case according to the parameter structure list and generate a corresponding executable file; The test module is used to import the executable file into the automated test framework for execution and generate test results.

9. A firmware automatic testing device based on fuzzy input, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of the firmware automatic testing method based on fuzzy input as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the firmware automatic testing method based on fuzzy input are implemented as described in any one of claims 1 to 7.

Images