Test case automatic generation method and electronic device

Abstract

The embodiment of the application discloses a test case automatic generation method and an electronic device, wherein the test case automatic generation method comprises the following steps: the electronic device determines a test tool of a to-be-tested source code according to a test configuration file; the electronic device determines an input parameter type and an input rule based on the test tool; the electronic device extracts parameters from the to-be-tested source code according to the input parameter type, and obtains a target parameter file; and the electronic device generates a test case of the to-be-tested source code according to the input rule and the target parameter file. The method of the embodiment of the application realizes the automatic generation of the test case under the Fuzzing test, improves the test efficiency, and reduces the dependence on professionals.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software analysis testing, in particular to a test case automatic generation method and an electronic device. BACKGROUND

[0002] Fuzzing testing, also known as Fuzzing testing or robustness testing, is a kind of security testing, which mainly refers to the testing of discovering potential security defects of a system, and can also be called the discovery of vulnerabilities. At present, the traditional C / C++ white-box Fuzzing testing tool needs a white-box tester to determine the core function code of the target program based on the understanding of the source code to be tested, and then test the core function code of the target program by manually writing test cases.

[0003] However, the above-mentioned method needs a lot of manual intervention, and the tester needs to spend a lot of time and effort to analyze the source code and manually write test cases, which is not suitable for testing a large number of different programs to be tested. At the same time, the above-mentioned method has high requirements for the professional and technical ability of the tester, which greatly affects the correctness of the test results. SUMMARY

[0004] The embodiments of the present application provide a test case automatic generation method and an electronic device, which can realize the automatic generation of test cases under Fuzzing testing, improve the test efficiency, and reduce the dependence on professionals.

[0005] The embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, the embodiments of the present application provide a test case automatic generation method, comprising:

[0007] The electronic device determines the test tool of the source code to be tested according to the test configuration file;

[0008] The electronic device determines the input parameter type and the input rule based on the test tool;

[0009] The electronic device extracts parameters from the source code to be tested according to the input parameter type, and obtains a target parameter file;

[0010] The electronic device generates test cases for the source code to be tested according to the input rule and the target parameter file.

[0011] It can be seen that in the embodiment, the automatic generation of test cases is realized without a large amount of manual intervention, the overall test efficiency is improved, and the test scenario for a large number of different programs to be tested can be applied. At the same time, according to the input rule of the test tool, the generated test case is not affected by the professional and technical ability of the tester, and can perfectly meet the needs of the test tool to ensure the objectivity and correctness of the test result.

[0012] In a possible implementation, when the test tool is AFL && AFL++ or LibFuzzer, the electronic device extracts parameters from the source code to be tested according to the input parameter type to obtain a target parameter file, including:

[0013] The electronic device obtains the header file of the source code to be tested.

[0014] The electronic device scans the header file according to the input parameter type, and extracts at least one first function entry in the header file.

[0015] The electronic device parses the header file according to the at least one first function entry, and determines the function input parameter type of each first function entry in the at least one first function entry.

[0016] The electronic device integrates the at least one first function entry and the function input parameter type of each first function entry to obtain the target parameter file.

[0017] In a possible implementation, the electronic device scans the header file according to the input parameter type, and extracts at least one first function entry in the header file, including:

[0018] The electronic device converts the header file into an abstract syntax tree, traverses the abstract syntax tree, extracts a node whose node type is a function, and obtains at least one first function entry.

[0019] In a possible implementation, before the electronic device integrates the at least one first function entry and the function input parameter type of each first function entry to obtain the target parameter file, the method further includes:

[0020] The electronic device determines a second function entry in the at least one first function entry, wherein the second function entry is a function input parameter type that is not recorded in the header file in the at least one first function entry.

[0021] The electronic device retrieves the source code to be tested according to the second function entry to obtain a first structure corresponding to the second function entry, wherein the first structure is used to define the second function entry.

[0022] The electronic device parses the first structure to determine the function input parameter type of the second function entry.

[0023] In a possible implementation, the electronic device acquires a header file of the source code to be tested, including:

[0024] The electronic device determines a header file path of the source code to be tested according to the configuration file;

[0025] The electronic device acquires the header file of the source code to be tested according to the header file path.

[0026] In a possible implementation, when the test tool is Syzkaller, the electronic device performs parameter extraction on the source code to be tested according to an input parameter type, to obtain a target parameter file, including:

[0027] The electronic device converts the source code to be tested into an intermediate language, where the intermediate language is used to convert an expression format of a structure in the source code to be tested into a same format;

[0028] The electronic device searches the intermediate language by using a preset first string, to determine device node information;

[0029] The electronic device globally searches the intermediate language, to determine at least one command code;

[0030] The electronic device performs backtracking analysis on each command code in the at least one command code, to determine a command code input parameter type corresponding to each command code;

[0031] The electronic device integrates the device node information, the at least one command code, and the command code input parameter type corresponding to each command code, to obtain the target parameter file.

[0032] In a possible implementation, the electronic device globally searches the intermediate language, to determine at least one command code, including:

[0033] The electronic device globally searches the intermediate language, to determine at least one second structure, where each second structure in the at least one second structure is used to register an input / output control instruction;

[0034] The electronic device performs information integration on the at least one second structure, to obtain at least one structure name;

[0035] The electronic device searches the intermediate language by using the at least one structure name, to obtain at least one third structure, where the at least one structure name includes a structure name of any one third structure in the at least one third structure;

[0036] The electronic device searches the at least one second structure and the at least one third structure by using a preset second string, to obtain at least one target code;

[0037] The electronic device searches at least one target code by using a preset third string to obtain at least one command code, wherein the at least one command code corresponds to the at least one target code one by one.

[0038] In a possible implementation, the electronic device generates a test case of the source code to be tested according to the input rule and the target parameter file, including:

[0039] The electronic device determines the input syntax and the input structure of the test tool according to the input rule.

[0040] The electronic device edits the parameters in the target parameter file according to the input syntax and the input structure to obtain the test case.

[0041] In a second aspect, the embodiments of the present application provide a test case automatic generation device, including:

[0042] The source code analysis module is configured to determine the test tool of the source code to be tested according to the test configuration file, and determine the input parameter type and the input rule based on the test tool.

[0043] The information integration module is configured to extract parameters from the source code to be tested according to the input parameter type to obtain a target parameter file.

[0044] The test case generation module is configured to generate a test case of the source code to be tested according to the input rule and the target parameter file.

[0045] In a possible implementation, when the test tool is AFL&&AFL++ or LibFuzzer, in the aspect of extracting parameters from the source code to be tested according to the input parameter type to obtain a target parameter file, the information integration module is specifically configured to:

[0046] Obtain a header file of the source code to be tested.

[0047] Scan the header file according to the input parameter type to extract at least one first function entry in the header file.

[0048] Parse the header file according to the at least one first function entry to determine the function input parameter type of each first function entry in the at least one first function entry.

[0049] Integrate the at least one first function entry and the function input parameter type of each first function entry to obtain the target parameter file.

[0050] In a possible implementation, in the aspect of scanning the header file according to the input parameter type to extract at least one first function entry in the header file, the information integration module is specifically configured to:

[0051] The header file is converted into an abstract syntax tree, the abstract syntax tree is traversed, a node type of a function is extracted from the node, and at least one first function entry is obtained.

[0052] In a possible implementation, before integrating the at least one first function entry and the function input parameter type of each first function entry to obtain the target parameter file, the information integration module is further configured to:

[0053] determine a second function entry in the at least one first function entry, wherein the second function entry is a function input parameter type that is not recorded in the header file in the at least one first function entry;

[0054] retrieve the to-be-tested source code according to the second function entry, and obtain a first structure corresponding to the second function entry, wherein the first structure is used to define the second function entry;

[0055] parse the first structure, and determine the function input parameter type of the second function entry.

[0056] In a possible implementation, in the aspect of obtaining the header of the to-be-tested source code, the information integration module is specifically configured to:

[0057] determine a header file path of the to-be-tested source code according to the configuration file;

[0058] obtain the header file of the to-be-tested source code according to the header file path.

[0059] In a possible implementation, when the test tool is Syzkaller, in the aspect of performing parameter extraction on the to-be-tested source code according to the input parameter type to obtain the target parameter file, the information integration module is specifically configured to:

[0060] convert the to-be-tested source code into an intermediate language, wherein the intermediate language is used to convert an expression format of a structure in the to-be-tested source code into a same format;

[0061] search the intermediate language by using a preset first string, and determine device node information;

[0062] globally search the intermediate language, and determine at least one command code;

[0063] perform backtracking analysis on each command code in the at least one command code, and determine a command code input parameter type corresponding to each command code;

[0064] integrate the device node information, the at least one command code, and the command code input parameter type corresponding to each command code to obtain the target parameter file.

[0065] In a possible implementation, in the aspect of globally searching the intermediate language and determining the at least one command code, the information integration module is specifically configured to:

[0066] searching the intermediate language globally to determine at least one second structure, wherein each of the at least one second structure is used to register an input-output control instruction;

[0067] integrating information of the at least one second structure to obtain at least one structure name;

[0068] searching the intermediate language by using the at least one structure name to obtain at least one third structure, wherein the at least one structure name comprises a structure name of any one of the at least one third structure;

[0069] searching the at least one second structure and the at least one third structure by using a preset second string to obtain at least one target code;

[0070] searching the at least one target code by using a preset third string to obtain at least one command code, wherein the at least one command code corresponds to the at least one target code one by one.

[0071] In a possible implementation, in terms of generating a test case of a to-be-tested source code according to an input rule and a target parameter file, the case generation module is specifically configured to:

[0072] determining an input syntax and an input structure of a test tool according to the input rule;

[0073] editing parameters in the target parameter file according to the input syntax and the input structure to obtain the test case.

[0074] In a third aspect, an electronic device is provided, which includes a processor and a memory. The memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory, so that the electronic device executes the method of the first aspect.

[0075] In a possible implementation, the electronic device is a server.

[0076] It can be seen that the second aspect and the third aspect are devices corresponding to the method of the first aspect. The description of the steps, the explanation of the terms, the various implementations, and the beneficial effects of the second aspect and the third aspect are also applicable to the foregoing description, and can be referred to in the first aspect. Therefore, the foregoing description is not repeated here.

[0077] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program. The computer program causes a computer to execute the method according to the first aspect. The descriptions of the steps, the explanations of the terms, the various implementation manners and the beneficial effects in the fourth aspect are also applicable to the foregoing descriptions, and thus will not be repeated here.

[0078] In a fifth aspect, the embodiments of the present application provide a computer program product, which comprises a non-transitory computer readable storage medium storing a computer program. The computer program is operable to cause a computer to execute any possible implementation manner of the method according to the first aspect. The descriptions of the steps, the explanations of the terms, the various implementation manners and the beneficial effects in the fifth aspect are also applicable to the foregoing descriptions, and thus will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0079] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some of the embodiments of the present application.

[0080] Figure 1 A flowchart of a test case automatic generation method provided by the embodiments of the present application;

[0081] Figure 2 A schematic diagram of a preset format of a target parameter file provided by the embodiments of the present application;

[0082] Figure 3 A schematic diagram of a target parameter file provided by the embodiments of the present application;

[0083] Figure 4 A schematic diagram of a unified format of a structure provided by the embodiments of the present application;

[0084] Figure 5 A flowchart of a method for determining at least one command code in a global search intermediate language provided by the embodiments of the present application;

[0085] Figure 6 A schematic diagram of another IOCTL embedded in a command code provided by the embodiments of the present application;

[0086] Figure 7 A schematic diagram of another preset format of a target parameter file provided by the embodiments of the present application;

[0087] Figure 8 A structural schematic diagram of a test case automatic generation device provided by the embodiments of the present application;

[0088] Figure 9 A structural schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0089] The technical solutions in the embodiments of the present application will be clearly and completely described in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0090] The terms "first", "second", "third", and "fourth" and the like in the specification of the present application and claims and drawings are used to distinguish different objects, rather than to describe a particular order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed or can optionally include other steps or units inherent to the process, method, product or device.

[0091] Reference to "embodiments" herein means that a particular feature, result or characteristic described in connection with an embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it mutually exclusive or alternative to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0092] First, in order to facilitate understanding, the related terms involved in the present application are explained:

[0093] AFL&&AFL++: American Fuzzy Lop, a coverage-guided fuzzing tool that can record the code coverage of input samples to adjust the input samples to improve coverage and increase the probability of finding vulnerabilities.

[0094] LibFuzzer: an in-process, coverage-guided, evolutionary fuzzing engine that is part of the Low Level Virtual Machine (LLVM) project.

[0095] Syzkaller: An unsupervised, coverage-guided fuzzer that supports: Akaros, FreeBSD, Fuchsia, gVisor, Linux, NetBSD, OpenBSD, Windows, and others.

[0096] Referring to Figure 1 , Figure 1 A flowchart of a test case automatic generation method provided by an embodiment of the present application is shown in FIG. 1. The method includes the following steps:

[0097] 101: The electronic device determines the test tool for the source code to be tested according to the test configuration file.

[0098] In the embodiment, before the test is performed, a test configuration file is configured for the source code to be tested. The test configuration file is used to record the test parameters for testing the source code to be tested, including the test type, the test tool, the test tool path, the source code path, the header file path of the source code to be tested, and the like. Specifically, in the test configuration file, each test parameter corresponds to a respective parameter name. Based on this, the test tool for the source code to be tested can be determined by querying the test configuration file through the parameter name corresponding to the test tool.

[0099] For example, the embodiment provides a configuration file, which is specifically as follows:

[0100]

[0101]

[0102] The meanings of the parameters are shown in Table 1:

[0103] Table 1:

[0104] Parameter name Explanation driver Driver class fuzzing test library Library class fuzzing test enable Whether to allow the class fuzzing test tool Fuzzing tool toolPath Tool path sourceCodePath Source code path includeFile To-be-tested source code header file path

[0105] Therefore, the test type of the source code to be tested can be determined by querying the parameter name “enable”, and then the parameter name “tool” is queried in the test type allowed, i.e., the value of “enable” is “True”, to determine the test tool of the source code to be tested. For the above example, both the driver class fuzzing test and the library class fuzzing test are allowed, and then two test cases, i.e., the driver class fuzzing test case and the library class fuzzing test case, can be generated. In the driver class fuzzing test, the test tool is Syzkaller; and in the library class fuzzing test, the test tool is AFL&&AFL++ or LibFuzzer.

[0106] 102: The electronic device determines the input parameter type and the input rule based on the test tool.

[0107] In the embodiment, the test tools can include AFL&&AFL++, LibFuzzer, and Syzkaller. For different test tools, due to the difference of their test objects, the input parameter types and input rules required are also different. Specifically, the test objects of AFL&&AFL++ and LibFuzzer are libraries, and test cases need to be implemented to call library interfaces according to the header files provided by the libraries, and then the test cases are compiled into executable binary files for testing. Among them, AFL&&AFL++ also needs to recompile the library to be tested using its own compiler afl-gcc / afc-g++ before testing, and when calling the test demo, an initial test seed input is provided according to the interface input parameters. The test object of Syzkaller is a driver interface in kernel code, and related information of the driver interface needs to be extracted and used to write test cases in syzlang syntax.

[0108] Based on this, in the embodiment, AFL&&AFL++ and LibFuzzer are divided into library fuzzing tests, and the input parameter types required are: function entry and function input parameter type of function entry; and Syzkaller is divided into driver fuzzing tests, and the input parameter types required are: device node information, command code, and command code input parameter type. The input rules, that is, the use or compilation method of the input parameters obtained by each test tool, and the syntax requirements can be determined by analyzing the running environment and the underlying logic of each test tool, which is not limited in the present application.

[0109] In the embodiment, the input parameter types and input rules of each test tool can be extracted in advance, and then associated with the corresponding test tool, and then after determining the test tool of the source code to be tested, the corresponding input parameter type and input rule can be determined directly through the association relationship.

[0110] 103: The electronic device extracts parameters from the source code to be tested according to the input parameter type, and obtains a target parameter file.

[0111] In the embodiment, as in step 102, based on the difference of the test objects, the test tools are divided into library fuzzing tests and driver fuzzing tests, and the method of extracting parameters from the source code to be tested according to the input parameter type to obtain a target parameter file under the two test types will be described below.

[0112] (1) Library fuzzing test:

[0113] The test tool corresponding to the library class fuzzing test is AFL && AFL++ and LibFuzzer, and the input parameter type is: function entry and function input parameter type of function entry.

[0114] In the embodiment, first, the header file of the source code to be tested can be obtained. Specifically, as in step 101, the header file path of the source code to be tested is included in the configuration file, so that the header file path of the source code to be tested can be determined according to the configuration file, and then the header file of the source code to be tested is obtained according to the header file path.

[0115] It should be noted that in addition to automatically obtaining the header file path when importing the source code to be tested, the user can also define the header file path in the configuration file. At this time, the user can manually sort the function entries to be tested in different header files of the source code to be tested in a new header file before testing, and then fill the path of the new header file into the configuration file for specified scanning. Thus, the scanned header file is converted from multiple header files in the original source code to be tested to one sorted header file, improving the scanning efficiency, and the function entries to be tested can be flexibly adjusted according to the test scheme and test purpose, improving the flexibility and efficiency of the test.

[0116] In the embodiment, after obtaining the header file, the header file can be scanned according to the input parameter type to extract at least one first function entry in the header file. Then, the header file is parsed based on the at least one first function entry to determine the function input parameter type of each first function entry in the at least one first function entry.

[0117] Specifically, the at least one first function entry can be understood as all function entries appearing in the header file. Based on this, in the embodiment, the header file can be converted into an abstract syntax tree (AST), and then the nodes of which the type is function are extracted by traversing the AST to obtain the at least one first function entry. At the same time, for each node of which the type is function, the function definition information under the node is extracted to determine the type of the input parameter of the node as the function input parameter type of the first function entry corresponding to the node.

[0118] Finally, the at least one first function entry and the function input parameter type of each first function entry are integrated according to a preset format to obtain the target parameter file. Specifically, the target parameter file can be a json file, and the preset format is as shown in the following table: Figure 2 wherein Interface is the function entry, and Input is the function input parameter type.

[0119] For example, for the following header file:

[0120]

[0121] The function entry is VulnerableFunctionl, and the input parameter types are uint8_t* and size_t. The target parameter file corresponding to the header file in the above example is shown in Figure 3 .

[0122] In addition, in an optional embodiment, there may be some function input parameter types in at least one first function entry that are not recorded in the header file. For these function entries, which are referred to as second function entries for ease of description, the function input parameter types of all first function entries cannot be obtained by parsing the header file. Therefore, in this embodiment, before the target parameter file is obtained by integrating at least one first function entry and the function input parameter types of each first function entry, the second function entry can be determined from at least one first function entry, and then the second function entry is used to search the source code to be tested to obtain a first structure used to define the second function entry in the source code to be tested. Finally, the function input parameter types corresponding to the second function entry are determined by parsing the first structure. Specifically, the parsing method of the first structure can be similar to the parsing method of the header file, that is, the function input parameter types are identified by converting the first structure into an AST, which will not be described again.

[0123] (2) Driver class fuzzing test:

[0124] The test tool corresponding to the driver class fuzzing test is Syzkaller, and the input parameter types are device node information, command codes, and command code input parameter types corresponding to the command codes.

[0125] In this embodiment, first, the source code to be tested can be converted into an intermediate language. Specifically, the intermediate language can convert the expression format of the structures in the source code to be tested into the same format. For example, the intermediate language can be a Low Level Virtual Machine (LLVM) language, and then the expression format of each structure in the source code to be tested is uniformly converted into the format of structure name+offset+interface as shown in Figure 4 . Thus, the expression format of each structure in the source code is unified, and the subsequent processing efficiency is improved.

[0126] Then, in the embodiment, the preset first string is searched in the intermediate language to determine the device node information. Specifically, taking the LLVM language as an example, the first string can be "alloc chrdev region", and the device node information can be the device name. The statement format corresponding to the string is: int alloc chrdev region (dev_t* dev, unsigned baseminor, unsigned count, const char* name), wherein the fourth parameter "const char* name" is the required device name, and then the device node information can be determined by performing parameter tracing on the retrieved field. For example, after searching, the field corresponding to "alloc chrdev region" in the source code converted into the LLVM language is as follows:

[0127] %66 = call i32 @alloc chrdev region (i32* @g tc ns client devt, i32 0, i321, i8* getelementptr inbounds ([13xi8], [13xi8]* @.str.80, i64 0, i640)),!dbg!10775

[0128] wherein the fourth parameter is "i8* getelementptr inbounds ([13xi8], [13xi8]* @.str.80, i64 0, i640)", and after performing device name recognition on the fourth parameter, the device node information is determined to be "@.str.80".

[0129] Then, in the embodiment, the intermediate language can be globally searched to determine at least one command code. Specifically, the embodiment provides a method for globally searching the intermediate language to determine at least one command code, as shown in Figure 5 The method comprises the following steps:

[0130] 501: An electronic device globally searches the intermediate language to determine at least one second structure body.

[0131] In this embodiment, each of the at least one second structure is used to register input / output control (IOCTL) instructions. Since all structures are converted into a unified format of structure name + offset + interface through an intermediate language, the structure used to register ioctl_handler (IOCTL) has an interface of "IOCTL". Based on this, a search can be performed in the intermediate language using "IOCTL" as the search string to obtain all structures whose interface part is "IOCTL", which are then used as the at least one second structure.

[0132] 502: The electronic device integrates information from at least one second structure to obtain at least one structure name.

[0133] like Figure 4 As shown, in this embodiment, the structure name is represented in the intermediate language in the form of "struct.xxx". Based on this, the string after "struct." in each second structure can be identified and extracted to obtain at least one structure name. It should be noted that in this step, the extracted structure names may be duplicated. If duplicates occur, they can be deduplicated before summarizing and counting to obtain the at least one structure name.

[0134] 503: The electronic device uses at least one structure name to search the intermediate language and obtains at least one third structure.

[0135] In this embodiment, the at least one structure name includes the structure name of any one of the at least one third structure. Specifically, based on each structure name in the at least one structure name, a search is performed in the intermediate language to find structures with the same name as each structure name and summarize them to obtain at least one third structure. For example, for the second structure "struct.file_operations,9,IOCTL", its structure name is "file_operations". After searching, two other structures with the same name are obtained: "struct.file_operations,2,FileRead" and "struct.file_operations,3,FileWrite". Then, "struct.file_operations,2,FileRead" and "struct.file_operations,3,FileWrite" are used as the third structures corresponding to the second structure "struct.file_operations,9,IOCTL".

[0136] 504: The electronic device searches the at least one second structure and the at least one third structure using the preset second string to obtain at least one target code.

[0137] Specifically, taking the LLVM language as an example, the second string can be "command". After searching each second structure or each third structure, the path of the target code corresponding to each second structure or each third structure is obtained, and then the target code corresponding to each second structure or each third structure can be obtained according to the path.

[0138] 505: The electronic device searches the at least one target code using a preset third string to obtain at least one command code.

[0139] In the embodiment, the at least one command code corresponds to the at least one target code one by one.

[0140] Specifically, taking the LLVM language as an example, the third string can be "switch case". By checking the corresponding "switch case" command in each target code, the command code of the structure corresponding to each target code can be obtained.

[0141] In an optional embodiment, as shown in Figure 6 , there is a possibility that the obtained command code nests another IOCTL structure, and then when searching down, the IOCTL structure needs to be continuously concerned. When it is found that there is nesting, the method such as steps 501-505 can be applied to analyze and confirm the nesting logic, and obtain the command code corresponding to the nested IOCTL structure.

[0142] 506: The electronic device performs backtracking analysis on each command code in the at least one command code to determine the command code input parameter type corresponding to each command code.

[0143] Specifically, the parameter type can be located by analyzing the data flow and backtracking the parameter transmission process.

[0144] 507: The electronic device integrates the device node information, the at least one command code, and the command code input parameter type corresponding to each command code to obtain a target parameter file.

[0145] Specifically, the target parameter file can be a json file, and the preset format is as shown in Figure 7 , wherein device is the device node information, handler is the second structure or the third structure, code is the command code, and value is the command code input parameter type.

[0146] 104: The electronic device generates a test case of the source code to be tested according to the input rule and the target parameter file.

[0147] In the embodiment, the input syntax and input structure of the test tool can be determined according to the input rule, and then the parameters in the target parameter file are edited according to the input syntax and input structure to obtain the test case. For example, an input template of the test tool can be generated according to the input syntax and input structure of the test tool, and then the content in the target parameter file is associated and filled into the input template to generate the test case of the source code to be tested. In addition, the input template can also be generated in advance, and then directly called when used to improve processing efficiency.

[0148] For example, taking LibFuzzer as an example, the input syntax and input rule are as follows:

[0149]

[0150] Wherein, “AAAA” corresponds to the function entry in the target parameter file, and “BBBB” corresponds to the function input parameter type in the target parameter file.

[0151] Then, the specific content of the “BBBB” part is adjusted according to the function input parameter type in the target parameter file. For example, taking the header file in the library fuzzing test example in step 103 as an example, the corresponding target parameter file is as follows:

[0152] Interface:VulnerableFunction1

[0153] Input:[uint8_t*,size_t]

[0154] It can be seen that it contains two input parameters, and the types are “uint8_t*” and “size_t” respectively. Based on this, combined with the input syntax and input rule of LibFuzzer, the following template can be obtained:

[0155]

[0156] Then, the content in the target parameter file is associated and filled into the input template, and the following test case can be obtained:

[0157]

[0158] In another embodiment, taking AFL&&AFL++ as an example, the input syntax and input rule are as follows: a test seed of the same type is generated according to the function input type, and then the function input parameter type in the test seed is imported through a main function with stdin to obtain the test case.

[0159] Based on this, the header file in the library class fuzzing test example in step 103 is continued to be used, the function input types of which are “uint8_t*” and “size_t”, and then a plurality of test seeds composed of data of the same type are generated based on the types, and the test case shown in the following is obtained:

[0160]

[0161] In another embodiment, taking Syzkaller as an example, the input syntax and input rules are: after the relevant information of the driver interface is extracted, the test case is written using the syzlang syntax. Based on this, in the embodiment, the general Syzkaller template meeting the requirements can be generated according to the syntax rules of syzlang based on the test logic of Syzkaller and the input parameter types obtained above. Then, the input parameter types obtained above are filled into the corresponding positions of the template to generate the test case.

[0162] In an optional embodiment, the generated Syzkaller test case can also be analyzed in advance to extract the general statements therein to construct a general Syzkaller template. Then, in actual testing, the pre-generated template can be directly called to generate the test case, thereby further improving the testing efficiency. In summary, the test case automatic generation method provided in the embodiment determines the test tool of the to-be-tested source code through the configuration file of the to-be-tested source code, and then determines the input parameter types and input rules of the test tool. Then, the to-be-tested source code is parameter-extracted according to the input parameter types to obtain a target parameter file, and then the test case of the to-be-tested source code is generated according to the input rules and the target parameter file. In this way, the automatic generation of the test case is realized, without the need for a large amount of manual intervention, the overall testing efficiency is improved, and the method can be applied to the testing scene of a large number of different to-be-tested programs. At the same time, according to the input rules of the test tool, the generated test case is not affected by the professional and technical ability of the tester, and can perfectly meet the needs of the test tool, ensuring the objectivity and correctness of the test results.

[0163] In order to better implement the above-mentioned scheme of the embodiments of the present application, correspondingly, the related device for cooperating with the implementation of the above-mentioned scheme is also provided below.

[0164] Figure 8 A structural schematic diagram of a test case automatic generation device provided in the embodiments of the present application, the test case automatic generation device 800 comprises:

[0165] The source code analysis module 801 is configured to determine the test tool of the to-be-tested source code according to the test configuration file, and determine the input parameter types and input rules based on the test tool.

[0166] The information integration module 802 is configured to perform parameter extraction on the source code to be tested according to the input parameter type, to obtain a target parameter file.

[0167] The use case generation module 803 is configured to generate a test case of the source code to be tested according to the input rule and the target parameter file.

[0168] The source code analysis module 801 can include a Syzkaller analysis unit 804 and an AFL&&AFL++&&LibFuzzer analysis unit 805 according to different test tools. When a different Fuzzing tool needs to be selected for testing, the analysis unit of the corresponding tool can be called to analyze the source code to be tested. The information integration module 802 can include a file generation unit 806, which is configured to write the extracted parameters in the source code to be tested into a.json file, to generate a.json file for generating a test case. The.json file is the target parameter file. The use case generation module 803 can include a file acquisition unit 807 and a use case template library 808. The file acquisition unit 807 is configured to acquire the generated.json file from the file generation unit 806, and the use case template library 808 is configured to store a pre-generated test case template.

[0169] In a possible implementation, when the test tool is AFL&&AFL++ or LibFuzzer, the information integration module 802 is specifically configured to perform parameter extraction on the source code to be tested according to the input parameter type, to obtain a target parameter file, in the following manner.

[0170] Acquire a header file of the source code to be tested;

[0171] Scan the header file according to the input parameter type, and extract at least one first function entry in the header file;

[0172] Parse the header file according to the at least one first function entry, and determine a function input parameter type of each first function entry in the at least one first function entry;

[0173] Integrate the at least one first function entry and the function input parameter type of each first function entry, to obtain the target parameter file.

[0174] In a possible implementation, when scanning the header file according to the input parameter type, and extracting at least one first function entry in the header file, the information integration module 802 is specifically configured to perform the following operations.

[0175] Convert the header file into an abstract syntax tree, traverse the abstract syntax tree, extract a node whose node type is a function, and obtain the at least one first function entry.

[0176] In a possible implementation, before integrating the at least one first function entry and the function input parameter type of each first function entry to obtain the target parameter file, the information integration module 802 is further configured to:

[0177] determine a second function entry in the at least one first function entry, wherein the second function entry is a function input parameter type that is not recorded in the header file in the at least one first function entry;

[0178] retrieve the source code to be tested according to the second function entry to obtain a first structure corresponding to the second function entry, wherein the first structure is used to define the second function entry;

[0179] parse the first structure to determine the function input parameter type of the second function entry.

[0180] In a possible implementation, in the aspect of obtaining the header of the source code to be tested, the information integration module 802 is specifically configured to:

[0181] determine the header file path of the source code to be tested according to the configuration file;

[0182] obtain the header file of the source code to be tested according to the header file path.

[0183] In a possible implementation, when the test tool is Syzkaller, in the aspect of performing parameter extraction on the source code to be tested according to the input parameter type to obtain the target parameter file, the information integration module 802 is specifically configured to:

[0184] convert the source code to be tested into an intermediate language, wherein the intermediate language is used to convert the expression format of the structure in the source code to be tested into the same format;

[0185] search the intermediate language by using a preset first string to determine device node information;

[0186] globally search the intermediate language to determine at least one command code;

[0187] perform backtracking analysis on each command code in the at least one command code to determine a command code input parameter type corresponding to each command code;

[0188] integrate the device node information, the at least one command code, and the command code input parameter type corresponding to each command code to obtain the target parameter file.

[0189] In a possible implementation, in the aspect of globally searching the intermediate language to determine the at least one command code, the information integration module 802 is specifically configured to:

[0190] searching the intermediate language according to the at least one structure name, to obtain at least one third structure, wherein the at least one structure name comprises a structure name of any one of the at least one third structure;

[0191] integrating information of the at least one second structure, to obtain at least one structure name;

[0192] searching the intermediate language according to the at least one structure name, to obtain at least one third structure, wherein the at least one structure name comprises a structure name of any one of the at least one third structure;

[0193] searching the at least one second structure and the at least one third structure according to the preset second string, to obtain at least one target code;

[0194] searching the at least one target code according to a preset third string, to obtain at least one command code, wherein the at least one command code corresponds to the at least one target code in a one-to-one manner.

[0195] In a possible implementation, in terms of generating a test case of a to-be-tested source code according to an input rule and a target parameter file, the case generation module 803 is specifically configured to:

[0196] determine an input syntax and an input structure of a test tool according to the input rule;

[0197] edit parameters in the target parameter file according to the input syntax and the input structure, to obtain the test case.

[0198] Figure 9 A structural schematic diagram of an electronic device is provided in an embodiment of the present application. The electronic device 900 includes a processor 901 and a memory 902, which are connected through a bus 903. The memory 902 stores program instructions and data.

[0199] The processor 901 is configured to read the program instructions in the memory 902 to perform the following operations:

[0200] determine a test tool of a to-be-tested source code according to a test configuration file;

[0201] determine an input parameter type and an input rule based on the test tool;

[0202] extract parameters of the to-be-tested source code according to the input parameter type, to obtain a target parameter file;

[0203] generate a test case of the to-be-tested source code according to the input rule and the target parameter file.

[0204] In a possible implementation, when the test tool is AFL && AFL++ or LibFuzzer, in the aspect of parameter extraction on the source code to be tested according to the input parameter type, the processor 901 is specifically configured to perform the following operations:

[0205] Obtain the header file of the source code to be tested.

[0206] Scan the header file according to the input parameter type, and extract at least one first function entry in the header file;

[0207] Parse the header file according to the at least one first function entry, and determine the function input parameter type of each first function entry in the at least one first function entry;

[0208] Integrate the at least one first function entry and the function input parameter type of each first function entry, and obtain a target parameter file.

[0209] In a possible implementation, in the aspect of scanning the header file according to the input parameter type and extracting at least one first function entry in the header file, the processor 901 is specifically configured to perform the following operations:

[0210] Convert the header file into an abstract syntax tree, traverse the abstract syntax tree, extract a node whose node type is a function, and obtain at least one first function entry.

[0211] In a possible implementation, before the aspect of integrating the at least one first function entry and the function input parameter type of each first function entry to obtain the target parameter file, the processor 901 is further configured to perform the following operations:

[0212] Determine a second function entry in the at least one first function entry, where the second function entry is a function input parameter type that is not recorded in the header file in the at least one first function entry;

[0213] Search the source code to be tested according to the second function entry, and obtain a first structure corresponding to the second function entry, where the first structure is used to define the second function entry;

[0214] Parse the first structure, and determine the function input parameter type of the second function entry.

[0215] In a possible implementation, in the aspect of obtaining the header of the source code to be tested, the processor 901 is specifically configured to perform the following operations:

[0216] Determine the header file path of the source code to be tested according to a configuration file;

[0217] Obtain the header file of the source code to be tested according to the header file path.

[0218] In a possible implementation, when the test tool is Syzkaller, the processor 901 is specifically configured to perform the following operations in parameter extraction on the source code to be tested according to the input parameter type to obtain the target parameter file:

[0219] transforming the source code to be tested into an intermediate language, where the intermediate language is used to transform the expression format of the structure in the source code to be tested into the same format;

[0220] searching the intermediate language by using a preset first string to determine the device node information;

[0221] globally searching the intermediate language to determine at least one command code;

[0222] backtracking and analyzing each command code in the at least one command code to determine a command code input parameter type corresponding to each command code;

[0223] integrating the device node information, the at least one command code, and the command code input parameter type corresponding to each command code to obtain the target parameter file.

[0224] In a possible implementation, the processor 901 is specifically configured to perform the following operations in globally searching the intermediate language to determine the at least one command code:

[0225] globally searching the intermediate language to determine at least one second structure, where each second structure in the at least one second structure is used to register an input-output control instruction;

[0226] integrating information of the at least one second structure to obtain at least one structure name;

[0227] searching the intermediate language by using the at least one structure name to obtain at least one third structure, where the at least one structure name includes a structure name of any one third structure in the at least one third structure;

[0228] searching the at least one second structure and the at least one third structure by using a preset second string to obtain at least one target code;

[0229] searching the at least one target code by using a preset third string to obtain at least one command code, where the at least one command code corresponds to the at least one target code in a one-to-one manner.

[0230] In a possible implementation, the processor 901 is specifically configured to perform the following operations in generating a test case of the source code to be tested according to the input rule and the target parameter file:

[0231] determining an input syntax and an input structure of the test tool according to the input rule;

[0232] According to the input syntax and the input structure, parameters in the target parameter file are edited to obtain a test case.

[0233] In a possible implementation, the electronic device can be a server.

[0234] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement some or all steps of any one of the test case automatic generation methods described in the above method embodiments.

[0235] The embodiments of the present application further provide a computer program product, which, when running on a computer or a processor, causes the computer or the processor to perform one or more steps of any one of the above communication methods. The constituent modules of the above-mentioned devices, if realized in the form of software function units and sold or used as independent products, can be stored in the computer readable storage medium.

[0236] It should be understood that the processor mentioned in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0237] It should also be understood that the memory referred to in the implementations of the application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable Programmable ROM (EPROM), an Electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a Random Access Memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM) and Direct Rambus RAM (DRRAM).

[0238] It should be noted that when the processor is a general processor, a DSP, an ASIC, a FPGA or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, the memory (storage module) is integrated in the processor.

[0239] It should be noted that for the foregoing method embodiments, in order to simply describe, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the application is not limited by the action order described, because according to the application, certain steps can be performed with other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily necessary for the application.

[0240] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can refer to the related description of other embodiments.

[0241] In several embodiments provided in the present application, it should be understood that the disclosed apparatus can be implemented in other manners. For example, the division of the apparatus embodiments described above is merely illustrative, and the division of units can be different, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0242] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0243] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in the form of hardware or in the form of a software program module.

[0244] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer readable memory. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application.

[0245] Those of ordinary skill in the art can understand that all or part of the steps of the various methods in the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer readable memory. The memory can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0246] The above has carried out the detailed introduction to the embodiment of the application, the principle and the implementation of the application are described in the text by applying specific examples, the above implementation of the method of the application and its core idea is only used for helping understanding; simultaneously, for the general technical personnel of the field, according to the idea of the application, there will be changes in specific implementation and application range, and on the basis, the content of the specification should not be understood as the limitation of the application.

Claims

1. A method for automatically generating test cases, characterized in that, The method includes: The electronic device determines the testing tool for the source code to be tested based on the test configuration file; The electronic device determines the input parameter type and input rules based on the testing tool; The electronic device extracts parameters from the source code under test according to the input parameter type to obtain a target parameter file; The electronic device determines the input syntax and input structure of the testing tool according to the input rules; The electronic device edits the parameters in the target parameter file according to the input syntax and the input structure to obtain the test cases; When the testing tool is AFL, AFL++, or LibFuzzer, the electronic device extracts parameters from the source code under test according to the input parameter type to obtain a target parameter file, including: The electronic device acquires the header file of the source code under test; The electronic device scans the header file according to the input parameter type and extracts at least one first function entry point within the header file; The electronic device parses the header file according to the at least one first function entry point to determine the function input parameter type of each first function entry point in the at least one first function entry point; The electronic device integrates each first function entry and the function input parameter type of each first function entry to obtain the target parameter file; When the testing tool is Syzkaller, the electronic device extracts parameters from the source code under test according to the input parameter type to obtain a target parameter file, including: The electronic device converts the source code under test into an intermediate language, wherein the intermediate language is used to convert the expression format of the structure in the source code under test into the same format; The electronic device uses a preset first string to search the intermediate language to determine the device node information; The electronic device performs a global search of the intermediate language to determine at least one command code; The electronic device performs back-tracking parsing on each of the at least one command codes to determine the command code input parameter type corresponding to each command code; The electronic device integrates the device node information, the at least one command code, and the command code input parameter type corresponding to each command code to obtain the target parameter file.

2. The method according to claim 1, characterized in that, The electronic device scans the header file according to the input parameter type and extracts at least one first function entry point within the header file, including: The electronic device converts the header file into an abstract syntax tree, traverses the abstract syntax tree, extracts nodes of type function, and obtains the at least one first function entry point.

3. The method according to claim 1, characterized in that, Before the electronic device integrates the at least one first function entry and the function input parameter types of each first function entry to obtain the target parameter file, the method further includes: The electronic device determines a second function entry among the at least one first function entry, wherein the second function entry is a function entry among the at least one first function entry whose function input parameter type is not recorded in the header file; The electronic device retrieves the source code to be tested according to the second function entry and obtains the first structure corresponding to the second function entry, wherein the first structure is used to define the second function entry; The electronic device parses the first structure to determine the type of function input parameters for the second function entry point.

4. The method according to claim 1, characterized in that, The electronic device acquires the header file of the source code under test, including: The electronic device determines the header file path of the source code to be tested according to the configuration file; The electronic device obtains the header file of the source code to be tested according to the header file path.

5. The method according to claim 1, characterized in that, The electronic device globally searches the intermediate language to determine at least one command code, including: The electronic device globally searches the intermediate language to determine at least one second structure, wherein each of the at least one second structure is used to register input / output control instructions; The electronic device integrates information from the at least one second structure to obtain at least one structure name; The electronic device uses the at least one structure name to search the intermediate language to obtain at least one third structure, wherein the at least one structure name includes the structure name of any one of the at least one third structures; The electronic device uses a preset second string to search for at least one second structure and at least one third structure to obtain at least one target code; The electronic device uses a preset third string to search for the at least one target code to obtain the at least one command code, wherein the at least one command code corresponds one-to-one with the at least one target code.

6. An electronic device, characterized in that, The device includes a processor and a memory, the processor and the memory being coupled; wherein the memory is used to store computer program instructions; when the computer program instructions are executed by the processor, the electronic device performs the method as described in any one of claims 1-5.

7. The electronic device according to claim 6, characterized in that, The electronic device is a server.

Images