Test case generation method, device, computer equipment and storage medium

Abstract

The present application relates to a test case generation method, device, computer equipment and storage medium, which are applied to the field of artificial intelligence. The method includes: executing the case to be identified by compiling the service program to be tested after the plugging, and obtaining the plugging output information corresponding to the case to be identified; based on the plugging output information, generating the execution flow graph corresponding to the case to be identified; the execution flow graph represents the running path composed of the basic blocks of the program passed during the execution of the case to be identified; obtaining the test case queue for the service program to be tested; the test case queue includes multiple valid test cases; if the execution flow graph corresponding to the case to be identified is different from the execution flow graph corresponding to each test case in the test case queue, it is determined that the case to be identified is valid, and the case to be identified is added to the test case queue. The use of this method can avoid the omission of branch paths in the service program to be tested.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a test case generation method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] Fuzz testing is a method of discovering software vulnerabilities by providing unexpected inputs to the target system and monitoring abnormal results. In fuzz testing, a program is attacked with random bad data (also called fuzz), and then which part of the program is damaged is observed to determine the program's vulnerabilities and make improvements or repairs.

[0003] Most current fuzz testing methods generate test cases based on code coverage, specifically generating some initial test sets, mutating the test files in the initial test sets, and adding them to the test queue if the mutated files update the coverage. However, this method only considers code coverage, and even if the code coverage reaches 100%, there may still be undiscovered vulnerabilities. Summary of the invention

[0004] Based on this, it is necessary to provide a test case generation method, apparatus, computer device, computer-readable storage medium and computer program product to address the technical problem that the above-mentioned test case generation method is prone to leaking vulnerabilities.

[0005] In a first aspect, the present application provides a test case generation method. The method comprises:

[0006] The case to be identified is executed by compiling and instrumenting the business program to be tested, and the instrumentation output information corresponding to the case to be identified is obtained; the business program to be tested after compiling and instrumenting includes multiple program basic blocks, and the instrumentation output information is the output information of each program basic block;

[0007] Based on the instrumentation output information, an execution flow graph corresponding to the case to be identified is generated; the execution flow graph represents a running path composed of program basic blocks passed during the execution of the case to be identified;

[0008] Acquire a test case queue for the business program to be tested; the test case queue includes a plurality of valid test cases;

[0009] If the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue, it is determined that the case to be identified is valid, and the case to be identified is added to the test case queue.

[0010] In one embodiment, generating an execution flow graph corresponding to the case to be identified based on the instrumentation output information includes:

[0011] Based on the instrumentation output information, determining the program basic block associated with the case to be identified and the association information between the associated program basic blocks;

[0012] According to the execution order of the associated program basic blocks and the associated information, an execution flow graph corresponding to the case to be identified is generated.

[0013] In one embodiment, the method further comprises:

[0014] Generate a first adjacency matrix according to the execution flow graph corresponding to the case to be identified, and generate a second adjacency matrix corresponding to each test case in the test case queue according to the execution flow graph corresponding to each test case;

[0015] If the first adjacency matrix is ​​different from each second adjacency matrix, it is determined that the execution flow graph corresponding to the case to be identified is different from the execution flow graph corresponding to each test case.

[0016] In one embodiment, generating a first adjacency matrix according to the execution flow graph corresponding to the case to be identified includes:

[0017] Obtaining the number of program basic blocks in the execution flow graph corresponding to the case to be identified, and determining an initial adjacency matrix according to the number of program basic blocks;

[0018] Based on the association relationship between the basic blocks of the program in the execution flow graph, the element value of each element in the initial adjacency matrix is ​​determined, and the first adjacency matrix is ​​obtained based on the element value.

[0019] In one embodiment, the service program to be tested is instrumented in the following manner:

[0020] Generate a control flow graph corresponding to the business program to be tested; the control flow graph includes all paths that can be traversed during the execution of the business program, each vertex in the control flow graph corresponds to a program basic block, and each edge corresponds to a branch instruction;

[0021] Compile and insert a stub for each program basic block in the control flow graph to obtain a compiled and inserted service program to be tested.

[0022] In one embodiment, generating a control flow graph corresponding to the business program to be tested includes:

[0023] Generate an abstract syntax tree corresponding to the business program to be tested;

[0024] Based on the abstract syntax tree, a control flow graph corresponding to the business program to be tested is generated.

[0025] In a second aspect, the present application also provides a test case generation device. The device comprises:

[0026] The instrumentation output module is used to execute the case to be identified by compiling and instrumenting the business program to be tested, and obtain the instrumentation output information corresponding to the case to be identified; the business program to be tested after compilation and instrumentation includes multiple program basic blocks, and the instrumentation output information is the output information of each program basic block;

[0027] An execution flow generation module generates an execution flow graph corresponding to the case to be identified based on the instrumentation output information; the execution flow graph represents a running path composed of program basic blocks passed during the execution of the case to be identified;

[0028] A case acquisition module, used to acquire a test case queue for the business program to be tested; the test case queue includes a plurality of valid test cases;

[0029] The validity identification module is used to determine that the case to be identified is valid and add the case to be identified to the test case queue if the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue.

[0030] In one of the embodiments, the execution flow generation module is specifically used to determine the program basic block associated with the case to be identified and the association information between the associated program basic blocks based on the instrumentation output information; and generate the execution flow graph corresponding to the case to be identified according to the execution order of the associated program basic blocks and the association information.

[0031] In one embodiment, the validity identification module is also used to generate a first adjacency matrix based on the execution flow graph corresponding to the case to be identified, and to generate a second adjacency matrix corresponding to each test case in the test case queue based on the execution flow graph corresponding to each test case; if the first adjacency matrix is ​​different from each second adjacency matrix, it is determined that the execution flow graph corresponding to the case to be identified is different from the execution flow graph corresponding to each test case.

[0032] In one of the embodiments, the validity identification module is also used to obtain the number of program basic blocks in the execution flow graph corresponding to the case to be identified, and determine the initial adjacency matrix according to the number of program basic blocks; based on the association relationship between the program basic blocks in the execution flow graph, determine the element value of each element in the initial adjacency matrix, and obtain the first adjacency matrix based on the element value.

[0033] In one of the embodiments, the device also includes a compilation and instrumentation module, which is used to generate a control flow graph corresponding to the business program to be tested; the control flow graph includes all paths that can be traversed during the execution of the business program, each vertex in the control flow graph corresponds to a program basic block, and each edge corresponds to a branch instruction; each program basic block in the control flow graph is compiled and instrumented to obtain the business program to be tested after compilation and instrumentation.

[0034] In one of the embodiments, the compiling and inserting module is further used to generate an abstract syntax tree corresponding to the business program to be tested; based on the abstract syntax tree, a control flow graph corresponding to the business program to be tested is generated.

[0035] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0036] The case to be identified is executed by compiling and instrumenting the business program to be tested, and the instrumentation output information corresponding to the case to be identified is obtained; the business program to be tested after compiling and instrumenting includes multiple program basic blocks, and the instrumentation output information is the output information of each program basic block;

[0037] Based on the instrumentation output information, an execution flow graph corresponding to the case to be identified is generated; the execution flow graph represents a running path composed of program basic blocks passed during the execution of the case to be identified;

[0038] Acquire a test case queue for the business program to be tested; the test case queue includes a plurality of valid test cases;

[0039] If the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue, it is determined that the case to be identified is valid, and the case to be identified is added to the test case queue.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0041] The case to be identified is executed by compiling and instrumenting the business program to be tested, and the instrumentation output information corresponding to the case to be identified is obtained; the business program to be tested after compiling and instrumenting includes multiple program basic blocks, and the instrumentation output information is the output information of each program basic block;

[0042] Based on the instrumentation output information, an execution flow graph corresponding to the case to be identified is generated; the execution flow graph represents a running path composed of program basic blocks passed during the execution of the case to be identified;

[0043] Acquire a test case queue for the business program to be tested; the test case queue includes a plurality of valid test cases;

[0044] If the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue, it is determined that the case to be identified is valid, and the case to be identified is added to the test case queue.

[0045] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0046] The case to be identified is executed by compiling and instrumenting the business program to be tested, and the instrumentation output information corresponding to the case to be identified is obtained; the business program to be tested after compiling and instrumenting includes multiple program basic blocks, and the instrumentation output information is the output information of each program basic block;

[0047] Based on the instrumentation output information, an execution flow graph corresponding to the case to be identified is generated; the execution flow graph represents a running path composed of program basic blocks passed during the execution of the case to be identified;

[0048] Acquire a test case queue for the business program to be tested; the test case queue includes a plurality of valid test cases;

[0049] If the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue, it is determined that the case to be identified is valid, and the case to be identified is added to the test case queue.

[0050] The above-mentioned test case generation method, device, computer equipment, storage medium and computer program product, after executing the case to be identified by compiling and instrumenting the business program to be tested, obtains the instrumentation output information corresponding to the case to be identified, generates an execution flow graph corresponding to the case to be identified based on the instrumentation output information, and compares the execution flow graph corresponding to the case to be identified with the execution flow graphs corresponding to each test case in the test case queue, thereby determining the validity of the case to be identified. This method determines the validity of the test case through the execution flow graph, which can avoid the omission of branch paths in the business program to be tested, thereby increasing the probability of detecting program vulnerabilities in the business program to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1A schematic diagram of a flow chart of a test case generation method in an embodiment;

[0052] Figure 2a An example diagram of a business program before compilation and plugging in an embodiment;

[0053] Figure 2b An example diagram of a business program after compilation and instrumentation in one embodiment;

[0054] Figure 3a In one embodiment, based on Figure 2a A schematic diagram of a control flow graph generated by a business program shown;

[0055] Figure 3b is a schematic diagram of an execution flow chart in one embodiment;

[0056] Figure 4 is a schematic diagram of a business procedure in an embodiment;

[0057] Figure 5 In one embodiment, based on Figure 2a A schematic diagram of an abstract syntax tree generated by a business program shown;

[0058] Figure 6 A schematic diagram of a flow chart of a test case generation method in another embodiment;

[0059] Figure 7 A structural block diagram of a test case generating device in an embodiment;

[0060] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0063] Existing fuzz testing generates test cases based on code coverage (i.e., if the mutated file updates the coverage, it is added to the test queue as a valid test case), but does not consider the branch paths through which the code is covered. Therefore, even if the code coverage reaches 100%, there may be omissions in branch paths, causing problems.

[0064] For example, in the following code, after the test cases a=200 and a=60 are executed, the code coverage has reached 100%, and the generation of fuzzy test cases is stopped at this time, but some abnormal paths are still not found. For example, when a=10, the program crashes.

[0065] Code Example 1:

[0066]

[0067] When a=200, return 100;

[0068] When a=60, return 20;

[0069] When a=10, crash.

[0070] Based on the above problems, the present application proposes a test case generation method based on execution flow comparison to solve the problem that the existing test cases generated based on code coverage are incomplete.

[0071] In one embodiment, Figure 1 As shown, a test case generation method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the terminal can be but is not limited to various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server can be implemented as an independent server or a server cluster consisting of multiple servers. In this embodiment, the method includes the following steps:

[0072] Step S110, executing the case to be identified by compiling and instrumenting the business program to be tested, and obtaining instrumentation output information corresponding to the case to be identified; the business program to be tested after compilation and instrumentation includes multiple program basic blocks, and the instrumentation output information is the output information of each program basic block.

[0073] Among them, the compilation stub is used to record the output information during the execution of the test case.

[0074] Among them, the cases to be identified are obtained by mutating the initial test files.

[0075] Among them, the basic block of the program is a section of program code without branch instructions. The program can only be entered from the beginning of this code and left from the last line of this code. There will be no jumps in the middle to leave this program.

[0076] The business program to be tested can be understood as the business code to be tested.

[0077] In a specific implementation, the method of compiling and instrumenting the business program to be tested can be: generating a control flow graph corresponding to the business program to be tested, compiling and instrumenting each program basic block in the control flow graph, and obtaining the business program to be tested after compilation and instrumentation. The control flow graph includes all paths that can be traversed during the execution of the business program, each vertex in the control flow graph corresponds to a program basic block, and each edge corresponds to a branch instruction.

[0078] refer to Figure 2a and Figure 2b , which are program example diagrams before and after compilation and instrumentation, Figure 2a The business program shown can be divided into program basic block A (lines 2-5 of code), program basic block B (line 6 of code), program basic block C (line 7 of code), program basic block D (line 8 of code) and program basic block E (line 10 of code). After compiling and inserting each program basic block, we get Figure 2b The program basic blocks A1, B1, C1, D1, E1 shown in the figure, each time a program basic block is executed, the corresponding stub code will be executed and the execution information will be output. Figure 2b When the business program to be tested as shown executes the case to be identified, output information of the case to be identified in each basic block of the program can be obtained, which is recorded as the plug-in output information.

[0079] Step S120, based on the instrumentation output information, generate an execution flow graph corresponding to the case to be identified; the execution flow graph represents a running path composed of program basic blocks passed during the execution of the case to be identified.

[0080] Among them, the execution flow graph is composed of program basic blocks.

[0081] In the specific implementation, the program basic blocks associated with the case to be identified during the execution process and the association information between the associated program basic blocks can be determined based on the instrumentation output information, and then the program basic blocks associated with the case to be identified and the association information between the associated program basic blocks can be determined to obtain the execution flow graph corresponding to the case to be identified.

[0082] More specifically, the program basic blocks associated with the case to be identified can be understood as the program basic blocks that the case to be identified passes through during the execution process, and the association information between the program basic blocks can be understood as the instruction information between the program basic blocks.

[0083] Step S130, obtaining a test case queue for the business program to be tested; the test case queue includes a plurality of valid test cases.

[0084] In a specific implementation, each time a valid test case for a business program to be tested is obtained, the test case can be stored in a preset test case queue, thereby obtaining a test case queue including a plurality of valid test cases.

[0085] Step S140: If the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue, it is determined that the case to be identified is valid, and the case to be identified is added to the test case queue.

[0086] It can be understood that the execution flow is based on the program control flow graph, which is an operation path graph composed of the nodes (program basic blocks) passed during the execution of the test case. Comparing the consistency of two execution flows is to compare the consistency of two operation paths.

[0087] In the specific implementation, since the execution flow graph can represent the execution path of the test case, if the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue, it means that the case to be identified covers a new path of the business program to be tested. Therefore, it can be determined that the case to be identified is a case that can detect vulnerabilities in the business program to be tested, that is, the case to be identified is determined to be a valid case, and the case to be identified is added to the test case queue of the business program to be tested.

[0088] More specifically, the consistency of the two execution flow graphs can be compared by constructing an adjacency matrix of the two execution flow graphs, that is, a first adjacency matrix can be generated according to the execution flow graph corresponding to the case to be identified, and a second adjacency matrix corresponding to each test case in the test case queue can be generated. If the first adjacency matrix is ​​different from each second adjacency matrix, it is determined that the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case.

[0089] In the above test case generation method, after executing the case to be recognized through the business program to be tested after compilation and instrumentation, and obtaining the instrumentation output information corresponding to the case to be recognized, based on the instrumentation output information, an execution flow graph corresponding to the case to be recognized is generated, and the execution flow graph corresponding to the case to be recognized is compared with the execution flow graphs corresponding to each test case in the test case queue, so as to determine the validity of the case to be recognized. Determining the validity of the test case through the execution flow graph by this method can avoid missing branch paths in the business program to be tested, thereby increasing the probability of detecting program vulnerabilities in the business program to be tested.

[0090] In an exemplary embodiment, in the above step S120, generating an execution flow graph corresponding to the case to be recognized based on the instrumentation output information can be specifically implemented through the following steps:

[0091] Step S1201, based on the instrumentation output information, determine the program basic blocks associated with the case to be recognized and the association information between the associated program basic blocks;

[0092] Step S1202, generate an execution flow graph corresponding to the case to be recognized according to the execution order and association information of the associated program basic blocks.

[0093] Among them, the association information between program basic blocks can be understood as instruction information, such as Figure 3a "else", "c>max", and "c<min" in

[0094] Specifically, the execution flow graph can be understood as a path of the control flow graph corresponding to the business program to be tested. For example, referring to Figure 3a , which is a schematic diagram of the control flow graph in an exemplary embodiment, A, B, C, D, and E represent program basic blocks. From the start to the end of the program, there are 3 paths, namely: A→B→E, A→C→E, and A→C→D→E, and each path can be used as an execution flow graph.

[0095] Assume that for the Figure 3a shown control flow graph, the instrumentation output information of the case to be recognized includes the information of program basic block A, the information of program basic block C, and the information of program basic block E, and the association information between program basic blocks is: when the condition of basic block A is not established (else), execute the program of basic block C, and when the condition of basic block C is not established (else), execute the program of module E. Then, based on the execution order and instruction information among program basic blocks A, C, and E, an execution flow graph as shown in Figure 3b can be obtained.

[0096] In this embodiment, by inserting the output information, the program basic block associated with the case to be identified and the association information between the associated program basic blocks are determined, and then an execution flow graph corresponding to the case to be identified is generated, so as to facilitate the subsequent determination of the validity of the case to be identified based on the execution flow graph.

[0097] In an exemplary embodiment, the consistency between the execution flow graph corresponding to the case to be identified and the execution flow graph corresponding to each test case can be determined in the following manner:

[0098] Step S1401, generating a first adjacency matrix according to the execution flow graph corresponding to the case to be identified, and generating a second adjacency matrix corresponding to each test case according to the execution flow graph corresponding to each test case in the test case queue;

[0099] Step S1402: If the first adjacency matrix is ​​different from each of the second adjacency matrices, it is determined that the execution flow graph corresponding to the case to be identified is different from the execution flow graph corresponding to each of the test cases.

[0100] Among them, the adjacency matrix is ​​used to record the flow relationship between each program basic block in the execution flow graph.

[0101] In the specific implementation, the consistency between the execution flow graphs can be determined by comparing the adjacency matrices. If the adjacency matrices corresponding to the two execution flow graphs are the same, then the two execution flow graphs are determined to be Figure 1 The same or identical.

[0102] More specifically, a first adjacency matrix can be generated according to the execution flow graph corresponding to the case to be identified, and a second adjacency matrix corresponding to each test case can be generated according to the execution flow graph corresponding to each test case in the test case queue, and the consistency between the first adjacency matrix and each second adjacency matrix can be compared. If the first adjacency matrix is ​​different from each second adjacency matrix, it is determined that the execution flow graph corresponding to the case to be identified is different from the execution flow graph corresponding to each test case. If the first adjacency matrix is ​​the same as any second adjacency matrix, it is determined that the execution flow graph corresponding to the case to be identified is the same as the execution flow graph corresponding to the corresponding test case, and it can be determined that the case to be identified is valid.

[0103] It is understandable that when a linear comparison algorithm is used to compare the basic blocks of a program from the beginning to the end of the execution flow one by one, due to the large number of loops that may exist in the program, this method will produce different comparison results for essentially the same execution flow, thereby generating invalid test cases.

[0104] For example, for Figure 4 The business procedure shown, if the number of loops iter = 3;

[0105] The execution flow is: U—>V—>W—>V—>W—>V—>W—>X;

[0106] If the number of loops iter = 4;

[0107] The execution flow is: U—>V—>W—>V—>W—>V—>W—>V—>W—>X.

[0108] But in fact, there is no essential difference between these two execution flows in terms of finding problems (the test cases corresponding to the execution flow that loops 4 times can hardly find new problems compared to the test cases corresponding to the execution flow that loops 3 times). Therefore, the present application optimizes the comparison algorithm and adopts the adjacency matrix comparison algorithm to avoid the influence of loops on the comparison results.

[0109] The embodiment provides a method for determining the consistency of the execution flow graph through the adjacency matrix, which can avoid the influence of loops on the comparison results.

[0110] In an exemplary embodiment, in the above step S1401, generating a first adjacency matrix according to the execution flow graph corresponding to the case to be identified includes:

[0111] Step S1401A, obtaining the number of program basic blocks in the execution flow graph corresponding to the case to be identified, and determining an initial adjacency matrix according to the number of program basic blocks;

[0112] Step S1401B, based on the association relationship between the basic blocks of the program in the execution flow graph, determine the element value of each element in the initial adjacency matrix, and obtain a first adjacency matrix based on the element value.

[0113] In the specific implementation, there are n program basic blocks, namely: A1, A2...An, which constitutes an n*n initial adjacency matrix M. If the program basic block Ai is the direct predecessor of Aj, the matrix element Mij is 1, otherwise it is 0. By constructing such an adjacency matrix, the influence of loop calls on comparison can be eliminated.

[0114] Still Figure 4 Taking the business program shown in the figure as an example, for the execution flow with the above loop number iter=3 and loop number iter=4, the results obtained by using the adjacency matrix comparison algorithm are shown in Table 1 below. It can be seen from Table 1 that the adjacency matrix can only represent the association relationship between the basic blocks of each program, and does not involve loops. It can be seen that the method of comparing the consistency of the execution flow through the adjacency matrix can remove the interference of the loop.

[0115] Table 1 Adjacency matrix

[0116] U V W X U 0 0 0 0 V 1 0 1 0 W 0 1 0 0 X 0 0 1 0

[0117] It should be noted that the method for determining the adjacency matrix of the execution flow graph corresponding to each test case in the test case queue is the same as the method for determining the first adjacency matrix of the case to be identified, which will not be repeated here.

[0118] In this embodiment, an adjacency matrix is ​​constructed by the number and association relationship of program basic blocks in the execution flow graph, so that the consistency of the execution flow graph can be determined later through the adjacency matrix.

[0119] In an exemplary embodiment, the service program to be tested is instrumented in the following manner:

[0120] Step S210, generating a control flow graph corresponding to the business program to be tested; the control flow graph includes all paths that can be traversed during the execution of the business program, each vertex in the control flow graph corresponds to a program basic block, and each edge corresponds to a branch instruction;

[0121] Step S220, compile and insert each program basic block in the control flow graph to obtain a compiled and inserted service program to be tested.

[0122] Among them, the control-flow graph (CFG) is a notation in computer science, which uses the representation of graphs in mathematics to mark all the paths taken during the execution of a computer program. Each vertex in the control-flow graph corresponds to a basic block of the program, and each edge in the control-flow graph corresponds to a branch instruction. For example, Figure 3a The control flow graph shown is Figure 2a The control flow graph corresponding to the business program shown.

[0123] Further, in an exemplary embodiment, step S210 generates a control flow graph corresponding to the service program to be tested, which can be specifically implemented by the following steps:

[0124] Step S2101, generating an abstract syntax tree corresponding to the business program to be tested;

[0125] Step S2102: Generate a control flow graph corresponding to the business program to be tested based on the abstract syntax tree.

[0126] Among them, the Abstract Syntax Tree (AST) is an abstract representation of the grammatical structure of the source code. It represents the structure of the programming language in a tree form. Each node AST Node of the tree represents a structure in the source code. AST can be regarded as a structured representation of the source code.

[0127] In the specific implementation, after obtaining the business program to be tested, the abstract syntax tree corresponding to the business program to be tested is generated first, and then the abstract syntax tree is traversed, and the control flow graph corresponding to the business program to be tested is generated in combination with the characteristics of the language of the business program to be tested. Each basic block of the program in the control flow graph is further compiled and instrumented to obtain the business program to be tested after compilation and instrumentation.

[0128] For example, refer to Figure 5 , based on Figure 2a The schematic diagram of the abstract syntax tree generated by the business program shown in FIG. 1 shows that the various codes of the business program are divided into different branches based on Figure 5 The abstract syntax tree shown generates Figure 3a The control flow graph shown in the figure is compiled and instrumented for the basic blocks A, B, C, D and E in the control flow graph, and the obtained Figure 2b The service program after the instrumentation shown includes program basic blocks A1, B1, C1, D1 and E1.

[0129] In this embodiment, an abstract syntax tree corresponding to the business program to be tested is generated, a control flow graph corresponding to the business program to be tested is generated based on the abstract syntax tree, and each program basic block in the control flow graph is further compiled and instrumented to obtain the compiled and instrumented business program to be tested, so that the output information during the execution of the case to be identified can be recorded through the compiled and instrumented business program to be tested.

[0130] In one embodiment, Figure 6 FIG. 1 is a flow chart of a test case generation method according to another exemplary embodiment. In this embodiment, the method includes the following steps:

[0131] Step S610, generating an abstract syntax tree corresponding to the business program to be tested;

[0132] Step S620, generating a control flow graph corresponding to the business program to be tested based on the abstract syntax tree;

[0133] Step S630, compile and insert each program basic block in the control flow graph to obtain a compiled and inserted service program to be tested;

[0134] Step S640, executing the case to be identified by compiling the instrumented business program to be tested to obtain instrumented output information corresponding to the case to be identified;

[0135] Step S650, generating an execution flow graph corresponding to the case to be identified based on the instrumentation output information, and generating a first adjacency matrix according to the execution flow graph corresponding to the case to be identified;

[0136] Step S660, obtaining a test case queue for the business program to be tested, and generating a second adjacency matrix corresponding to each test case according to the execution flow graph corresponding to each test case in the test case queue;

[0137] Step S670: If the first adjacency matrix is ​​different from each of the second adjacency matrices, it is determined that the case to be identified is valid, and the case to be identified is added to the test case queue.

[0138] The test case generation method provided in this embodiment solves the problem of branch path omission in existing fuzz testing. Compared with the existing fuzz testing based on code coverage, the present application considers the program execution path and solves the problem that branch paths may be missed even if the code coverage of existing fuzz testing reaches 100%.

[0139] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0140] Based on the same inventive concept, the embodiment of the present application also provides a test case generation device for implementing the test case generation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more test case generation device embodiments provided below can refer to the limitations of the test case generation method above, and will not be repeated here.

[0141] In one embodiment, Figure 7 As shown, a test case generation device is provided, including: an instrumentation output module 710, an execution flow generation module 720, a case acquisition module 730 and a validity identification module 740, wherein:

[0142] The instrumentation output module 710 is used to execute the case to be identified by compiling the instrumented business program to be tested, and obtain the instrumentation output information corresponding to the case to be identified; the compiled instrumented business program to be tested includes multiple program basic blocks, and the instrumentation output information is the output information of each program basic block;

[0143] The execution flow generation module 720 generates an execution flow graph corresponding to the case to be identified based on the instrumentation output information; the execution flow graph represents the running path composed of the basic blocks of the program passed during the execution of the case to be identified;

[0144] The case acquisition module 730 is used to acquire a test case queue for the business program to be tested; the test case queue includes multiple valid test cases;

[0145] The validity identification module 740 is used to determine that the case to be identified is valid and add the case to be identified to the test case queue if the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue.

[0146] In one embodiment, the above-mentioned execution flow generation module 720 is specifically used to determine the program basic block associated with the case to be identified and the association information between the associated program basic blocks based on the plug-in output information; and generate the execution flow graph corresponding to the case to be identified according to the execution order and association information of the associated program basic blocks.

[0147] In one embodiment, the validity identification module 740 is specifically used to generate a first adjacency matrix according to the execution flow graph corresponding to the case to be identified, and to generate a second adjacency matrix corresponding to each test case according to the execution flow graph corresponding to each test case in the test case queue; if the first adjacency matrix is ​​different from each second adjacency matrix, it is determined that the execution flow graph corresponding to the case to be identified is different from the execution flow graph corresponding to each test case.

[0148] In one embodiment, the validity identification module 740 is also used to obtain the number of program basic blocks in the execution flow graph corresponding to the case to be identified, and determine the initial adjacency matrix according to the number of program basic blocks; based on the association relationship between the program basic blocks in the execution flow graph, determine the element value of each element in the initial adjacency matrix, and obtain the first adjacency matrix based on the element value.

[0149] In one embodiment, the above-mentioned device also includes a compilation and instrumentation module, which is used to generate a control flow graph corresponding to the business program to be tested; the control flow graph includes all paths that can be traversed during the execution of the business program, each vertex in the control flow graph corresponds to a program basic block, and each edge corresponds to a branch instruction; each program basic block in the control flow graph is compiled and instrumented to obtain the business program to be tested after compilation and instrumentation.

[0150] In one embodiment, the compiling and inserting module is further used to generate an abstract syntax tree corresponding to the business program to be tested; based on the abstract syntax tree, a control flow graph corresponding to the business program to be tested is generated.

[0151] Each module in the above test case generation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0152] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a test case generation method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0153] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0154] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0155] The case to be identified is executed by compiling and instrumenting the business program to be tested, and the instrumentation output information corresponding to the case to be identified is obtained; the business program to be tested after the compilation and instrumentation includes multiple program basic blocks, and the instrumentation output information is the output information of each program basic block;

[0156] Based on the instrumentation output information, an execution flow graph corresponding to the case to be identified is generated; the execution flow graph represents the running path composed of the basic blocks of the program passed during the execution of the case to be identified;

[0157] Obtain a test case queue for the business program to be tested; the test case queue includes multiple valid test cases;

[0158] If the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue, the case to be identified is determined to be valid, and the case to be identified is added to the test case queue.

[0159] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0160] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0161] The case to be identified is executed by compiling and instrumenting the business program to be tested, and the instrumentation output information corresponding to the case to be identified is obtained; the business program to be tested after the compilation and instrumentation includes multiple program basic blocks, and the instrumentation output information is the output information of each program basic block;

[0162] Based on the instrumentation output information, an execution flow graph corresponding to the case to be identified is generated; the execution flow graph represents the running path composed of the basic blocks of the program passed during the execution of the case to be identified;

[0163] Obtain a test case queue for the business program to be tested; the test case queue includes multiple valid test cases;

[0164] If the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue, the case to be identified is determined to be valid, and the case to be identified is added to the test case queue.

[0165] In one embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0166] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0167] The case to be identified is executed by compiling and instrumenting the business program to be tested, and the instrumentation output information corresponding to the case to be identified is obtained; the business program to be tested after the compilation and instrumentation includes multiple program basic blocks, and the instrumentation output information is the output information of each program basic block;

[0168] Based on the instrumentation output information, an execution flow graph corresponding to the case to be identified is generated; the execution flow graph represents the running path composed of the basic blocks of the program passed during the execution of the case to be identified;

[0169] Obtain a test case queue for the business program to be tested; the test case queue includes multiple valid test cases;

[0170] If the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue, the case to be identified is determined to be valid, and the case to be identified is added to the test case queue.

[0171] In one embodiment, a computer program product is also provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0172] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0173] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0174] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A test case generation method, characterized in that: The method comprises: The case to be identified is executed by compiling and instrumenting the business program to be tested, and the instrumentation output information corresponding to the case to be identified is obtained; the business program to be tested after compiling and instrumenting includes multiple program basic blocks, and the instrumentation output information is the output information of each program basic block; Based on the instrumentation output information, an execution flow graph corresponding to the case to be identified is generated; the execution flow graph represents a running path composed of program basic blocks passed during the execution of the case to be identified; Acquire a test case queue for the business program to be tested; the test case queue includes a plurality of valid test cases; If the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue, it is determined that the case to be identified is valid, and the case to be identified is added to the test case queue.

2. The method according to claim 1, characterized in that The step of generating an execution flow graph corresponding to the case to be identified based on the instrumentation output information includes: Based on the instrumentation output information, determining the program basic block associated with the case to be identified and the association information between the associated program basic blocks; According to the execution order of the associated program basic blocks and the associated information, an execution flow graph corresponding to the case to be identified is generated.

3. The method according to claim 1, characterized in that The method further comprises: Generate a first adjacency matrix according to the execution flow graph corresponding to the case to be identified, and generate a second adjacency matrix corresponding to each test case in the test case queue according to the execution flow graph corresponding to each test case; If the first adjacency matrix is ​​different from each second adjacency matrix, it is determined that the execution flow graph corresponding to the case to be identified is different from the execution flow graph corresponding to each test case.

4. The method according to claim 3, characterized in that The step of generating a first adjacency matrix according to the execution flow graph corresponding to the case to be identified includes: Obtaining the number of program basic blocks in the execution flow graph corresponding to the case to be identified, and determining an initial adjacency matrix according to the number of program basic blocks; Based on the association relationship between the basic blocks of the program in the execution flow graph, the element value of each element in the initial adjacency matrix is ​​determined, and the first adjacency matrix is ​​obtained based on the element value.

5. The method according to claim 1, characterized in that The business program to be tested is compiled and instrumented in the following manner: Generate a control flow graph corresponding to the business program to be tested; the control flow graph includes all paths that can be traversed during the execution of the business program, each vertex in the control flow graph corresponds to a program basic block, and each edge corresponds to a branch instruction; Compile and insert a stub for each program basic block in the control flow graph to obtain a compiled and inserted service program to be tested.

6. The method according to claim 5, characterized in that The generating of a control flow graph corresponding to the business program to be tested includes: Generate an abstract syntax tree corresponding to the business program to be tested; Based on the abstract syntax tree, a control flow graph corresponding to the business program to be tested is generated.

7. A test case generating device, characterized in that: The device comprises: The instrumentation output module is used to execute the case to be identified by compiling and instrumenting the business program to be tested, and obtain the instrumentation output information corresponding to the case to be identified; the business program to be tested after compilation and instrumentation includes multiple program basic blocks, and the instrumentation output information is the output information of each program basic block; An execution flow generation module generates an execution flow graph corresponding to the case to be identified based on the instrumentation output information; the execution flow graph represents a running path composed of program basic blocks passed during the execution of the case to be identified; A case acquisition module, used to acquire a test case queue for the business program to be tested; the test case queue includes a plurality of valid test cases; The validity identification module is used to determine that the case to be identified is valid and add the case to be identified to the test case queue if the execution flow graph corresponding to the case to be identified is different from the execution flow graphs corresponding to each test case in the test case queue.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the test case generation method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the test case generation method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the test case generation method according to any one of claims 1 to 6 are implemented.

Images