Program testing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a program testing method and device, electronic equipment and a storage medium. The program testing method comprises the steps that a seed queue is obtained; calculating the path similarity between any seed case in the seed queue and other seed cases in the seed queue; determining a target seed case from the seed queue based on the path similarity; based on the coverage rate of the target seed case, determining to distribute energy to the target seed case; generating a test case based on the target seed case after energy distribution; and testing by adopting the test case to generate vulnerability information. According to the embodiment of the invention, potential errors and vulnerability in the program can be effectively found out.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a program testing method, a program testing device, an electronic device and a storage medium. Background Art

[0002] Computer software functions are becoming increasingly powerful and complex, resulting in a large number of security risks in the software, and software security issues are becoming increasingly prominent. Fuzz testing is one of the most widely used software automation testing technologies, and has successfully discovered a large number of security vulnerabilities in complex programs. Its core theory is to create a large amount of variant data for the target program, input this data into the target program, monitor the execution status of the program, and record the exceptions generated by the program (such as assertion failures, crashes, etc.) to discover potential errors and vulnerabilities in the program. However, the current gray box fuzz testing technology still faces problems such as low code coverage, high test case redundancy, and low efficiency. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a program testing method, a program testing device, an electronic device and a storage medium that overcome the above problems or at least partially solve the above problems.

[0004] In order to solve the above problems, in a first aspect of the present invention, an embodiment of the present invention discloses a program testing method, comprising:

[0005] Get the seed queue;

[0006] Calculating the path similarity between any seed use case in the seed queue and other seed use cases in the seed queue;

[0007] determining a target seed use case from the seed queue based on the path similarity;

[0008] Based on the coverage of the target seed use case, determine to allocate energy to the target seed use case;

[0009] Generate test cases based on the target seed cases after energy allocation;

[0010] The test case is used to perform testing to generate vulnerability information.

[0011] Optionally, the step of calculating the path similarity between any seed use case in the seed queue and other seed use cases in the seed queue includes:

[0012] Calculating the Hamming distance between any seed case in the seed queue and other seed cases in the seed queue;

[0013] Determining a minimum value among the Hamming distances;

[0014] A path similarity is determined based on the minimum value.

[0015] Optionally, the step of determining the path similarity based on the minimum value includes:

[0016] The reciprocal of the minimum value is calculated as the path similarity.

[0017] Optionally, the step of calculating the Hamming distance between any seed use case in the seed queue and other seed use cases in the seed queue includes:

[0018] Determine any seed use case in the seed queue as a current calculation use case, and other seed use cases in the seed queue as use cases to be compared;

[0019] Performing an XOR calculation on the bitmap corresponding to the current calculation case and the bitmap corresponding to the case to be compared to determine the relationship between the edges;

[0020] Perform an AND operation based on the relationship between the edges to determine the number of different bits;

[0021] The different bit numbers are determined as the Hamming distance of the current calculation case.

[0022] Optionally, the step of determining a target seed use case from the seed queue based on the path similarity includes:

[0023] The seed case with the smallest path similarity in the seed queue is determined as the target seed case.

[0024] Optionally, the step of determining energy allocation to the target seed use case based on the coverage of the target seed use case includes:

[0025] Determine the total number of edges executed by the target seed use case as coverage;

[0026] Based on the allocation mapping relationship between the coverage rate and the preset energy, energy is allocated to the target seed use case; wherein the allocation mapping relationship is negatively correlated.

[0027] Optionally, the method further comprises:

[0028] Repair the vulnerability based on the vulnerability information.

[0029] In a second aspect of the present invention, an embodiment of the present invention discloses a program testing device, comprising:

[0030] The acquisition module is used to obtain the seed queue;

[0031] A path similarity module, used to calculate the path similarity between any seed use case in the seed queue and other seed use cases in the seed queue;

[0032] a target determination module, configured to determine a target seed use case from the seed queue based on the path similarity;

[0033] An allocation module, configured to determine energy allocation to the target seed use case based on the coverage of the target seed use case;

[0034] A generation module, used for generating test cases based on target seed cases after energy allocation;

[0035] The test module is used to perform testing using the test case and generate vulnerability information.

[0036] According to a third aspect of the present invention, an embodiment of the present invention discloses an electronic device, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the program testing method as described above when executed by the processor.

[0037] In a fourth aspect of the present invention, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the program testing method described above are implemented.

[0038] The embodiments of the present invention include the following advantages:

[0039] The embodiment of the present invention obtains a seed queue; calculates the path similarity between any seed case in the seed queue and other seed cases in the seed queue; determines a target seed case from the seed queue based on the path similarity; determines the energy to be allocated to the target seed case based on the coverage of the target seed case; generates a test case based on the target seed case after energy allocation; and uses the test case for testing to generate vulnerability information. Seed cases are selected based on path similarity, and seeds with low-frequency paths are more likely to be executed during subsequent runtime; and energy allocation is guided based on coverage, providing more opportunities for blurring of test cases that are more likely to find errors and crashes. By optimizing the seed screening strategy and the energy allocation strategy, the ability to discover errors and crashes is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flowchart of steps of an embodiment of a program testing method of the present invention;

[0041] Figure 2 is a flowchart of another program testing method embodiment of the present invention;

[0042] Figure 3 It is a structural schematic diagram of a test case of the present invention;

[0043] Figure 4 is a schematic diagram of a test example of the present invention;

[0044] Figure 5 is a structural block diagram of an embodiment of a program testing device of the present invention;

[0045] Figure 6 is a structural block diagram of an electronic device provided by an embodiment of the present invention;

[0046] Figure 7 It is a structural block diagram of a storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] Grey-box fuzz testing refers to effectively discovering software errors with partial understanding of the program. It often uses instrumentation to obtain feedback information from the target program to adjust the mutation strategy and create test cases that may cover more execution paths or discover bugs (vulnerabilities) faster. Currently, many technologies have been used in the field of fuzz testing, such as manual analysis, symbolic execution, machine learning, deep learning, etc., including optimization and improvement of seed generation and selection, energy scheduling, and mutation strategies. However, grey-box fuzz testing still has the following limitations:

[0049] 1. Screening seeds through simple favorite (a strategy for screening test cases), not making full use of the characteristics of seeds and program execution paths, so that seeds that can cause crashes or discover new paths cannot be executed within a limited period of time, thus affecting the efficiency of fuzz testing.

[0050] 2. Energy is allocated only based on coverage, execution time, and file size, and too much energy is allocated to seeds that execute high-frequency paths. Energy cannot be allocated adaptively, resulting in less testing of areas where crashes may exist, thus affecting bug discovery.

[0051] Reference Figure 1 , shows a flowchart of a program testing method embodiment of the present invention, the program testing method may specifically include the following steps:

[0052] Step 101, obtaining a seed queue;

[0053] In an embodiment of the present invention, the fuzzer can initialize and provide a seed queue. The seed queue can be obtained.

[0054] Step 102, calculating the path similarity between any seed use case in the seed queue and other seed use cases in the seed queue;

[0055] For any seed use case in the seed queue, compare the seed use case with other seed use cases in the seed queue to determine the path similarity between them. In addition, when a seed use case is added to the seed queue suddenly, only the path similarity between the newly added seed use case and other seed use cases in the seed queue can be calculated. The existing seed use cases can use the path similarity calculated previously.

[0056] Step 103, determining a target seed use case from the seed queue based on the path similarity;

[0057] The greater the path similarity, the more areas are tested repeatedly. Therefore, the target seed use case can be determined based on the path similarity seed queue, so that the test area can be expanded, so that more areas can be covered, the effectiveness of the test can be improved, and the omission of the test can be avoided.

[0058] Step 104, determining energy allocation to the target seed use case based on the coverage of the target seed use case;

[0059] Then determine the coverage of the target seed case, and allocate energy to the target seed case based on the coverage. The greater the energy, the more mutations there are, so that more different tests can be performed.

[0060] Step 105, generating a test case based on the target seed case after energy allocation;

[0061] The target seed cases after energy allocation are processed to generate actual test cases.

[0062] Step 106: Perform testing using the test case to generate vulnerability information.

[0063] Use test cases to test the program based on the test requirements and processes, detect various vulnerabilities of the program, and generate vulnerability information. The defects of the program are displayed through vulnerability information so that relevant personnel can understand the defects of the program.

[0064] The embodiment of the present invention obtains a seed queue; calculates the path similarity between any seed case in the seed queue and other seed cases in the seed queue; determines a target seed case from the seed queue based on the path similarity; determines the energy to be allocated to the target seed case based on the coverage of the target seed case; generates a test case based on the target seed case after energy allocation; and uses the test case for testing to generate vulnerability information. Seed cases are selected based on path similarity, and seeds with low-frequency paths are more likely to be executed during subsequent runtime; and energy allocation is guided based on coverage, providing more opportunities for blurring of test cases that are more likely to find errors and crashes. By optimizing the seed screening strategy and the energy allocation strategy, the ability to discover errors and crashes is significantly improved.

[0065] Reference Figure 2 , shows a flowchart of another program testing method embodiment of the present invention, the program testing method may specifically include the following steps:

[0066] Step 201, obtaining a seed queue;

[0067] The initial input provided by the user is run in the fuzzer and added to the seed queue to participate in subsequent fuzz testing. The fuzzer can be AFL (American Fuzzy Lop, a coverage-guided fuzzer). The corresponding seed queue can be obtained.

[0068] Step 202, calculating the path similarity between any seed use case in the seed queue and other seed use cases in the seed queue;

[0069] The path similarity between any seed use case in the seed queue and other seed use cases in the same seed queue can be calculated, and seed use cases of low-frequency paths can be discovered through path similarity.

[0070] In an optional embodiment of the present invention, the step of calculating the path similarity between any seed use case in the seed queue and other seed use cases in the seed queue includes:

[0071] Sub-step S2021, calculating the Hamming distance between any seed use case in the seed queue and other seed use cases in the seed queue;

[0072] In the embodiment of the present invention, the path similarity can be characterized by the Hamming distance by calculating the Hamming distance between any seed use case in the seed queue and other seed use cases in the same seed queue.

[0073] Specifically, the step of calculating the Hamming distance between any seed use case in the seed queue and other seed use cases in the seed queue includes: determining any seed use case in the seed queue as the current calculation use case, and other seed use cases in the seed queue as the use cases to be compared; performing an XOR calculation on the bitmap corresponding to the current calculation use case and the bitmap corresponding to the use case to be compared to determine the relationship between the edges; performing an AND operation based on the relationship between the edges to determine the number of different bits; and determining the number of different bits as the Hamming distance of the current calculation use case.

[0074] It should be noted that the edges of the program can be Figure 3 As shown, it includes multiple code basic blocks and corresponding stub codes, such as code basic block A, code basic block B, code basic block C, code basic block D, code basic block F, code basic block G, code basic block H, code basic block I, and code basic block Z. The execution path can be ACFHZ; and the combination of two code basic blocks is called an edge, such as: AC, CF, FH, HZ.

[0075] First, any seed case in the seed queue can be determined as the current calculation case, and the current calculation case is the seed case for comparison with other seed cases. Other seed cases in the seed queue are to-be-compared cases, which are used to compare with the current calculation case.

[0076] Perform bitwise XOR calculation on the bitmap corresponding to the current calculation case and the bitmap corresponding to the case to be compared to determine the edge relationship, that is, determine the edge relationship of the seed. Then perform bitwise AND operation based on the edge relationship to determine the number of different bits; that is, the number of different edge relationships. Based on the number of different and similar edge relationships, the difference between the two edges can be determined, and the number of different bits can be determined as the Hamming distance of the current calculation case.

[0077] For example, a corpus queue is used to save the initial input and the test cases that trigger the new state. The queue is used as input to execute the code and perform state analysis. A bitmap is used to record the edge information (such as the number of edge hits) to obtain the execution flow and code coverage of the target program. If the test case executes a new edge or a new edge hit occurs, the corresponding test case is added to the queue. And by maintaining the top_rate (attribute) array, the test case set with the lowest cost is selected. The test cases in the array can trigger all the edges that have been triggered currently.

[0078] When a test case mutates, a new trace_bits array is generated, which records which edges the test case accessed. Each bit of the trace_mini attribute of each test case corresponds to an edge in the bitmap. If the test case accesses an edge in the bitmap, the corresponding bit of trace_mini is set to 1. Algorithm 2 extracts the relationship between different edges through XOR (shown in Formula 1), and then uses the bitwise operation "AND" (shown in Formula 2) to calculate the number of different bits in the trace_mini of two test cases, that is, the Hamming distance.

[0079]

[0080] Where Sx[i], Tx[i] is the i-th bit in the attribute trace_mini of test cases S and T (i.e., an edge in the bitmap), tmp is the comparison result of the two test cases on this bit, and distance is the Hamming distance between the two test cases.

[0081] Sub-step S2022, determining the minimum value of the Hamming distance;

[0082] The Hamming distance of the minimum value among these Hamming distances can be determined as the Hamming distance of the seed case.

[0083] Sub-step S2023, determining the path similarity based on the minimum value.

[0084] Then, since the larger the Hamming distance is, the smaller the similarity is, the path similarity can be determined based on the minimum Hamming distance based on the negative correlation between the Hamming distance and the path similarity.

[0085] Specifically, the step of determining the path similarity based on the minimum value includes: calculating the reciprocal of the minimum value as the path similarity.

[0086] For the calculation of path similarity, due to the negative correlation between the Hamming distance and the path similarity, the inverse of the minimum Hamming distance can be calculated as the path similarity.

[0087] For example, after a new test case is generated, the path similarity between them is evaluated by calculating the Hamming distance between it and the execution path of the test cases in the queue, and the minimum Hamming distance is calculated. The inverse of the minimum Hamming distance (min_dis) is used as the path similarity of the execution path of the test case, as shown in formula (3).

[0088]

[0089] Step 203, determining a target seed use case from the seed queue based on the path similarity;

[0090] Based on the path similarity, the target seed cases for fuzzification are filtered out from the seed queue, so that low-frequency paths can be used for testing.

[0091] In an optional embodiment of the present invention, the step of determining the target seed use case from the seed queue based on the path similarity includes: determining the seed use case with the smallest path similarity in the seed queue as the target seed use case.

[0092] In an embodiment of the present invention, when performing seed selection, test cases with low similarity have higher priority. The larger the minimum Hamming distance of the current test case, the lower the similarity between its execution path and the execution paths of other test cases, that is, the input of the execution low-frequency path is preferentially selected. The seed case with the smallest path similarity in the seed queue can be determined as the target seed case.

[0093] In addition, when there are two or more seed cases with the same path similarity, the seed case with the smallest cost can be selected as the target seed case to participate in the next round of fuzzification based on the cost of the seed case. The cost of the seed case is the number of test cases generated after mutation, which can be represented by execution time and case length.

[0094] Step 204, determining energy allocation to the target seed use case based on the coverage of the target seed use case;

[0095] The coverage of the target seed case can be determined, and energy can be allocated to the target seed case based on the size of the coverage. The energy of the seed case refers to the number of times the seed case is mutated (or blurred). The greater the energy of the seed case, the greater the number of mutations (or blurs).

[0096] Step 205, generating a test case based on the target seed case after energy allocation;

[0097] In this embodiment, by maintaining a bitmap_size array (the edge is recorded only once for repeated executions), energy can be allocated to the generated test cases to determine the number of test cases that can be generated by mutation next time. The larger the bitmap_size, the higher the coverage of the test case, and the higher the energy is allocated to it, that is, the greater the coverage, the greater the chance of being selected. Based on the target seed case after energy allocation, the target seed case can be mutated (or fuzzy) to generate test cases for testing.

[0098] In an optional embodiment of the present invention, the step of determining the energy allocation to the target seed use case based on the coverage of the target seed use case includes: determining the total number of edges executed by the target seed use case as the coverage; allocating energy to the target seed use case based on the allocation mapping relationship between the coverage and a preset energy; wherein the allocation mapping relationship is negatively correlated.

[0099] The total number of edges executed by the target seed use case can be determined as the coverage rate; and based on the allocation mapping relationship between the coverage rate and the preset energy, the energy value corresponding to the coverage rate is determined, and the energy value is allocated to the target seed use case, thereby completing the energy allocation of the target seed use case. Among them, the allocation mapping relationship is negatively correlated. That is, the larger the coverage rate, the smaller the corresponding energy; conversely, the smaller the coverage rate, the larger the corresponding energy.

[0100] For example, the total number of edges executed by the test case (the number of times the edge is repeated is recorded) bitmap_size_dup can be calculated. If the bitmap_size_dup array value is larger, it means that the test case is fuzzified more times, and lower energy is allocated to it. That is, the smaller the number of fuzzification times, the higher the allocated energy.

[0101] Step 206: Perform testing using the test case to generate vulnerability information.

[0102] After obtaining the test cases, you can use the test cases to perform relevant tests on the program, identify relevant defects, and generate vulnerability information.

[0103] Step 207: Repair the vulnerability according to the vulnerability information.

[0104] These vulnerability information can be used to repair program vulnerabilities, thereby further improving the reliability of the program.

[0105] The embodiment of the present invention obtains a seed queue; calculates the path similarity between any seed case in the seed queue and other seed cases in the seed queue; determines the target seed case from the seed queue based on the path similarity; determines the energy to be allocated to the target seed case based on the coverage of the target seed case; generates a test case based on the target seed case after energy allocation; uses the test case for testing to generate vulnerability information; and performs vulnerability repair based on the vulnerability information. The seed selection strategy is optimized by evaluating the similarity of the execution path based on the Hamming distance, executing the low-frequency path, and discovering deeper errors or crashes. The energy allocation strategy based on the path coverage realizes adaptive energy allocation, so that seeds that can discover errors or crashes have more opportunities to be blurred, thereby improving the efficiency of vulnerability discovery.

[0106] Furthermore, in order to evaluate the effects of the embodiments of the present invention, the following experiments were conducted for evaluation.

[0107] The experimental environment is a virtual machine with an operating system of Ubuntu, a disk size of 20GB, and a memory of 4G. By comparing with the standard AFL, the embodiment of the present invention selects five applications, namely xmllint, bmp2tiff, tiff2bw, pdftotext, and pdftoppm, as the target programs of the experiment. The detailed information of these target programs is shown in Table 1.

[0108]

[0109] Table 1

[0110] The initial seed input is constructed using several types of files. For the xmllint application, the files named dtd* in the test folder of libxml2 are used; for the pdftotext and pdftoppm applications, the pdf dataset in aflsmart is used; for bmp2tiff, the bmp format files are downloaded from the AFL official website, and then the dataset is obtained after being simplified using afl-cmin; and for the tiff2bw application, the dataset for testing libtiff is downloaded from github[3]. The details of each initial input set are shown in Table 2:

[0111]

[0112] Table 2

[0113] The effectiveness of the embodiment of the present invention is verified by comparing the tool implemented by the embodiment of the present invention with the standard AFL, and the initial input is the file shown in Table 2. Each application is tested 4 times, each time for 24 hours, and then the average of the four results is taken as the fuzzy test result. Table 3 shows the specific data of the comparative experiment.

[0114]

[0115] Table 3

[0116] It can be seen that the method after improving the search strategy and energy allocation has certain improvements in the total number of paths, path depth, and crash number. For xmllint, compared with the original AFL, the total number of paths in the embodiment of the present invention has increased by 82.5%, the path depth has increased by 19 layers, and the number of crashes has increased by 82.5%; for pdftotext, the total number of paths in the embodiment of the present invention has increased by 3.3%, and the number of crashes has increased by 5.6%; for pdftoppm, the total number of paths in the embodiment of the present invention has increased by 10.3%, and for bmp2tiff, the total number of paths in the embodiment of the present invention has increased by 9.8%, and the number of crashes has increased by 29.0%; for tiff2bw, the total number of paths in the embodiment of the present invention has increased by 31.5%, the number of layers has increased by 1 layer, and the number of crashes has increased by 123%.

[0117] In order to make the implementation process of the embodiment of the present invention clear to those skilled in the art, an example may be used for illustration:

[0118] You can refer to Figure 4 First, the fuzzer runs the initial input provided by the user and adds it to the seed queue to participate in subsequent fuzz testing. To improve the efficiency of fuzz testing, in the fuzzy big loop, a seed selection strategy based on path similarity is used to select the next test case to be mutated, so that the test cases that execute low-frequency paths have higher priority and are more likely to be selected for mutation. In this way, it avoids wasting time and resources by executing a large number of redundant test cases, and the test cases that mutate the execution of low-frequency paths are more likely to reach deeper code areas and trigger deeper crashes.

[0119] Then, before the test cases mutate, energy is allocated to them according to their execution time and file length. In order to improve the efficiency of program path and vulnerability discovery, a coverage-based energy allocation strategy is introduced to optimize the energy scheduling of test cases based on their execution time and length, so that test cases with fewer mutations have higher priority, and test cases that execute low-frequency paths have longer mutation times, so as to generate more test cases to discover deeper crashes and errors.

[0120] During the program running process, the program execution information is obtained by using the instrumentation, and feedback information such as bitmap and path similarity is recorded to guide subsequent decision making. During the mutation process, if the mutated test case executes a new path, the test case is added to the queue and its path similarity is updated. If the mutated test case does not execute a new path, the test case is discarded.

[0121] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0122] Reference Figure 5 , shows a structural block diagram of an embodiment of a program testing device of the present invention, which may specifically include the following modules:

[0123] An acquisition module 501 is used to acquire a seed queue;

[0124] A path similarity module 502, used to calculate the path similarity between any seed use case in the seed queue and other seed use cases in the seed queue;

[0125] A target determination module 503, configured to determine a target seed use case from the seed queue based on the path similarity;

[0126] An allocation module 504, configured to determine energy allocation to the target seed use case based on the coverage of the target seed use case;

[0127] A generating module 505, for generating a test case based on the target seed case after energy allocation;

[0128] The testing module 506 is used to perform testing using the test case to generate vulnerability information.

[0129] In an optional embodiment of the invention, the path similarity module 502 includes:

[0130] A first calculation submodule, configured to calculate the Hamming distance between any seed use case in the seed queue and other seed use cases in the seed queue;

[0131] A minimum determination submodule, used to determine the minimum value of the Hamming distance;

[0132] The path similarity submodule is used to determine the path similarity based on the minimum value.

[0133] In an optional embodiment of the invention, the path similarity submodule includes:

[0134] The path similarity unit is used to calculate the reciprocal of the minimum value as the path similarity.

[0135] In an optional embodiment of the invention, the first calculation submodule includes:

[0136] A definition unit, used to determine any seed use case in the seed queue as a current calculation use case, and other seed use cases in the seed queue as use cases to be compared;

[0137] An XOR calculation unit, used to perform an XOR calculation on the bitmap corresponding to the current calculation case and the bitmap corresponding to the case to be compared, to determine the relationship between the edges;

[0138] An AND operation unit, used for performing an AND operation based on the relationship between the edges to determine the number of different bits;

[0139] A Hamming distance unit is used to determine the Hamming distance of the different bit quantities as the current calculation use case.

[0140] In an optional embodiment of the invention, the target determination module 503 includes:

[0141] The target determination submodule is used to determine the seed use case with the smallest path similarity in the seed queue as the target seed use case.

[0142] In an optional embodiment of the invention, the allocation module 504 includes:

[0143] A coverage determination submodule, used to determine the total number of edges executed by the target seed use case as coverage;

[0144] An allocation submodule is used to allocate energy to the target seed use case based on an allocation mapping relationship between the coverage rate and a preset energy; wherein the allocation mapping relationship is negatively correlated.

[0145] In an optional embodiment of the invention, the device further comprises:

[0146] Repair the vulnerability based on the vulnerability information.

[0147] The embodiment of the present invention obtains a seed queue; calculates the path similarity between any seed case in the seed queue and other seed cases in the seed queue; determines a target seed case from the seed queue based on the path similarity; determines the energy to be allocated to the target seed case based on the coverage of the target seed case; generates a test case based on the target seed case after energy allocation; and uses the test case for testing to generate vulnerability information. Seed cases are selected based on path similarity, and seeds with low-frequency paths are more likely to be executed during subsequent runtime; and energy allocation is guided based on coverage, providing more opportunities for blurring of test cases that are more likely to find errors and crashes. By optimizing the seed screening strategy and the energy allocation strategy, the ability to discover errors and crashes is significantly improved.

[0148] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0149] Reference Figure 6 , an embodiment of the present invention further provides an electronic device, including:

[0150] A processor 601 and a storage medium 602, wherein the storage medium 602 stores a computer program executable by the processor 601, and when the electronic device is running, the processor 601 executes the computer program to perform the program testing method as described in any one of the embodiments of the present invention.

[0151] The path similarity between any seed case in the program test method subqueue and other seed cases in the seed queue;

[0152] determining a target seed use case from the seed queue based on the path similarity;

[0153] Based on the coverage of the target seed use case, determine to allocate energy to the target seed use case;

[0154] Generate test cases based on the target seed cases after energy allocation;

[0155] The test case is used to perform testing to generate vulnerability information.

[0156] Optionally, the step of calculating the path similarity between any seed use case in the seed queue and other seed use cases in the seed queue includes:

[0157] Calculating the Hamming distance between any seed case in the seed queue and other seed cases in the seed queue;

[0158] Determining a minimum value among the Hamming distances;

[0159] A path similarity is determined based on the minimum value.

[0160] Optionally, the step of determining the path similarity based on the minimum value includes:

[0161] The reciprocal of the minimum value is calculated as the path similarity.

[0162] Optionally, the step of calculating the Hamming distance between any seed use case in the seed queue and other seed use cases in the seed queue includes:

[0163] Determine any seed use case in the seed queue as a current calculation use case, and other seed use cases in the seed queue as use cases to be compared;

[0164] Performing an XOR calculation on the bitmap corresponding to the current calculation case and the bitmap corresponding to the case to be compared to determine the relationship between the edges;

[0165] Perform an AND operation based on the relationship between the edges to determine the number of different bits;

[0166] The different bit numbers are determined as the Hamming distance of the current calculation case.

[0167] Optionally, the step of determining a target seed use case from the seed queue based on the path similarity includes:

[0168] The seed case with the smallest path similarity in the seed queue is determined as the target seed case.

[0169] Optionally, the step of determining energy allocation to the target seed use case based on the coverage of the target seed use case includes:

[0170] Determine the total number of edges executed by the target seed use case as coverage;

[0171] Based on the allocation mapping relationship between the coverage rate and the preset energy, energy is allocated to the target seed use case; wherein the allocation mapping relationship is negatively correlated.

[0172] Optionally, the method further comprises:

[0173] Repair the vulnerability based on the vulnerability information.

[0174] The embodiment of the present invention obtains a seed queue; calculates the path similarity between any seed case in the seed queue and other seed cases in the seed queue; determines a target seed case from the seed queue based on the path similarity; determines the energy to be allocated to the target seed case based on the coverage of the target seed case; generates a test case based on the target seed case after energy allocation; and uses the test case for testing to generate vulnerability information. Seed cases are selected based on path similarity, and seeds with low-frequency paths are more likely to be executed during subsequent runtime; and energy allocation is guided based on coverage, providing more opportunities for blurring of test cases that are more likely to find errors and crashes. By optimizing the seed screening strategy and the energy allocation strategy, the ability to discover errors and crashes is significantly improved.

[0175] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the processor.

[0176] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0177] Reference Figure 7 The embodiment of the present invention further provides a computer-readable storage medium 701, on which a computer program is stored. When the computer program is run by a processor, the program testing method as described in any one of the embodiments of the present invention is executed.

[0178] The path similarity between any seed case in the program test method subqueue and other seed cases in the seed queue;

[0179] determining a target seed use case from the seed queue based on the path similarity;

[0180] Based on the coverage of the target seed use case, determine to allocate energy to the target seed use case;

[0181] Generate test cases based on the target seed cases after energy allocation;

[0182] The test case is used to perform testing to generate vulnerability information.

[0183] Optionally, the step of calculating the path similarity between any seed use case in the seed queue and other seed use cases in the seed queue includes:

[0184] Calculating the Hamming distance between any seed case in the seed queue and other seed cases in the seed queue;

[0185] Determining a minimum value among the Hamming distances;

[0186] A path similarity is determined based on the minimum value.

[0187] Optionally, the step of determining the path similarity based on the minimum value includes:

[0188] The reciprocal of the minimum value is calculated as the path similarity.

[0189] Optionally, the step of calculating the Hamming distance between any seed use case in the seed queue and other seed use cases in the seed queue includes:

[0190] Determine any seed use case in the seed queue as a current calculation use case, and other seed use cases in the seed queue as use cases to be compared;

[0191] Performing an XOR calculation on the bitmap corresponding to the current calculation case and the bitmap corresponding to the case to be compared to determine the relationship between the edges;

[0192] Perform an AND operation based on the relationship between the edges to determine the number of different bits;

[0193] The different bit numbers are determined as the Hamming distance of the current calculation case.

[0194] Optionally, the step of determining a target seed use case from the seed queue based on the path similarity includes:

[0195] The seed case with the smallest path similarity in the seed queue is determined as the target seed case.

[0196] Optionally, the step of determining energy allocation to the target seed use case based on the coverage of the target seed use case includes:

[0197] Determine the total number of edges executed by the target seed use case as coverage;

[0198] Based on the allocation mapping relationship between the coverage rate and the preset energy, energy is allocated to the target seed use case; wherein the allocation mapping relationship is negatively correlated.

[0199] Optionally, the method further comprises:

[0200] Repair the vulnerability based on the vulnerability information.

[0201] The embodiment of the present invention obtains a seed queue; calculates the path similarity between any seed case in the seed queue and other seed cases in the seed queue; determines a target seed case from the seed queue based on the path similarity; determines the energy to be allocated to the target seed case based on the coverage of the target seed case; generates a test case based on the target seed case after energy allocation; and uses the test case for testing to generate vulnerability information. Seed cases are selected based on path similarity, and seeds with low-frequency paths are more likely to be executed during subsequent runtime; and energy allocation is guided based on coverage, providing more opportunities for blurring of test cases that are more likely to find errors and crashes. By optimizing the seed screening strategy and the energy allocation strategy, the ability to discover errors and crashes is significantly improved.

[0202] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0203] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0204] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0205] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0206] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0207] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0208] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0209] The program testing method, program testing device, electronic device and storage medium provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A program testing method, characterized in that: include: Get the seed queue; Calculating the path similarity between any seed use case in the seed queue and other seed use cases in the seed queue; determining a target seed use case from the seed queue based on the path similarity; Based on the coverage of the target seed use case, determine to allocate energy to the target seed use case; Generate test cases based on the target seed cases after energy allocation; The test case is used to perform testing to generate vulnerability information.

2. The method according to claim 1, characterized in that The step of calculating the path similarity between any seed use case in the seed queue and other seed use cases in the seed queue comprises: Calculating the Hamming distance between any seed case in the seed queue and other seed cases in the seed queue; Determining a minimum value among the Hamming distances; A path similarity is determined based on the minimum value.

3. The method according to claim 2, characterized in that The step of determining the path similarity based on the minimum value comprises: The reciprocal of the minimum value is calculated as the path similarity.

4. The method according to claim 2, characterized in that: The step of calculating the Hamming distance between any seed use case in the seed queue and other seed use cases in the seed queue comprises: Determine any seed use case in the seed queue as a current calculation use case, and other seed use cases in the seed queue as use cases to be compared; Performing an XOR calculation on the bitmap corresponding to the current calculation case and the bitmap corresponding to the case to be compared to determine the relationship between the edges; Perform an AND operation based on the relationship between the edges to determine the number of different bits; The different bit numbers are determined as the Hamming distance of the current calculation case.

5. The method according to claim 1, characterized in that The step of determining a target seed use case from the seed queue based on the path similarity comprises: The seed case with the smallest path similarity in the seed queue is determined as the target seed case.

6. The method according to claim 1, characterized in that The step of determining energy allocation to the target seed use case based on the coverage of the target seed use case includes: Determine the total number of edges executed by the target seed use case as coverage; Based on the allocation mapping relationship between the coverage rate and the preset energy, energy is allocated to the target seed use case; wherein the allocation mapping relationship is negatively correlated.

7. The method according to claim 1, characterized in that The method further comprises: Repair the vulnerability based on the vulnerability information.

8. A program testing device, characterized in that: include: The acquisition module is used to obtain the seed queue; A path similarity module, used to calculate the path similarity between any seed use case in the seed queue and other seed use cases in the seed queue; a target determination module, configured to determine a target seed use case from the seed queue based on the path similarity; An allocation module, configured to determine energy allocation to the target seed use case based on the coverage of the target seed use case; A generation module, used for generating test cases based on target seed cases after energy allocation; The test module is used to perform testing using the test case and generate vulnerability information.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the program testing method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of program testing according to any one of claims 1 to 7 are implemented.

Images