Fuzzy test optimization method and device based on test case filtering

Abstract

The invention discloses a fuzzy test optimization method and device based on test case filtering, and the method comprises the steps: training a prediction model, starting a fuzzy test tool to test a tested target, collecting a test case and corresponding execution path data as a training data set, and carrying out the supervised model training through a recurrent neural network model, constructing a prediction model; applying the prediction model, starting the fuzzy test tool for testing, recording a test case and corresponding execution path data, counting the test coverage rate in real time, and when the fuzzy test tool generates a new test case, generating a prediction path based on the prediction model; for the prediction path, outputting a frequency score S; aiming at the frequency score S, filtering based on a filtering rule, and reserving a high-value low-frequency test case; and testing by using the low-frequency test case. The invention aims to filter out high-frequency test cases which are ineffective for improving the test coverage rate, and retain low-frequency test cases with rare paths, so that the test efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fuzzy testing, and in particular to a fuzzy testing optimization method and device based on test case filtering. Background Art

[0002] Fuzz testing technology based on test coverage is one of the most popular and efficient software testing technologies. This technology combines the advantages of fuzz testing and path coverage analysis. By tracking the execution path of the program, it attempts to generate test cases that can trigger new paths, thereby continuously improving test coverage and increasing the chances of vulnerability discovery.

[0003] However, the existing fuzz testing technology based on test coverage still has certain defects. With the in-depth advancement of fuzz testing, the test cases generated by fuzz tools are mostly concentrated on the high-frequency paths of the program, and the number of test cases that can trigger new paths is far less than the total number of generated test cases. This situation leads to a lot of meaningless time costs and operating overheads being wasted during the testing process, reducing the efficiency of fuzz testing, making it difficult to significantly improve the test coverage, thereby limiting the efficiency of vulnerability discovery. Summary of the invention

[0004] In view of the problems and shortcomings of the prior art, the present invention provides a fuzzy testing optimization method and device based on test case filtering, which aims to filter out high-frequency test cases that are ineffective in improving test coverage and retain low-frequency test cases of rare paths, thereby improving test efficiency.

[0005] The embodiment of the present invention provides a fuzzy test optimization method based on test case filtering, which is characterized in that the method includes:

[0006] Prediction model training: start the fuzz testing tool to test the target, collect test cases and their corresponding execution path data as training data sets, and use the recurrent neural network model to perform supervised model training based on the training data to build a prediction model for predicting the test case execution path;

[0007] The prediction model is applied to start the fuzz testing tool to test the target under test, record the test cases and their corresponding execution path data, and calculate the test coverage in real time. When the fuzz testing tool generates a new test case, the execution path of the new test case is predicted based on the prediction model to generate a predicted path;

[0008] Scoring the predicted path based on the scoring rules and outputting a frequency score S;

[0009] For the frequency score S, filtering is performed based on filtering rules to remove low-value high-frequency test cases and retain high-value low-frequency test cases;

[0010] The low-frequency test case is used to further test the target under test.

[0011] Preferably, the fuzz testing tool is used to discover potential execution paths for the target under test by generating random test cases.

[0012] Preferably, the trained prediction model predicts the execution path for each new test case generated by the fuzz testing tool.

[0013] Preferably, the scoring rule scores the occurrence frequency of the predicted path based on the historical execution path data of the tested target collected during the current test.

[0014] Preferably, the scoring rules include:

[0015] Each predicted path corresponds to an edge in the logic tree of the measured target. The predicted path is divided into n segmented paths based on two adjacent nodes in the edge. The number of segments of each segmented path is X. n is the number of segments marked in sequence from the starting point of the predicted path in the execution order of the path, the number of segments X n Increasing successively, said n is a positive integer;

[0016] The calculation formula of the execution frequency value of one of the segmented paths is as follows:

[0017]

[0018] The frequency score S of the predicted path is calculated as follows:

[0019]

[0020] in:

[0021] c n is the total number of times the nth segment of one of the segmented paths appears in all the historical execution paths;

[0022] C n It is the total number of occurrences of all possible segment paths of the nth segment in all the historical execution paths.

[0023] Preferably, a higher frequency score S indicates that the predicted path is rarer, and the value of the corresponding new test case is higher;

[0024] The lower the frequency score S is, the higher the frequency of the predicted path is, and the lower the value of the corresponding new test case is.

[0025] Preferably, the filtering rule is to compare the frequency score S with a preset threshold value, and when the frequency score S is less than or equal to the threshold value, the new test case is a low-value high-frequency test case, and the high-frequency test case is removed;

[0026] When the frequency score S is greater than the threshold, the new test case is a high-value low-frequency test case, and the low-frequency test case is retained.

[0027] The embodiment of the present invention also provides a fuzzy test optimization device based on test case filtering, the device comprising:

[0028] A prediction model training module is used to start the fuzz testing tool to test the target, collect test cases and their corresponding execution path data as a training data set, and based on the training data, use a recurrent neural network model to perform supervised model training to build a prediction model for predicting the test case execution path;

[0029] A prediction model application module is used to start the fuzz testing tool to test the target, record the test cases and their corresponding execution path data, and count the test coverage in real time. When the fuzz testing tool generates a new test case, the execution path of the new test case is predicted based on the prediction model to generate a predicted path.

[0030] A frequency score output module, which is used to score the predicted path based on the scoring rules and output the frequency score S;

[0031] A high-value low-frequency test case acquisition module, which is used to filter the frequency score S based on the filtering rules, remove low-value high-frequency test cases, and retain high-value low-frequency test cases;

[0032] A further testing module is used to perform further testing on the target under test using the low-frequency test case.

[0033] An embodiment of the present invention further provides an electronic device, including:

[0034] A processor; and a memory for storing executable instructions for the processor.

[0035] An embodiment of the present invention further provides a computer-readable storage medium, in which computer-executable instructions are stored.

[0036] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:

[0037] A fuzzy test optimization method and device based on test case filtering, the method comprising: predictive model training, starting a fuzzy test tool to test a target under test, collecting test cases and their corresponding execution path data as a training data set, based on the training data, using a recurrent neural network model to perform supervised model training, and constructing a predictive model for predicting the test case execution path; predictive model application, starting a fuzzy test tool to test the target under test, recording test cases and their corresponding execution path data, and real-time statistical test coverage, when the fuzzy test tool generates a new test case, predicting the execution path of the new test case based on the predictive model to generate a predicted path; scoring the predicted path based on a scoring rule, and outputting a frequency score S; filtering the frequency score S based on a filtering rule, removing low-value high-frequency test cases, and retaining high-value low-frequency test cases; and using the low-frequency test cases to further test the target under test. The present invention realizes the filtering of a large number of high-frequency path test cases generated in the fuzzy testing process based on test coverage, effectively reduces the invalid testing time caused by high-frequency path test cases, improves the test coverage, increases the possibility of vulnerability discovery, and significantly improves the efficiency of fuzzy testing.

[0038] Furthermore, the present invention performs supervised model training on the recurrent neural network model based on the recorded test cases and their corresponding execution path data, and performs automatic and intelligent prediction on the execution path of new test cases generated by the fuzzy testing tool based on the trained prediction model.

[0039] Furthermore, the present invention scores the predicted path, removes low-value high-frequency test cases based on filtering rules, retains high-value low-frequency test cases, and performs further fuzz testing. By filtering out low-value high-frequency test cases, the test coverage of the fuzz test is improved, thereby improving the efficiency of the fuzz test. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present invention, not all embodiments. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0041] Figure 1 The figure is a flow chart of a fuzzy testing optimization method based on test case filtering according to a preferred embodiment of the present invention.

[0042] Figure 2 Schematic diagram illustrating the prediction path scoring rule of a preferred embodiment of the present invention.

[0043] Figure 3 Schematic diagram of the process of training the prediction model according to a preferred embodiment of the present invention.

[0044] Figure 4 Schematic diagram of the process of testing based on the prediction model according to a preferred embodiment of the present invention.

[0045] Figure 5 A module diagram of a fuzzy test optimization device based on test case filtering according to a preferred embodiment of the present invention.

[0046] Figure 6 FIG. 4 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0048] The technical solution of the present invention is described in detail below with reference to specific embodiments.

[0049] Based on the problems existing in the prior art, an embodiment of the present invention provides a fuzzy testing optimization method and device based on test case filtering, which aims to filter out high-frequency test cases that are ineffective in improving test coverage and retain low-frequency test cases of rare paths, thereby improving test efficiency.

[0050] like Figure 1 As shown, this embodiment provides a fuzzy test optimization method based on test case filtering, the method comprising:

[0051] Step S101: Prediction model training, start the fuzz testing tool to test the target, collect test cases and their corresponding execution path data as training data sets, and use the recurrent neural network model to perform supervised model training based on the training data to build a prediction model for predicting the test case execution path.

[0052] Among them, the fuzz testing tool is used to discover potential execution paths by generating random test cases for the target under test.

[0053] Step S102: Apply the prediction model, start the fuzz testing tool to test the target, record the test cases and their corresponding execution path data, and count the test coverage in real time. When the fuzz testing tool generates a new test case, the execution path of each new test case generated by the fuzz testing tool is predicted based on the trained prediction model to generate a predicted path.

[0054] Step S103: Score the predicted path based on the scoring rules and output a frequency score S.

[0055] The scoring rule is to score the occurrence frequency of the predicted path based on the historical execution path data of the target under test during this test. Further, the scoring rule is:

[0056] Each predicted path corresponds to an edge in the logic tree of the target to be measured. Based on two adjacent nodes in an edge, the predicted path is divided into n segments. The number of segments in each segment path is X. n is the number of segments marked in the order of execution of the path starting from the starting point of the predicted path, the number of segments X n Increment successively, and n is a positive integer.

[0057] The execution frequency value of a segment path is calculated as follows:

[0058]

[0059] The frequency score S of the predicted path is calculated as follows:

[0060]

[0061] in:

[0062] c n is the total number of times one of the segmented paths appears in the nth segment in all historical execution paths;

[0063] C n The total number of times all possible segment paths of segment n appear in all historical execution paths.

[0064] For example, Figure 2 A in the figure shows the execution process of a certain target under test, and the corresponding logic tree is as follows Figure 2 As shown in B, the predicted path is an edge (a, b, c) in the logic tree. This edge is divided into two segmented paths by two adjacent nodes, namely the segmented path (a, b) and the segmented path (b, c). The number of segments of the segmented path (a, b) in the predicted path is X 1 1; the number of segments of the segmented path (b, c) in the predicted path is X 2 is 2.

[0065] Among them, the segmented path (a, b) may be high frequency; the segmented path (b, c) may be low frequency. If you want to calculate the frequency score S of the predicted path (a, b, c), you need to calculate the frequency value F of the segmented path (a, b) and the segmented path (b, c). n , and then the frequency values ​​F of the segmented path (a, b) and the segmented path (b, c) n A weighted sum is performed to calculate the frequency score S of the predicted path (a, b, c).

[0066] The total number of times the segment path (a, b) appears in the first segment in all historical data is recorded as c 1 .

[0067] C 1 is the number of occurrences of all possible segment paths of segment 1 in all historical execution paths, Figure 2 Take B in the example, C 1 is the total number of occurrences of segmented paths (a, b) and segmented paths (a, e).

[0068] c 1 / C 1 is the frequency of the first segment of the segment path (a, b) in all historical execution paths, expressed as X 1 As a weight, the execution frequency value of the segment path (a, b) is calculated as follows:

[0069]

[0070] The total number of times the segment path (b, c) appears in the second segment in all historical data is recorded as c 2 .

[0071] C 2 is the number of possible segment paths of segment 2 in all historical execution paths, Figure 2 Take B in the example, C 2 is the total number of occurrences of segmented path (b, c) and segmented path (b, d).

[0072] c 2 / C 2 is the frequency of the second segment of the segment path (b, c) in all historical execution paths, expressed as X 2 As a weight, the frequency value of the segment path (b, c) is calculated as follows:

[0073]

[0074] Calculate the frequency score S of the predicted path (a, b, c) using the following formula:

[0075]

[0076] When the calculation result, that is, the higher the frequency score S is, the rarer the predicted path is, and the higher the value of the corresponding new test case is.

[0077] When the calculation result is, that is, the lower the frequency score S is, the higher the frequency of the predicted path is, and the lower the value of the corresponding new test case is.

[0078] Step S104: filtering based on the filtering rules for the frequency score S to remove low-value high-frequency test cases and retain high-value low-frequency test cases.

[0079] Among them, the filtering rule is to compare the frequency score S with the preset threshold. When the frequency score S is less than or equal to the threshold, the new test case is a low-value high-frequency test case, and the high-frequency test case is removed; when the frequency score S is greater than the threshold, the new test case is a high-value low-frequency test case, and the low-frequency test case is retained.

[0080] Step S105: Use low-frequency test cases to further test the target under test.

[0081] like Figure 3 As shown in the figure, the workflow of prediction model training is:

[0082] Step S301: Start the fuzz testing tool.

[0083] Step S302: The fuzz testing tool performs seed mutation based on the feedback execution path.

[0084] Step S303: Generate test cases based on the mutated seeds.

[0085] Step S304: input the generated test case into the target to be tested for testing.

[0086] Step S305: During the testing process, test cases and their corresponding execution path data are collected as training data sets.

[0087] Step S306: Based on the training data, supervised model training is performed using a recurrent neural network model to construct a prediction model for predicting the execution path of the test case.

[0088] like Figure 4 As shown in the figure, the workflow for testing based on the prediction model is:

[0089] Step S401: Start the fuzz testing tool to test the target, record the test cases and their corresponding execution path data, and count the test coverage in real time.

[0090] Step S402: The fuzz testing tool performs seed mutation based on the feedback execution path.

[0091] Step S403: Generate new test cases based on the mutated seeds.

[0092] Step S404: When the fuzz testing tool generates a new test case, the execution path of the new test case is predicted based on the prediction model to generate a predicted path.

[0093] Step S405: For the predicted path, score it based on the historical execution path data of the tested target during this test, and output the frequency score S.

[0094] Step S406: Compare the frequency score S with a preset threshold.

[0095] Step S407: When the frequency score S is less than or equal to the threshold, the new test case is a low-value high-frequency test case, and the high-frequency test case is filtered out.

[0096] Step S408: When the frequency score S is greater than the threshold, the new test case is a high-value low-frequency test case, and the low-frequency test case is retained.

[0097] Step S409: Use low-frequency test cases to further test the target under test.

[0098] like Figure 5 As shown, this embodiment provides a fuzzy test optimization device based on test case filtering, the device comprising:

[0099] A prediction model training module 51 is used to start the fuzz testing tool to test the target, collect test cases and their corresponding execution path data as a training data set, and based on the training data, use a recurrent neural network model (RNN, Recurrent Neural Networks) to perform supervised model training to build a prediction model for predicting the test case execution path;

[0100] The prediction model application module 52 is used to start the fuzz testing tool to test the target, record the test cases and their corresponding execution path data, and count the test coverage in real time. When the fuzz testing tool generates a new test case, the execution path of the new test case is predicted based on the prediction model to generate a predicted path.

[0101] The frequency score output module 53 is used to score the predicted path based on the scoring rules and output the frequency score S.

[0102] The high-value low-frequency test case acquisition module 54 is used to filter the frequency score S based on the filtering rule to remove the low-value high-frequency test cases and retain the high-value low-frequency test cases.

[0103] The further testing module 55 is used to perform further testing on the target under test using low-frequency test cases.

[0104] Figure 6 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. Figure 6 As shown, an electronic device 600 provided in this embodiment includes: a processor 601 and a memory 602; wherein:

[0105] The memory 602 is used to store computer programs, and the memory may also be a flash memory.

[0106] The processor 601 is used to execute the execution instructions stored in the memory to implement each step in the above method. For details, please refer to the relevant description in the above method embodiment.

[0107] Optionally, the memory 602 may be independent or integrated with the processor 601 .

[0108] When the memory 602 is a device independent of the processor 601, the electronic device 600 may further include:

[0109] The bus 603 is used to connect the memory 602 and the processor 601 .

[0110] This embodiment further provides a readable storage medium, in which a computer program is stored. When at least one processor of an electronic device executes the computer program, the electronic device executes the methods provided in the above-mentioned various implementation modes.

[0111] This embodiment also provides a program product, which includes a computer program, which is stored in a readable storage medium. At least one processor of the electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device implements the methods provided by the various embodiments described above. The technical solution of the embodiment of the present invention has the following beneficial effects:

[0112] A fuzzy test optimization method and device based on test case filtering, the method comprising: predictive model training, starting a fuzzy test tool to test a target under test, collecting test cases and their corresponding execution path data as a training data set, based on the training data, using a recurrent neural network model to perform supervised model training, and constructing a predictive model for predicting the test case execution path; predictive model application, starting a fuzzy test tool to test the target under test, recording test cases and their corresponding execution path data, and real-time statistical test coverage, when the fuzzy test tool generates a new test case, predicting the execution path of the new test case based on the predictive model to generate a predicted path; scoring the predicted path based on a scoring rule, and outputting a frequency score S; filtering the frequency score S based on a filtering rule, removing low-value high-frequency test cases, and retaining high-value low-frequency test cases; and using the low-frequency test cases to further test the target under test. The present invention realizes the filtering of a large number of high-frequency path test cases generated in the fuzzy testing process based on test coverage, effectively reduces the invalid testing time caused by high-frequency path test cases, improves the test coverage, increases the possibility of vulnerability discovery, and significantly improves the efficiency of fuzzy testing.

[0113] Furthermore, the present invention performs supervised model training on the recurrent neural network model based on the recorded test cases and their corresponding execution path data, and performs automatic and intelligent prediction on the execution path of new test cases generated by the fuzzy testing tool based on the trained prediction model.

[0114] Furthermore, the present invention scores the predicted path, removes low-value high-frequency test cases based on filtering rules, retains high-value low-frequency test cases, and performs further fuzz testing. By filtering out low-value high-frequency test cases, the test coverage of the fuzz test is improved, thereby improving the efficiency of the fuzz test.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A fuzzy testing optimization method based on test case filtering, characterized in that: The method comprises: Prediction model training: start the fuzz testing tool to test the target, collect test cases and their corresponding execution path data as training data sets, and use the recurrent neural network model to perform supervised model training based on the training data to build a prediction model for predicting the test case execution path; The prediction model is applied to start the fuzz testing tool to test the target under test, record the test cases and their corresponding execution path data, and calculate the test coverage in real time. When the fuzz testing tool generates a new test case, the execution path of the new test case is predicted based on the prediction model to generate a predicted path; Scoring the predicted path based on the scoring rules and outputting a frequency score S; For the frequency score S, filtering is performed based on filtering rules to remove low-value high-frequency test cases and retain high-value low-frequency test cases; The low-frequency test case is used to further test the target under test.

2. A fuzzy testing optimization method based on test case filtering as described in claim 1, characterized in that: The fuzz testing tool is used to discover potential execution paths for the target under test by generating random test cases.

3. A fuzzy testing optimization method based on test case filtering as described in claim 1, characterized in that: The trained prediction model predicts the execution path for each new test case generated by the fuzz testing tool.

4. A fuzzy testing optimization method based on test case filtering as claimed in claim 1, characterized in that: The scoring rule scores the occurrence frequency of the predicted path based on the historical execution path data of the tested target during the current test.

5. A fuzzy testing optimization method based on test case filtering as described in claim 4, characterized in that: The scoring rules include: Each predicted path corresponds to an edge in the logic tree of the measured target. The predicted path is divided into n segmented paths based on two adjacent nodes in the edge. The number of segments of each segmented path is X. n is the number of segments marked in sequence from the starting point of the predicted path in the execution order of the path, the number of segments X n Increasing successively, said n is a positive integer; The calculation formula of the execution frequency value of one of the segmented paths is as follows: The frequency score S of the predicted path is calculated as follows: in: c n is the total number of times the nth segment of one of the segmented paths appears in all the historical execution paths; C n It is the total number of occurrences of all possible segment paths of the nth segment in all the historical execution paths.

6. A fuzzy testing optimization method based on test case filtering as described in claim 5, characterized in that: The higher the frequency score S is, the rarer the predicted path is, and the higher the value of the corresponding new test case is; The lower the frequency score S is, the higher the frequency of the predicted path is, and the lower the value of the corresponding new test case is.

7. A fuzzy testing optimization method based on test case filtering as described in claim 6, characterized in that: The filtering rule is to compare the frequency score S with a preset threshold value. When the frequency score S is less than or equal to the threshold value, the new test case is a low-value high-frequency test case, and the high-frequency test case is removed; When the frequency score S is greater than the threshold, the new test case is a high-value low-frequency test case, and the low-frequency test case is retained.

8. A fuzzy test optimization device based on test case filtering, characterized in that: The device comprises: A prediction model training module is used to start the fuzz testing tool to test the target, collect test cases and their corresponding execution path data as a training data set, and based on the training data, use a recurrent neural network model to perform supervised model training to build a prediction model for predicting the test case execution path; A prediction model application module is used to start the fuzz testing tool to test the target, record the test cases and their corresponding execution path data, and count the test coverage in real time. When the fuzz testing tool generates a new test case, the execution path of the new test case is predicted based on the prediction model to generate a predicted path. A frequency score output module, which is used to score the predicted path based on the scoring rules and output the frequency score S; A high-value low-frequency test case acquisition module, which is used to filter the frequency score S based on the filtering rules, remove low-value high-frequency test cases, and retain high-value low-frequency test cases; A further testing module is used to perform further testing on the target under test using the low-frequency test case.

9. An electronic device, characterized in that: include: processor; as well as, A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images