Metamorphic test group generation method based on isolated forest and adaptive random test

Abstract

The invention discloses a metamorphic test group generation method based on an isolated forest and an adaptive random test, which comprises the following steps of: firstly, generating a group of initial test cases, and judging whether defects of a to-be-tested program can be detected or not; if no program defect is detected, putting the program defect into a selected test case set; constructing an isolated forest model by using the selected test case set; thirdly, constructing a candidate test case set, judging whether a program defect is detected by the candidate original test case, and if the program defect is not detected, generating a candidate subsequent test case according to the metamorphic relationship; and calculating abnormal scores of the candidate test cases by using an isolated forest model, putting the test cases of which the abnormal scores are higher than an abnormal threshold value into a selected test case group set, and finally judging whether program defects are detected or not. Through the method provided by the invention, an effective method is provided for generating the test case of the high-dimensional input scientific calculation program in the metamorphic test, and the generation efficiency is improved while the boundary effect problem in the generation of the test case is relieved.

Description

Technical Field

[0001] The present invention relates to a test case generation method for metamorphic testing, and particularly to a method for generating metamorphic test cases based on isolation forest and adaptive random testing, belonging to the field of software testing. Background Art

[0002] Metamorphic testing is a software testing technique used to alleviate the "test oracle problem". The original intention of this testing technique is to generate additional test cases according to the metamorphic relationship based on existing test cases, and then check whether these two test cases satisfy the metamorphic relationship through the corresponding metamorphic relationship. The former test case is called the original test case (Original Test Cases: STC), and the latter test case is called the follow-up test case (Follow-up Test Cases: FTC).

[0003] In metamorphic testing, if the metamorphic relationship is determined, the random value strategy is usually used to generate the original test cases. Although this random value testing technique is simple, it does not provide useful information for test coverage during the test case generation process. Therefore, the random value strategy is blind, resulting in limited test case generation efficiency. In recent years, an adaptive random strategy has been proposed, which introduces the distance between test cases as a metric to adaptively generate the next test case, thereby improving the effectiveness and test coverage of test cases. However, in scientific computing programs with high-dimensional inputs, the adaptive random strategy based on distance leads to an increase in algorithm complexity and also presents a boundary effect due to the introduction of the distance algorithm.

[0004] Isolation forest model: It is an anomaly detection method that starts from outliers, divides according to specified rules, and makes a judgment based on the number of divisions. Summary of the Invention

[0005] Technical Problem to be Solved

[0006] In order to improve the algorithm complexity, ensure the uniform distribution of test cases, and solve its boundary effect at the same time, the purpose of the present invention is to provide a method for generating a metamorphic test suite based on isolation forest and adaptive random testing.

[0007] Research Idea of the Present Invention

[0008] Based on the original test cases generated adaptively and randomly in metamorphic testing, an isolation forest algorithm is introduced to measure the next test case. The adaptive random strategy ensures the uniform distribution of test cases, thus effectively improving the test coverage. However, its distance algorithm metric makes the algorithm complexity relatively high. The isolation forest algorithm is a machine learning algorithm for anomaly detection, with low computational complexity and the ability to handle high-dimensional data problems. Using the isolation forest algorithm as a metric can improve the algorithm complexity and solve the boundary effect of test cases.

[0009] The technical solution of the present invention:

[0010] A method for generating a metamorphic test suite based on isolation forest and adaptive random testing, comprising the following steps:

[0011] Step 1: Obtain the input domain of the program to be tested; the program to be tested contains multiple input parameters;

[0012] Step 2: Based on the program to be tested, construct a single-fault program by implanting errors, and use the comparison of the results of the program to be tested and the single-fault program as the basis for whether a program defect is detected;

[0013] Step 3: According to the parameter range of the input domain, adopt a random value strategy to generate the original test cases STC, and then generate the subsequent test cases FTC in combination with the metamorphic relationship. The original test cases STC and the subsequent test cases FTC are constructed into an initial test case set;

[0014] Step 4: Use the test cases in the initial test case set to detect whether the results of the program under test and the single-fault program are consistent. If they are inconsistent, a defect is detected; if no program defect is detected, the test cases without detected defects are put into the selected test case set. When the number of elements in the selected test case set reaches N, use the selected test case set to construct an isolation forest model;

[0015] Step 5: Divide the input domain into partitions according to the value ranges of the input parameters. In the regions that do not contain the selected test case set, adopt a random value strategy to generate candidate original test cases, and determine whether each candidate original test case detects a defect in turn. If so, proceed to detect the next candidate original test case; otherwise, proceed to Step 7;

[0016] Step 6: If a candidate original test case does not detect a defect, then generate candidate subsequent test cases in combination with the metamorphic relationship, and determine whether the candidate subsequent test cases fall into the region covered by the test cases under test. If they fall into the covered region, return to Step 5 until candidate subsequent test cases that do not fall into the region covered by the test cases under test are obtained; put the candidate original test cases and the finally obtained candidate subsequent test cases into the candidate test case set;

[0017] Step 7: Input the candidate test cases in the candidate test case set into the isolation forest model, and calculate the anomaly score S of the candidate test cases A , and select the candidate test cases with anomaly scores higher than the anomaly score threshold as the next set of test cases and add them to the selected test case set;

[0018] Step 8: Use the selected test case set to detect the program under test and check the defect detection situation. If no program defect is detected or the end condition is not met, repeat Steps 5 - 7; until a program defect is found or the end condition is met.

[0019] For further improvement, in Step 1, obtain the input domain of the program under test according to the design document of the program under test; the input parameters have their respective value ranges.

[0020] For further improvement, in Step 2, use first-order mutation operators or arithmetic operators to implant the program under test to construct a single-fault program, for example, replace "+" with "-", and "&&" with "||".

[0021] For further improvement, in Step 3, adopt a random value strategy to generate the original test case STC.

[0022] For further improvement, in Step 4, N ≥ 4.

[0023] For further improvement, in Step 8, the end condition is to reach a preset number of loops.

[0024] For further improvement, in Step 7, the anomaly score S A has the following formula:

[0025]

[0026] where x is the candidate test case, an isolation forest is constructed using the selected test case set, h(x) is the height of x in the isolation tree, E(h(x)) is the average value of h(x), and c(n) is the average height of the isolation tree when the candidate test case input is equal to n; the calculation formula is as follows: where H(i) is the harmonic number, is approximately equal to ln(i) + 0.57721, (0.57721 is the Euler constant, and n is the size of the number of test cases). c(n) can be used to normalize h(x), and the closer S A is to 1, the greater the degree of anomaly;

[0027] Beneficial effects:

[0028] The method for generating a metamorphic test suite based on isolation forest and adaptive random testing proposed by the present invention can be used to generate test cases for high-dimensional scientific computing programs, improve the efficiency of test case generation, and solve the boundary effect problem generated when test cases are evenly distributed. As Figure 2 shown, on the left is the distribution result of test cases of the MT-ART algorithm, and on the right is the result of the present algorithm. Among them, the blue dots are candidate test cases, and the red dots are executed test cases. It can be seen from the blue dots that the present algorithm effectively alleviates the boundary effect. Also, because the isolation forest algorithm is introduced, the algorithm complexity is O(n), which is fast in processing high-dimensional data and has low memory consumption, and can be used for network attack detection, financial transaction fraud detection, etc. Description of the Drawings

[0029] Figure 1 It is a flowchart of an embodiment of the present invention.

[0030] Figure 2 It is a comparison chart of test case generation results. Detailed Embodiment

[0031] The present invention will be further described in detail below with reference to the drawings and embodiments.

[0032] Referring to the attached Figure 1 , taking the TCAS program of the air traffic alert and collision avoidance system as an example, this embodiment includes the following steps: (1): Preparation work. Introduce the program under test, and determine the input domain range of the program under test TCAS as follows:

[0033]

[0034] Its metamorphic relationship is:

[0035]

[0036] And in order to obtain a single-fault program under test, the arithmetic operator replacement (AOR) is used to construct a single fault for the TCAS program.

[0037] (2) Initialization. Initialize the initial test case set OriginalTestCases, the selected test case set SelectedTestCases, the candidate test case set CandidatesTestCases, and the function TcasDetected() for detecting program defects;

[0038] (3) Construct m initial test cases. Take m as 4, generate 4 values using the random value strategy, and put them into the initial test case set OriginalTestCases. Determine whether a program defect is detected. If a program defect is detected, end.

[0039] (4) Obtain the selected test case set SelectedTestCases. If the initial test cases do not detect program defects, put them into the selected test case set. The number of selected test cases is N, where N is taken as 4. When N is reached, use the selected test case set to construct an isolation forest model.

[0040] (5) Partition the input domain of the program under test. For example, if the parameter range is (0, 2000), it is partitioned into [0, 50), [50, 100), [100, 150) ……, and other parameter ranges are partitioned by one discrete value each. For example, (0, 3) is partitioned into 0, 1, …… Only when each parameter of the input value is in a partition marked as covered is the input value judged to be in a covered partition, and the covered partition is marked as true.

[0041] (6) Put the selected test cases into their respective partitions, traverse all partitions to find the uncovered partitions, and randomly select a partition to generate a candidate original test case newSTC.

[0042] (7) Determine whether the candidate original test case newSTC can detect program defects. If program defects are detected, end.

[0043] (8) If no program defects are detected, generate a candidate additional test case newFTC, and determine whether newFTC falls into the area covered by other test cases. If it falls into the area covered by other test cases, return to (6). If newFTC falls into an uncovered area, put both newSTC and newFTC into the candidate test case group set.

[0044] (9) Finally, use the isolation forest model to calculate the anomaly score of the candidate test cases. Let the anomaly score threshold be 0.30. If it is higher than the score threshold, put it into the selected test case set and determine whether it can detect program defects. If no program defects are detected or the end condition is not reached, repeat (6) - (8) until program defects are found or the end condition is reached.

[0045] The formula for calculating the anomaly score S A in step nine is: where x is the candidate test case, an isolation forest is constructed using the selected test case set, h(x) is the height of x in the isolation tree, E(h(x)) is the average value of h(x), and c(n) is the average height of the isolation tree when the candidate test case input is equal to n. The calculation formula is as follows: c(n) = 2H(n - 1) - (2(n - 1) / n), where H(i) is the harmonic parameter, which can be estimated by ln(i) + 0.5772156649 (Euler's constant). At the same time, c(n) can be used to normalize h(x), S AThe closer to 1, the greater the degree of abnormality.

[0046] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the scope of the present invention. After reading the present invention, various equivalent forms of modification made by those skilled in the art to the present invention all fall within the scope defined by the appended claims of this application.

Claims

1. A method for generating a metamorphic test suite based on isolation forest and adaptive random testing, characterized in that, It includes the following steps: Step 1: Obtain the input domain of the program to be tested; the program to be tested contains multiple input parameters; Step 2: Based on the program to be tested, construct a single-fault program by implanting errors, and use the comparison of the results of the program to be tested and the single-fault program as the basis for whether a program defect is detected; Step 3: According to the parameter range of the input domain, adopt a random value strategy to generate the original test case STC, and then generate the subsequent test case FTC in combination with the metamorphic relationship. The original test case STC and the subsequent test case FTC are constructed into an initial test case set; Step 4: Use the test cases in the initial test case set to detect whether the results of the program under test and the single-fault program are consistent. If they are inconsistent, a defect is detected; If no program defect is detected, put the test cases without detected defects into the selected test case set. After the number of elements in the selected test case set reaches N, use the selected test case set to construct an isolation forest model; Step 5: Divide the input domain into partitions according to the value ranges of the input parameters. In the area that does not contain the selected test case set, use the random value strategy to generate candidate original test cases, and determine whether each candidate original test case detects a defect in turn. If it does, proceed to the detection of the next candidate original test case; otherwise, proceed to Step 7; Step 6: If the candidate original test case does not detect a defect, then generate a candidate subsequent test case in combination with the metamorphic relationship, and determine whether the candidate subsequent test case falls into the area covered by the test cases under test. If it falls into the covered area, return to Step 5 until a candidate subsequent test case that does not fall into the area covered by the test cases under test is obtained; put the candidate original test case and the finally obtained candidate subsequent test case into the candidate test case set; Step 7: Input the candidate test cases in the candidate test case set into the isolation forest model, and calculate the anomaly score S of the candidate test cases A , and select the candidate test cases with anomaly scores higher than the anomaly score threshold as the next set of test cases to be added to the selected test case set; Step 8: Use the selected test case set to detect the program under test and check the defect detection situation. If no program defect is detected or the end condition is not reached, repeat Steps 5 - 7; until a program defect is found or the end condition is reached.

2. The method for generating a metamorphic test suite based on isolation forest and adaptive random testing according to claim 1, wherein In Step 1, obtain the input domain of the program under test according to the design document of the program under test; the input parameters have their respective value ranges.

3. The metamorphic test suite generation method based on isolated forest and adaptive random testing according to claim 1, wherein In Step 2, use a first-order mutation operator or an arithmetic operator to implant the program to be tested to construct a single-fault program.

4. The method for generating a metamorphic test suite based on isolation forest and adaptive random testing according to claim 1, wherein In Step 3, adopt a random value strategy to generate the original test case STC.

5. The metamorphic test suite generation method based on isolation forest and adaptive random testing according to claim 1, characterized in that In Step 4, N≥4.

6. The metamorphic test suite generation method based on isolation forest and adaptive random testing according to claim 1, wherein In Step 8, the end condition is to reach a preset number of loop iterations.

7. The metamorphic test suite generation method based on isolation forest and adaptive random testing according to claim 1, characterized in that, In the seventh step, the anomaly score S A is calculated by the following formula: Where x is the candidate test case, use the selected test case set to construct an isolation forest, h(x) is the height of x in the isolation tree, E(h(x)) is the average value of h(x), and c(n) is the average height of the isolation tree when the input of the candidate test case is equal to n.

Images