Multi-objective test case ranking method based on adaptive weight vector decomposition

Abstract

The present invention discloses a multi-objective test case sorting method based on decomposed weight vector adaptation. Aiming at the problems existing in the traditional test case sorting problem, such as single objective, high time cost, and unsatisfactory effect, the present invention creatively introduces the MOEA / D-VW algorithm, and takes the average branch coverage and effective execution time as optimization targets. The method can effectively achieve multi-objective test case sorting in a shorter time. The invention improves the early detection rate of defects and effectively reduces the cost of regression testing. At the method level, the present invention improves the traditional multi-objective optimization algorithm, adds an adaptive weight vector change strategy, so that the results have better distribution, and the obtained test case sorting sequence has more selectivity.

Description

Technical Field

[0001] The invention relates to the field of software maintenance, and in particular to a multi-objective test case sorting method based on decomposition-based weight vector adaptation. Background Art

[0002] During the software update and evolution process, testers are usually required to conduct more diversified software regression testing. Software regression testing means that when new functions are introduced to the software, testers must retest the original functions to ensure that the introduction of new functions does not affect the original functions. Test case prioritization is to sort the test cases that need to be regression tested to increase the early detection rate of defects and thus reduce testing costs. With the continuous improvement of industrial testing requirements, in the actual testing process, test case sorting needs to consider the impact of multiple factors on software quality, such as test cost, time, code coverage, repair difficulty, etc. The multi-objective test case prioritization problem (Multi-Objective Test Case Prioritization, MOTCP) is an important problem that needs to be solved urgently in regression testing.

[0003] At present, most researchers use greedy strategies or heuristic algorithms to solve MoTCP. In the process of selecting test cases, Yoo et al. used code coverage and test case execution cost as optimization targets based on the Pareto idea, and used multi-objective optimization methods such as NSGA-II and greedy strategies to set the test case priority. Although NSGA-II runs fast and has good convergence of the solution set, the congestion calculation process in each iteration is complicated and the algorithm performance is not ideal. Bian Yi et al. introduced CPU and GPU into the MOTCP problem and used the pre-optimization technology of NSGA-II to solve the MOTCP problem, which significantly improved the efficiency of the algorithm. However, it is still optimized based on the principle of dominance, and the algorithm complexity is high. When the test case set is large, the performance is not ideal. Xing Xing et al. used the ant colony algorithm to process MOTCP, and introduced the biological gene segment principle to expand the way of pheromone update, and achieved good results in solving the MOTCP problem. In addition, Megala Tulasiraman et al. used the peer-optimal clone selection algorithm, Tyagi Manika et al. used the discrete particle swarm algorithm, and Deb Kalyanmoy used the NSGA-II algorithm with an elite strategy to sort test cases. In principle, these algorithms are implemented based on a multi-objective optimization algorithm based on dominance. The multi-objective optimization algorithm based on dominance is more complex than the decomposition-based method and cannot introduce user preference information. The present invention creates an adaptive multi-objective optimization algorithm MOEA / D-VW. In solving MOTCP, this method can effectively improve the efficiency of regression testing and facilitate the early discovery of software defects. Summary of the invention

[0004] When performing software regression testing, due to the influence of multiple factors such as test cost and time, multiple test objectives usually need to be met. Therefore, the quality of a test case set ranking result needs to be evaluated from multiple indicators. In multi-objective test case ranking, it is difficult to achieve the optimal multiple objectives at the same time, and it is necessary to solve the non-dominated solution set of the problem. However, the non-dominated solution set obtained by traditional methods is not widely distributed and has poor fitness value. Therefore, it is necessary to find a more efficient multi-objective optimization algorithm to solve it.

[0005] The technical solution of the present invention is: a multi-objective test case sorting method based on decomposed weight vector adaptation, comprising the following steps:

[0006] S1. Assume that a program to be tested has m branches and uses n test cases to perform regression testing. The test case set is represented as Ψ={T 1 , T 2 , ..., T n}, where T i(1≤i≤n) is the i-th test case in the test case set, constructing the branch coverage matrix A n×m and the effective execution time vector V = (t 1 , t 2 , ..., t n ), where t i (i=1, 2, ..., n) is the effective execution time corresponding to the i-th test case. If the i-th test case covers the j-th branch during execution, then A ij =1, otherwise A ij =0;

[0007] S2. Coding: Given a test program and a set of test cases, number each test case in the set from 1 to n. The test case priority sequence is a full permutation of the test case numbers.

[0008] S3. Set the fitness function:

[0009] S4. Weight vector initialization:

[0010] S5. Initialization:

[0011] S6. Iteration process: Let gen = 0; according to each weight vector w u And the fitness function set F(x) to obtain Q subproblems ζ u (x), u = 1, 2, ..., Q;

[0012] S7. If gen≤maxGen, return to S6, otherwise go to S8;

[0013] S8. Output EP. The sequence corresponding to each individual in EP is an optimized sorting of test cases. Users can choose the appropriate solution based on their preference for the target.

[0014] Furthermore, the S3 specific method is as follows:

[0015] The average percentage of branch coverage (APBC) and effective execution time (EET) are selected as optimization targets to measure the quality of a test case priority sequence; assuming that after executing the first n′ (n′≤n) test cases in the sequence, all branches of the tested program can be covered, thus constructing a fitness function set F(x), F(x) = (APBC, EET), where TB j Indicates the position of the first test case in the test case priority sequence that covers the j-th branch in the program;

[0016] Furthermore, the specific method of S4 is as follows:

[0017] Create a set of l two-dimensional unit vectors E = {e 1 , e 2 , ..., e l} as a candidate set of weight vectors, and an identity matrix , calculate each unit vector The sum of the Euclidean distances to each subvector in W, and the unit vector e with the largest sum of distances is selected s Add it to W and remove it from E; repeat this operation; when the number of unit vectors in W reaches Q (Q is the number of solutions in the final solution set expected, and Q≤l+2), stop the operation and get the final weight vector set L;

[0018] Furthermore, the specific method of S5 is as follows:

[0019] Set the maximum number of iterations maxGen; randomly generate a scale of N (N = m * n)’s initial population P 0 ={x 1 , x 2 , ..., x N}; Calculate each individual is a full permutation of n test cases) with each weight vector w u (w u ∈L), find the distance between each weight vector w u The nearest n p individuals, and are labeled w u Neighborhood Set the initial ideal point z * =(0,0), initialize storage container EP=Φ:

[0020] Furthermore, the specific method of S6 is as follows:

[0021] S61. Genetic process: let iter = 0;

[0022] S611.Select: For each sub-problem ζ u , calculate its neighborhood B u For each individual x uv (v=1, 2, ..., n p )’s fitness f uv , and further calculate the probability of being selected Using the roulette algorithm, select two individuals x ua and xub ;

[0023] S612. Crossover: x through location-based crossover algorithm (PBX) ua and x ub Perform the crossover operation. The specific process is as follows: Randomly generate different integers and Generate Sequence middle The value of the position is related to x ua middle The values ​​of the positions are the same; find x ua middle Position value at x ub In the position, and then x ub The values ​​corresponding to the remaining positions in are placed in order In the remaining positions of Empathy Generation

[0024] S613. Mutation: Yes Perform mutation operations respectively, that is, for and Generate two positions randomly and swap the values ​​at these two positions to get

[0025] S614.Update: Comparison and The size of Zeji Otherwise remember Will and x ua Perform Chebyshev comparison if Use Replace x ua ; Compare similarly and The size of Zeji Otherwise remember Will and x ub Perform Chebyshev comparison if Use Replace x ub ; iter = iter + 1;

[0026] S615. If This returns to S611;

[0027] S62. Select the best x from each neighborhood uv Put it into EP and delete all the x from EP uv Pareto dominant individuals;

[0028] S63. Update ideal point z * : Compare each individual The fitness value of each target in the F(x) target space is calculated, and the minimum value is taken as z * The value on that dimension;

[0029] S64. Update the weight vector: And unitize it

[0030] S65. Update each subproblem ζ u The neighborhood B of (x) u ;Set gen=gen+1;

[0031] Compared with the traditional classification method, the present invention has the following creativity:

[0032] 1. The present invention creatively uses a decomposition-based multi-objective optimization algorithm to process the multi-objective test case priority sorting problem, and its performance is better.

[0033] 2. At the method level, the present invention improves the traditional multi-objective optimization algorithm and adds an adaptive weight vector change strategy, so that the results have better distribution and the obtained test case sorting sequence has more selectivity. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flow chart of the present invention;

[0035] Figure 2 Description of the PBX crossover process; DETAILED DESCRIPTION

[0036] Taking the test case priority sorting of the JavaScript unit testing framework Jasmine as an example, combined with the attached Figure 1 The specific implementation of the multi-objective test case priority sorting method for regression testing proposed by the present invention is described.

[0037] S1. For the program to be tested, Jasmine, its n = 24 test cases are used as the regression test suite. The Jasmine source program has 95 branches, and the test case set is represented as Ψ = {T 1 , T 2 , ..., T i , ..., T 24}, where Ti (1≤i≤24) is the i-th test case in the test case set, constructing the branch coverage matrix A 24×95 and the effective execution time vector V = (t 1 , t 2 , ..., t i , ..., t 24 ), where t i (i=1, 2, ..., 24) is the effective execution time corresponding to the ith test case. If the ith test case covers the jth branch during execution, then A ij =1, otherwise A ij =0;

[0038] S2. Coding: Given a test program and a set of test cases, number each test case in the set from 1 to 24. The test case priority sequence is the full permutation of the test case numbers.

[0039] S3. Set the fitness function: Select the average percentage of branch coverage (APBC) and effective execution time (EET) as the optimization goals to measure the quality of a test case priority sequence; assume that after executing the first n′ (n′≤24) test cases in the sequence, all branches of the tested program can be covered, thereby constructing a fitness function set F(x), F(x) = (APBC, EET), where TB j Indicates the position of the first test case in the test case priority sequence that covers the j-th branch in the program;

[0040] S4. Weight vector initialization: Create a set E = {e 1 , e 2 , ..., e 2000} as a candidate set of weight vectors, and an identity matrix Calculate each unit vector e k The sum of the Euclidean distances between (k=1, 2, ..., 2000) and each sub-vector in W, and select the unit vector e with the largest sum of distances s Add it to W and remove it from E; repeat this operation; when the number of unit vectors in W reaches Q = 10 (Q is the number of solutions in the final solution set expected to be obtained), stop the operation and obtain the final weight vector set L;

[0041] S5. Initialization: Set the maximum number of iterations maxGen = 300; randomly generate an initial population P of size N = 150 0 ={x 1 , x 2 , ..., x 150}; Calculate each individual is a full permutation of n test cases) with each weight vector w u (w u ∈L), find the distance between each weight vector w u The 10 most recent individuals, marked as w u Neighborhood Set the initial ideal point z * =(0,0), initialize storage container EP=Φ;

[0042] S6. Iteration process: Let gen = 0; according to each weight vector w u And the fitness function set F(x) to obtain Q subproblems ζ u (x), u = 1, 2, ..., 10;

[0043] S61. Genetic process: let iter = 0;

[0044] S611.Select: For each sub-problem ζ u , calculate its neighborhood B u For each individual x uv (v=1, 2, ..., n p )’s fitness f uv , and further calculate the probability of being selected Using the roulette algorithm, select two individuals x ua and x ub ;

[0045] S612. Crossover: x through location-based crossover algorithm (PBX) ua and x ub Perform the crossover operation. The specific process is as follows ( Figure 2 The specific process of the PBX crossover algorithm is as follows. For a clearer description, the size of n is simulated to be 10); 6 different integers are randomly generated. and Generate New Sequence middle The value of the position is related to x ua middle The values ​​of the positions are the same, ensuring that the position of the selected gene in the offspring is the same as that in the parent; find x ua middle Position value at x ub In the position, and then x ub The values ​​corresponding to the remaining positions in are placed in order In the remaining positions of Generate Sequence middle The value of the position is related to x ub middle The values ​​of the positions are the same; find x ub middle Position value at x ua In the position, and then x ua The values ​​corresponding to the remaining positions in are placed in order In the remaining positions of

[0046] S613. Mutation: Yes Perform mutation operations respectively, that is, for or Randomly generate two positions and swap the values ​​at these two positions to get

[0047] S614.Update: Comparison and The size of Zeji Otherwise remember Will and x ua Perform Chebyshev comparison if Use Replace x ua ; Compare similarly and The size of Zeji Otherwise remember Will and x ub Perform Chebyshev comparison if Use Replace x ub ; iter = iter + 1;

[0048] S615. If iter < 12, then return to S611;

[0049] S62. Select the best x from each neighborhood uv Put it into EP and delete all the x from EP uv Pareto dominant individuals;

[0050] S63. Update ideal point z * : Compare each individual xkp The fitness value of each target in the F(x) target space is calculated, and the minimum value is taken as z * The value on that dimension;

[0051] S64. Update the weight vector:

[0052] And unitize it

[0053] S65. Update each subproblem ζ u The neighborhood B of (x) u , gen=gen+1;

[0054] S7. If gen≤maxGen, return to S6, otherwise go to S8;

[0055] S8. Output EP. The sequence corresponding to each individual in EP is an optimized test case ranking; the user can choose the appropriate solution according to the preference for the target. A multi-objective test case ranking algorithm based on decomposition combined with an adaptive weight vector is used for branch coverage information of the Iasmine program. Each experiment takes 300 iterations and is repeated 30 times independently. The example results show that the test case ranking scheme obtained by the multi-objective test case ranking algorithm based on decomposition combined with an adaptive weight vector proposed in the present invention has good quality, low time cost, and a wide distribution range of solution sets. It is an effective test case priority ranking method for regression testing.

Claims

1. A multi-objective test case ranking method based on decomposed weight vector adaptation, characterized in that: The following steps are involved: S1. Assume that a program to be tested has m branches and uses n test cases to perform regression testing on it. The test case set is represented as Ψ = {T1, T2, ..., T n }, where T i For the i-th test case in the test case set, 1≤i≤n, construct the branch coverage matrix A n×m and the effective execution time vector V = (t1, t2, ..., t n ), where t i is the effective execution time corresponding to the i-th test case. If the i-th test case covers the j-th branch during execution, then A ij =1, otherwise A ij =0; S2. Coding: Given a test program and a set of test cases, number each test case in the set from 1 to n. The test case priority sequence is a full permutation of the test case numbers. S3. Set the fitness function: S4. Weight vector initialization: S5. Initialization: S6. Iteration process: Let gen = 0; according to each weight vector w u And the fitness function set F(x) to obtain Q subproblems ζ u (x), u = 1, 2, ..., Q; S7. If gen≤maxGen, return to S6, otherwise go to S8; S8. Output EP. The sequence corresponding to each individual in EP is an optimized test case sorting. The user selects the appropriate solution based on the preference for the target. The specific method of S4 is as follows: Create a set E containing two-dimensional unit vectors = {e1, e2, ..., e l } as a candidate set of weight vectors, and an identity matrix Calculate each unit vector The sum of the Euclidean distances to each subvector in W, where k l =1,2,...,l, and select the unit vector e with the largest sum of distances s Add it to W and remove it from E; repeat this operation; when the number of unit vectors in W reaches Q, Q is the number of solutions in the expected final solution set, and Q≤l+2, stop the operation and get the final weight vector set L; S5 specific methods are as follows: Set the maximum number of iterations maxGen; randomly generate an initial population P0 of size N = {x1, x2, ..., x N }, where N = m*n; calculate each individual With each weight vector w u The Euclidean distance between is a full permutation of n test cases, w u ∈L; Find the distance to each weight vector w u The nearest n p individuals, and are labeled w u Neighborhood Set the initial ideal point z*=(0,0) and initialize the storage container EP=Φ; S6 specific methods are as follows: S61. Genetic process: let iter = 0; S611.Select: For each sub-problem ζ u , calculate its neighborhood B u For each individual x uv The fitness f uv , where v = 1, 2, ..., n p , and further calculate the probability of being selected Using the roulette algorithm, select two individuals x ua and x ub ; S612. Crossover: PBX to x through location-based crossover algorithm ua and x ub Perform the crossover operation. The specific process is as follows: Randomly generate different integers and Generate Sequence middle The value of the position is related to x ua middle The values ​​of the positions are the same; find x ua middle Position value at x ub In the position, and then x ub The values ​​corresponding to the remaining positions in are placed in order In the remaining positions of Empathy Generation S613. Mutation: Yes Perform mutation operations respectively, that is, for and Generate two positions randomly and swap the values ​​at these two positions to get S614.Update: Comparison and The size of Zeji Otherwise remember Will and x ua Perform Chebyshev comparison if Use Replace x ua ; Compare similarly and The size of Zeji Otherwise remember Will and x ub Perform Chebyshev comparison if Use Replace x ub ; iter = iter + 1; S615. If This returns to S611; S62. Select the best x from each neighborhood uv Put it into EP and delete all the x from EP uv Pareto dominant individuals; S63. Update ideal point z * : Compare each individual The fitness value of each target in the F(x) target space, and take its minimum value as the value of z* in the dimension; S64. Update the weight vector: And unitize it S65. Update each subproblem ζ u The neighborhood B of (x) u ;Set gen=gen+1.

2. The multi-objective test case ranking method based on decomposition weight vector adaptation according to claim 1 is characterized in that: The specific method of S3 is as follows: The average branch coverage APBC and the effective execution time EET are selected as optimization targets to measure the quality of a test case priority sequence; assuming that after executing the first n' test cases in the sequence, all branches of the tested program can be covered, where n'≤n, and thus the fitness function set F(x) is constructed, F(x) = (APBC, EET), where TB j Represents the position of the first test case in the test case priority sequence that covers the j-th branch in the program.

Images