Two-parameter combination test method based on genetic algorithm

Abstract

The invention discloses a two-parameter combination testing method based on a genetic algorithm, and belongs to the technical field of software testing. The two-parameter combination test method comprises the following steps: defining a priority value for each value of each parameter to be tested through a priority determination rule; obtaining M candidate test cases by adopting a greedy algorithm according to a one-dimensional expansion strategy and the priority of each value of the to-be-tested parameter; encoding candidate test cases to obtain an initial population, then using the genetic algorithm provided by the invention to evolve the initial population, and when the genetic algorithm stops, selecting an optimal individual and adding the optimal individual into a test case set; and limiting the execution of the steps for certain times according to test conditions, and testing according to the sequence of the test cases obtained in the test case set during testing. The problems that under the condition that resources are limited, key parameters and combinations cannot be fully tested, the test case generation time is too long, and the defect detection rate cannot pass are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of software testing, and particularly relates to a two-parameter combinatorial testing method based on a genetic algorithm. Background Art

[0002] Software testing is a key link in building highly reliable software. Statistical data shows that this link generally accounts for more than 50% of the total software development cost, and an effective testing method is the key to reducing software development costs. Current computer systems are becoming increasingly large and complex, often having a large number of input parameters, and each parameter may have multiple different values or equivalence class partitions. The most comprehensive testing method is to design a test case set that covers all combinations between parameters, but the scale of the generated test case set is often too large to be acceptable in terms of cost. For example: for a system under test with k parameters, these parameters have v1, v2, …, v k possible values respectively, and completely testing this system requires test cases. For a general system under test, this combination number is a very large number. How to select a subset with a smaller scale as the test case set is a very important issue in test case generation. A compromise between test performance and cost is combinatorial testing.

[0003] Because according to observations, for many applications, many program errors are caused by the interaction of a few parameters. For example: Kuhn and Reilly analyzed the error report records of the Mozilla browser and found that more than 70% of the errors were triggered by the interaction of two certain parameters, and more than 90% of the errors were caused by the interaction of no more than 3 parameters. Thus, we can select some test cases such that for any t (t is a small positive integer, generally 2 or 3) parameters, all possible value combinations of these t parameters are covered by at least one test case. We call this testing criterion t-combinatorial testing. When the value of t is 2, this 2-combinatorial testing is called two-parameter combinatorial testing. For example, for an e-commerce system with 4 parameters and each parameter having 3 optional values, completely testing this system requires 34 = 81 test cases. Adopting the two-parameter combinatorial testing criterion, only 9 test cases are needed during testing to cover all value combinations of any two parameters.

[0004] The two-parameter combination testing method is very effective in system testing. For a system with k parameters, the minimum number of test cases required to complete two-parameter combination testing grows logarithmically with k. Kuhn et al. from the National Institute of Standards and Technology (NIST) in the United States studied the error detection rate of combinatorial testing using 4 software systems. The results show that the error detection rate of two-parameter combination testing has exceeded 80%. Grindal's research points out that the two-parameter combination testing method has the characteristics of simple model, low requirements for testers, and can effectively handle large-scale testing requirements. It is a feasible and practical testing solution.

[0005] A large part of the research on classical combinatorial testing uses traditional constraint solving or optimization methods to directly search for covering arrays. Due to the complexity of this problem being NP-complete, most methods are local search algorithms. These methods cannot guarantee obtaining the optimal solution, but the processing time is relatively less. These methods mainly include greedy algorithms and heuristic search methods. In addition, a small amount of research uses global search algorithms, which can obtain the optimal solution for problems of a certain scale. However, the current methods cannot guarantee preferentially testing the most critical parameter combinations when test resources are limited. The present invention assigns priority values to combinatorial parameters to ensure that critical parameter combinations are preferentially tested when the test set does not fully cover all cases. Summary of the Invention

[0006] The object of the present invention is to solve the problems raised in the background technology and propose a two-parameter combination testing method based on genetic algorithms.

[0007] To achieve the object of the present invention, the present invention discloses a two-parameter combination testing method based on genetic algorithms, including steps of establishing a priority model and generating test cases, and the specific steps are as follows:

[0008] Step 1: Define the priority of parameter values;

[0009] Step 2: Calculate the priority values of parameter value pairs and the priority values of test cases;

[0010] Step 3: Generate candidate test cases through a greedy algorithm and generate optimal tests using a genetic algorithm.

[0011] Further, in Step 1, the priority influencing factors include: code coverage, cost metric, time metric since the last modification, modification frequency, user usage frequency, value range; the priority calculation formula is as Formula (1):

[0012]

[0013] Among them, η1 to η6 respectively represent the proportions of each influencing factor in the total priority value, all of which are decimals between 0 and 1, and η1 + η2 + η3 + η4 + η5 + η6 = 1; w represents the priority value of a certain parameter value; c represents the code coverage rate, which is a decimal between 0 and 1; p represents the test cost of this test case, p max represents the maximum cost in the test case set, p min represents the minimum cost; r represents the time metric from the modification to the current time, which satisfies the following formula (2); m represents the modification frequency, m* represents the actual number of modifications of this parameter, m max represents the number of modifications of the parameter with the most modifications in the system; u represents the user usage frequency, u* represents the estimated value of the actual number of times this parameter is used, u max represents the estimated value of the number of times the parameter with the most usage times in the system is used; v represents the value range, v* represents the actual number of values of this parameter, v max represents the number of values of the parameter with the most number of values in the system;

[0014]

[0015] Among them, t is the time elapsed since the last modification, α is a constant, and formula (2) conforms to the forgetting law curve, that is, the measurement value of the code just modified is the largest at 1 and gradually becomes smaller as time goes by.

[0016] Furthermore, in step 2, the priority value of the parameter value binary group is the product of the weights of the two values in the binary group; the priority value of the test case is the sum of the weights of the first-occurring binary group it covers.

[0017] Furthermore, in step 3, the greedy strategy of the greedy algorithm is defined as: select a value of a parameter such that the combined weight of all uncovered binary groups formed by it and the already fixed parameters is the largest; the coding method of the genetic algorithm is defined as: using binary coding, if the possible number of values of a parameter f is t, and 2n - 1 < t ≤ 2n, then the coding bit number of this parameter is n; if the number of coding representations is m, the number of parameter values is n, and m > n, then the last m - n codings represent the m - n parameter values with the largest weights; the fitness function of the genetic algorithm is defined as: using the combined weight of the test case as the fitness, that is, the sum of the weights of the first-occurring binary group it covers, and the fitness function is used to calculate this combined weight; the single-point crossover of the genetic algorithm is defined as: disconnect two coding sequences at the same point and cross-join the disconnected parts.

[0018] Further, step 2 is specifically as follows: According to the priority calculation formula, calculate the priority value of each value of each parameter, find out the binary tuples composed of the values of all parameters and their weights, and put them into the uncovered binary tuple set Uncover.

[0019] Further, step 3 is specifically as follows:

[0020] Step 3-1: According to the greedy algorithm, obtain M candidate test cases from the Uncover set according to the priority value situation of the parameter values;

[0021] Step 3-2: According to the genetic algorithm, encode the candidate test cases obtained in step 3-1 and perform an evolution operation. When the genetic algorithm stops, select the optimal individual and add it to the test case set, and delete the covered binary tuples from the Uncover set;

[0022] If the Uncover set is not empty and the test resources still allow testing more test cases, then go to step 3-1; otherwise go to step 3-3;

[0023] Step 3-3: Test each parameter in the system one by one according to the priority order of the obtained test case set.

[0024] Further, step 3-1 is specifically as follows:

[0025] Step 3-1-1: Select the M binary tuples t with the largest weights from the Uncover set i , where 1 ≤ i ≤ M. If the number of binary tuples in the Uncover set is less than M, then select all of them;

[0026] Step 3-1-2: Determine the values of the two parameters in the candidate test case test i according to the binary tuple t i . The definition of i is the same as that in step 3-1-1;

[0027] Step 3-1-3: For the unfixed parameters of the M test cases, determine the values in turn according to the greedy strategy in order, and finally obtain test i , and end step 3-1.

[0028] Further, step 3-2 is specifically as follows:

[0029] Step 3-2-1: Encode the M test cases obtained in step 3-1;

[0030] Step 3-2-2: Use the fitness function to find the fitness of the test cases;

[0031] If the number of evolution times is sufficient, then go to step 3-2-6; otherwise go to step 3-2-3;

[0032] Step 3-2-3: Select the top P max individuals with relatively high fitness, and at the same time select the last P min individuals with relatively low fitness to participate in the evolutionary process of the next generation;

[0033] Step 3-2-4: Perform single-point crossover on the individuals selected in Step 3-2-3;

[0034] Step 3-2-5: Randomly perform a binary negation operation on a certain bit in the sequence of the individuals obtained in Step 3-2-4 with a probability of Pm; Go back to Step 3-2-2;

[0035] Step 3-2-6: Select the individual with the optimal fitness and add it to the test case set, and delete the covered binary tuples from the Uncover set, and end Step 3-2

[0036] To achieve the object of the present invention, the present invention also discloses a two-parameter combination testing system based on a genetic algorithm, including the following modules: a priority definition module; defining the priority of parameter values; a priority value calculation module; calculating the priority value of a binary tuple of parameter values and the priority value of a test case; a testing module; generating candidate test cases through a greedy algorithm and generating optimal tests using a genetic algorithm.

[0037] To achieve the object of the present invention, the present invention also discloses an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, it implements a two-parameter combination testing method based on a genetic algorithm.

[0038] Compared with the prior art, the significant progress of the present invention lies in: the priority-based two-parameter combination testing method provided by the present invention uses the definition of priority, so that the obtained test cases can effectively test the key parameters of the system under limited resources; through the greedy algorithm plus the genetic algorithm, the generation of test cases can be accelerated and the cost of generating test cases can be reduced.

[0039] To more clearly illustrate the functional characteristics and structural parameters of the present invention, the following further explains in conjunction with the drawings and specific embodiments. Description of the Drawings

[0040] The drawings described herein are used to provide a further understanding of the present invention, form a part of this application, and the schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0041] Figure 1 is the flowchart of the embodiment of the present invention;

[0042] Figure 2It is the flowchart of the greedy algorithm in the embodiments of the present invention;

[0043] Figure 3 It is the flowchart of the genetic algorithm in the embodiments of the present invention. Specific embodiments

[0044] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0045] As Figure 1 shown, the priority-based two-parameter combination testing method proposed in this embodiment is divided into four steps:

[0046] Step S1, establish a priority model. The definition of priority has been given in the invention content. For a system, first, it is necessary to evaluate the priority influencing factors for each value of each parameter; then, calculate the priority value of each parameter value according to the priority value calculation formula in the invention; then list the combinations of each parameter value and calculate the priority values of these binary groups. The following gives the specific operations for setting the priority model:

[0047] Suppose a system has three parameters A, B, and C. Among them, parameter A has 4 values (A1, A2, A3, A4), parameter B has 2 values (B1, B2), and parameter C has 3 values (C1, C2, C3). The values of the parameters and the values of the priority influencing factors are shown in Table 1:

[0048] Table 1 Values of parameters and values of priority influencing factors

[0049]

[0050] According to the formula proposed in the present invention, the priority value of each parameter value can be calculated. The formula is as follows:

[0051]

[0052] Here, we assign values to η1 to η6 respectively, η1 = 0.2, η2 = 0.2, η3 = 0.15, η4 = 0.15, η5 = 0.15, η6 = 0.15. Here, each ratio can be adjusted according to the actual situation. Substitute the data in the above table into the formula to calculate the priority value of each parameter value. The calculated priority value situation is shown in Table 2:

[0053] Table 2 Priority values of parameter values

[0054]

[0055] After obtaining the priority values of each parameter value, calculate the combination of parameter values and its weight. The combined weight is the product of the priority values of the two values, as shown in Table 3:

[0056] Table 3 Parameter value combinations and their priority values

[0057]

[0058] Since there are estimations in some value-taking processes, the final priority value only needs to retain two decimal places to reflect the priority situation of the combination. Therefore, approximate processing is performed on all calculated priority values in Table 3.

[0059] After the priority model is established, initialize the set of uncovered pairs Uncover and put all binary combinations into it.

[0060] Step S2, according to the greedy algorithm, obtain M candidate test cases from the Uncover set according to the priority value situation of the parameter values. The algorithm process is as Figure 2 shown, and the specific steps are as follows:

[0061] Step S2-1, pick out the M binary pairs t with the largest weights from the Uncover set i (1 ≤ i ≤ M. If the number of binary pairs in the Uncover set is less than M, pick out all of them);

[0062] Sort the Uncover set in descending order to obtain the set as shown in Table 4:

[0063] Table 4 Uncover set sorted in descending order of priority value

[0064]

[0065]

[0066] Here, we set M = 9, and the value of M can also be adjusted according to the actual situation. That is, pick out the first 8 binary pairs with the largest weights from the Uncover set, which are: A4C1, A4C2, B2C1, A1C1, A4C3, B1C1, B2C2, A1C2, B1C2.

[0067] Step S2-2, determine the candidate test case test according to the binary pair t i Determine the values of the two parameters in i (i is defined the same as in Step S2-1). That is, the values in these 8 binary pairs are fixed and they are the determined parts in the 8 candidate test cases;

[0068] Step S2-3: For the unfixed parameters remaining in the M test cases, determine their values in sequence according to the greedy strategy. Finally, obtain test i .

[0069] The greedy strategy of the present invention is to select a value for a parameter such that the combined weight of all uncovered binary tuples formed by it and the already fixed parameters is the largest. For example, for A4C1, the value of parameter B is missing, so it can be combined with B1 or B2. If combined with B1, its combined weight is:

[0070] Weight of A4B1 + Weight of A4C1 + Weight of B1C1 = 0.20 + 0.32 + 0.27 = 0.79

[0071] If combined with B2, its combined weight is:

[0072] Weight of A4B2 + Weight of A4C1 + Weight of B2C1 = 0.22 + 0.32 + 0.29 = 0.83

[0073] It can be seen that if combined with B2, the combined weight is greater. Therefore, B2 is selected for combination with it to generate a candidate test case A4B2C1. Perform such an operation on each binary tuple to obtain all candidate test cases.

[0074] Step S3: According to the genetic algorithm, encode the candidate test cases obtained in step S2 and perform evolutionary operations. When the genetic algorithm stops, select the optimal individual and add it to the test case set, and delete the covered binary tuples from the Uncover set. The genetic algorithm process is as Figure 3 , and the specific steps are as follows;

[0075] Step S3-1: Encode the 9 test cases obtained in step S2;

[0076] The number of bits of the binary code needs to be determined according to the number of parameter values. For example, if parameter A has 4 values, parameter B has 2 values, and parameter C has 3 values, then parameter A is represented by 2 bits, parameter B is represented by 1 bit, and parameter C is represented by 2 bits. The corresponding encoding and value relationship is shown in Table 5:

[0077] Table 5 Encoding and value relationship

[0078]

[0079] Parameter C has only 3 values, and the encoding can represent 4. Therefore, the last encoding represents the value with the largest priority weight.

[0080] The encoding of a test case like A4B2C1 is 11100; for a given encoding, the corresponding values can be found according to Table 5. For example, 10101 corresponds to A3B2C2.

[0081] Step S3-2: Use the fitness function to calculate the fitness of these test cases. Here, we take the composite weight of a test case as the fitness.

[0082] If the number of evolution times is sufficient, go to step S3-6; otherwise, go to step S3-3.

[0083] Step S3-3: Select the top P max proportion of individuals with higher fitness, and at the same time select the bottom P min proportion of individuals with lower fitness to participate in the next generation's evolution process. Here, we can select P max and P min to be 60% and 20% respectively.

[0084] Step S3-4: Perform single-point crossover on the individuals selected in step S3-3. Randomly select a crossover point. When performing crossover, the partial structures of the two individuals before or after this point are swapped to generate two new individuals. For example:

[0085] Individual A: 001↑11 → 00101 (new individual A')

[0086] Individual B: 100↑01 → 10011 (new individual B')

[0087] Step S3-5: Randomly perform a binary bitwise inversion operation on a certain bit in the sequence of the individuals obtained in step S3-4 with probability Pm. Pm is an empirical value and is adjusted according to the actual situation. Then go to step S3-2.

[0088] Step S3-6: Select the individual with the optimal fitness and add it to the test case set, and delete the covered binary tuples from the Uncover set.

[0089] For example, if the optimal individual obtained through the first round of genetic algorithm is A4B2C1, then all the binary tuples it contains need to be deleted from the Uncover set. The binary tuples to be deleted are A4B2, A4C1, and B2C1.

[0090] If the Uncover set is not empty and the test resources still allow testing more test cases, go to step S2; otherwise, go to step S4.

[0091] Step S4: Test each parameter in the system according to the test case set obtained based on the present invention. Since the test cases earlier in the test case set have greater compliance weights, testing in this test case order can ensure that the key parameters of the system are tested, thereby passing the defect detection rate.

[0092] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0093] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A two-parameter combinatorial testing method based on genetic algorithm, characterized in that, Steps for establishing a priority model and generating test cases are as follows: Step 1: Define the priority of parameter values. Step 2: Calculate the priority values of parameter value pairs and the priority value of the test case. Step 3: Generate candidate test cases through a greedy algorithm and generate optimal tests using a genetic algorithm.

2. The two-parameter combination testing method based on genetic algorithm according to claim 1, characterized in that, In Step 1, the priority influencing factors include: code coverage, cost metric, time metric since the last modification, modification frequency, user usage frequency, value range; the priority calculation formula is as shown in Formula (1): Among them, η1 to η6 respectively represent the proportions of each influencing factor in the total priority value, all of which are decimals between 0 and 1, and η1 + η2 + η3 + η4 + η5 + η6 = 1; w represents the priority value of a certain parameter value; c represents the code coverage rate, which is a decimal between 0 and 1; p represents the test cost of this test case, p max represents the maximum cost in the test case set, p min represents the minimum cost; r represents the time metric from the modification to the current, and it satisfies the following formula (2); m represents the modification frequency, m* represents the actual number of modifications of this parameter, m max represents the number of modifications of the parameter with the most modifications in the system; u represents the user usage frequency, u* represents the estimated value of the actual number of times this parameter is used, u max represents the estimated value of the number of times the parameter with the most usage times in the system is used; v represents the value range, v* represents the actual number of values of this parameter, v max represents the number of values of the parameter with the most values in the system; Among them, t is the time elapsed since the last modification, α is a constant, and Formula (2) conforms to the forgetting curve, that is, the metric value of the code just modified is the largest at 1 and gradually decreases over time.

3. A two-parameter combination testing method based on a genetic algorithm according to claim 1, characterized in that In Step 2, the priority value of the parameter value pair is the product of the weight values of the two values in the pair; the priority value of the test case is the sum of the weight values of the first-occurring pairs it covers.

4. A two-parameter combination testing method based on a genetic algorithm according to claim 1, characterized in that, In Step 3, the greedy strategy of the greedy algorithm is defined as: select a value of a parameter such that the combined weight value of all uncovered pairs formed by it and the already fixed parameters is the largest; the coding method of the genetic algorithm is defined as: using binary coding, if the number of possible values of a parameter f is t, and 2n - 1 < t ≤ 2n, then the coding length of this parameter is n; if the number of coding representations is m, the number of parameter values is n, and m > n, then the last m - n coding representations represent the m - n parameter values with the largest weights; the fitness function of the genetic algorithm is defined as: using the combined weight value of the test case as the fitness, that is, the sum of the weight values of the first-occurring pairs it covers, and the fitness function is used to calculate this combined weight value; the single-point crossover of the genetic algorithm is defined as: disconnect two coding sequences at the same point and cross-join the disconnected parts.

5. A two-parameter combination testing method based on a genetic algorithm according to claim 1, characterized in that, Step 2 is specifically: According to the priority calculation formula, calculate the priority value of each value of each parameter, find the pairs and their weights formed by the values of all parameters, and put them into the set of uncovered pairs Uncover.

6. The dual-parameter combined testing method based on genetic algorithm according to claim 5, characterized in that Step 3 is specifically: Step 3-1: According to the greedy algorithm, obtain M candidate test cases from the Uncover set based on the priority values of parameter values. Step 3-2: According to the genetic algorithm, encode the candidate test cases obtained in Step 3-1 and perform evolutionary operations. When the genetic algorithm stops, select the optimal individual and add it to the test case set, and delete the covered pairs from the Uncover set. If the Uncover set is not empty and the test resources still allow testing more test cases, then go to Step 3-1; Otherwise, go to Step 3-3; Step 3-3: Test each parameter in the system according to the priority order of the obtained test case set.

7. A two-parameter combination testing method based on a genetic algorithm according to claim 6, characterized in that, Step 3-1 is specifically as follows: Step 3-1-1, pick out the top M binary tuples t with the largest weights from the Uncover set i , where 1 ≤ i ≤ M. If the number of binary tuples in the Uncover set is less than M, pick out all of them; Step 3-1-2, according to the binary tuple t i determine the values of the two parameters in the candidate test case test i where i is defined the same as in Step 3-1-1; Step 3-1-3: For the unfixed parameters remaining in the M test cases, determine the values in sequence according to the greedy strategy, and finally obtain test i , and end Step 3-1.

8. A two-parameter combination testing method based on a genetic algorithm according to claim 6, characterized in that, Step 3-2 is specifically as follows: Step 3-2-1: Encode the M test cases obtained in Step 3-1. Step 3-2-2: Use the fitness function to find the fitness of the test cases. If the number of evolutionary times is sufficient, then go to Step 3-2-6; otherwise, go to Step 3-2-3; Step 3-2-3, select the top P max proportion of individuals with higher fitness, and at the same time select the bottom P min proportion of individuals with lower fitness to participate in the evolution process of the next generation; Step 3-2-4: Perform single-point crossover on the individuals selected in Step 3-2-3; Step 3-2-5: Randomly perform a binary negation operation on a certain bit in the sequence of the individuals obtained in Step 3-2-4 with probability Pm; Go back to Step 3-2-2; Step 3-2-6: Select the individual with the optimal fitness and add it to the test case set, and delete the covered binary tuples from the Uncover set, ending Step 3-2.

9. A dual-parameter combination testing system based on a genetic algorithm, the system being based on the method according to any one of claims 1-8, characterized in that, It includes the following modules: Priority definition module: Define the priority of parameter value selection; Priority value calculation module: Calculate the priority value of the binary tuple of parameter values and the priority value of the test case; Testing module; Generate candidate test cases through the greedy algorithm and generate the optimal test using the genetic algorithm.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a two-parameter combination testing method based on the genetic algorithm as described in any one of Claims 1-8.

Images