Orthogonal table-based test case automatic generation method

Abstract

The application discloses a test case automatic generation method based on an orthogonal table and relates to the technical field of software testing, which comprises the following steps: step 1: receiving inputted factors and corresponding levels, and matching an orthogonal table by using an orthogonal table matching strategy according to the factors and the levels; step 2: generating a plurality of test case sets according to data mapping of the matched orthogonal table; and step 3: evaluating the test case sets and selecting an optimal test case set according to a preset evaluation rule. The application can effectively reduce the number of test cases, reduce the test cost, and flexibly and efficiently generate test cases.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software testing, and more particularly to a test case automatic generation method based on orthogonal table. BACKGROUND

[0002] Poor software quality is the primary cause of project failure, and a large number of low-quality codes make software projects unable to be completed on time and cost controlled. Therefore, efficient and high-quality software coverage testing is an effective method to control the risk and cost of software development projects.

[0003] At present, most software systems have complex structures, numerous logical branches and functional modules, and it is impossible to achieve complete test coverage, but a sufficient and necessary test case set can be designed to improve the test coverage. A large number of test cases are needed to test complex software systems, and the design of test cases by manual work is not comprehensive, and the level and subjectivity of the test case designers will affect the effect of the test cases. Therefore, the generation of test cases in many cases needs the assistance of automation technology. With the improvement of software test automation, the automatic generation of test case data is the most promising in the design of test cases.

[0004] At the same time, software coverage testing also needs to consider efficiency, and the number of test cases is not the more the better. A large number of test cases will seriously affect the test efficiency and increase the test cost. In the test case design process, multiple input parameters are often encountered, and each parameter has multiple values. For example, assuming that there are m input parameters, each input parameter has n values, and according to the different combinations of input parameters, n m test cases are needed to cover all combinations. Due to time and cost constraints, it is impossible to complete all test cases, and comprehensive testing is not desirable. Therefore, another key problem of software testing is: among the possible test cases, how to select the test cases that are most likely to find the most errors for testing, so as to improve the test efficiency.

[0005] Therefore, how to intelligently generate test case data and improve the test coverage rate as much as possible, and improve the accuracy of software test sufficiency evaluation are problems that need to be solved by those skilled in the art. SUMMARY

[0006] Therefore, the present application provides a test case automatic generation method based on orthogonal table, which can intelligently generate test cases in batches, save labor costs, be more optimized and efficient, have high coverage rate, and have higher software error discovery accuracy.

[0007] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0008] A test case automatic generation method based on orthogonal table, comprising the following steps:

[0009] Step 1: receiving inputted factors and corresponding levels, matching orthogonal table according to factors and levels by orthogonal table matching strategy;

[0010] Step 2: generating several test case sets according to data mapping of matched orthogonal table;

[0011] Step 3: evaluating test case sets, and selecting optimal test case set according to preset evaluation rule.

[0012] Preferably, since the inputted data is very complex, different matching processing methods are used according to different situations, and the orthogonal table matching strategy comprises:

[0013] Step 11: if the levels of factors are same, searching for known standard orthogonal table with same factor number and level number, matching corresponding standard orthogonal table if existing, or entering step 12; if the levels of factors are different, entering step 13;

[0014] Step 12: searching for standard orthogonal table with same level number, greater factor number and minimum difference between factor number; searching for standard orthogonal table with same factor number, greater level number and minimum difference between level number; searching for standard orthogonal table greater than factor number and greater than level number, and minimum difference between factor number and level number; matching corresponding standard orthogonal table; comparing table level number with level number, and comparing table factor number with factor number;

[0015] Step 13: searching for known mixed level orthogonal table with same factor number and level number, matching corresponding mixed level orthogonal table if existing, or entering step 14;

[0016] Step 14: if the levels of factors are only two, judging whether the factor number and level number satisfy the condition of n 1 ×2 n order mixed level orthogonal table, matching n order Hadamard matrix if satisfying, constructing new orthogonal table L 2n (n 1 ×2 n ) as matched orthogonal table according to Hadamard matrix, or entering step 15; if the levels of factors are more than two, entering step 15;

[0017] Step 15: find the standard orthogonal table with the same number of factors and the same maximum number of levels, match the corresponding standard orthogonal table if it exists, otherwise go to step 16; construct a new orthogonal table according to the quasi-level method; construct a new orthogonal table according to the modified two-level orthogonal table method; construct a new orthogonal table according to the combined factor method; match the corresponding orthogonal table;

[0018] Step 16: find the standard orthogonal table with the same maximum number of levels, greater than the number of factors, and the smallest difference in the number of factors; find the standard orthogonal table with the same number of factors, greater than the maximum number of levels, and the smallest difference in the number of levels; find the standard orthogonal table greater than the number of factors, greater than the maximum number of levels, and the smallest difference in the number of factors and the number of levels; match the corresponding standard orthogonal table.

[0019] Preferably, the process of constructing an orthogonal table using the quasi-level method is as follows:

[0020] Step 111: find the standard orthogonal table with the same number of factors and the same maximum number of levels;

[0021] Step 112: sort the factors according to the number of levels, combine the factors with adjacent different levels, and virtually combine the small levels into large levels, then match the mixed-level orthogonal table with the same number of factors and levels.

[0022] Preferably, the process of constructing an orthogonal table using the modified two-level orthogonal table method is as follows:

[0023] Step 121: select all factors with levels other than two;

[0024] Step 122: virtually split the selected factors into several two-level factors by making the corresponding levels multiples of two;

[0025] Step 123: match the standard orthogonal table with the same number of factors and levels after virtualization.

[0026] Preferably, the process of constructing an orthogonal table using the combined factor method is as follows:

[0027] Step 131: select the maximum number of levels among all factors;

[0028] Step 132: combine factors with the same level two by two, and the combined level is the product of the original two levels, and if the combined level is greater than the maximum level, terminate the corresponding combination;

[0029] Step 133: match the orthogonal table according to the combined factors and the corresponding levels using the quasi-level method.

[0030] Preferably, in step 2, record the orthogonal table data in the matched orthogonal table and generate test cases.

[0031] Preferably, the process of selecting the optimal test case set according to the preset evaluation rule is:

[0032] Step 31: Sort the test case sets according to the number of test cases from small to large, select the second test case set, take the number of test cases corresponding to the selected test case set as the lower limit num_min, and set the acceptable test case number gap value num_gap;

[0033] Step 32: Select the test case set with the number of test cases less than or equal to num_min+num_gap from all test case sets;

[0034] Step 33: Calculate the evaluation index corresponding to the selected test case set;

[0035] Step 34: According to the evaluation index and the preset index constraint, the optimal test case set is selected.

[0036] Preferably, the evaluation index includes whether each factor of each level is covered by the test case, the uniformity of the corresponding level distribution of each factor, the average weight of the test case, and the case reduction rate.

[0037] Preferably, the case reduction rate is calculated by using the evaluation selection model, and the test case set with a large case reduction rate is selected, and the expression is:

[0038]

[0039] Wherein, OrthogonalNum represents the number of generated test cases; CompleteNum represents the number of test cases generated by using the comprehensive test method; and the case reduction rate represents the test burden reduced based on the orthogonal table test case generation method.

[0040] According to the above technical solution, compared with the prior art, the application provides a test case automatic generation method based on an orthogonal table, which includes an orthogonal table matching strategy, mapping test cases, and evaluation selection of test case sets. In the orthogonal table matching strategy, the quasi-level method, the modified two-level orthogonal table method, and the combined factor method are used for orthogonal table matching for complex factors and level data. Then, test cases are generated according to the matched orthogonal table, and the most suitable test case set is selected through evaluation selection. The evaluation selection takes the number of test cases of the test case set and the coverage of the test points as the main evaluation basis. In the test case set with little difference in the number of test cases, the test case set is comprehensively evaluated and analyzed from the level coverage, the average uniformity, the average weight of the test case, and the case reduction rate, and the optimal test case set is selected. The test case generation method of the application can effectively reduce the number of test cases and the test cost, and can flexibly and efficiently generate test cases. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the accompanying drawings in the following description only only the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0042] Figure 1 A flow chart of a test case automatic generation method based on an orthogonal table is provided in the present application.

[0043] Figure 2 An orthogonal representation intention matching method using quasi-horizontal method is provided in the present application.

[0044] Figure 3 An orthogonal representation intention matching method using modified two-level orthogonal table is provided in the present application.

[0045] Figure 4 An orthogonal representation intention matching method using combination factor method is provided in the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0047] The embodiments of the present application disclose a test case automatic generation method based on an orthogonal table, as shown in the following figure, including the following steps: Figure 1

[0048] S1: receiving inputted factors and corresponding levels, and matching an orthogonal table according to the factors and levels using an orthogonal table matching strategy;

[0049] S2: generating several test case sets according to the data mapping of the matched orthogonal table;

[0050] S3: evaluating the test case sets, and selecting an optimal test case set according to a preset evaluation rule.

[0051] Further, since the inputted data of the user is very complex, different matching processing methods will be used according to different situations, and the orthogonal table matching strategy includes:

[0052] ​S11: if the number of levels of each factor is the same, find the known standard orthogonal table with the same number of factors and levels, match the corresponding standard orthogonal table if it exists, otherwise go to S12; if the number of levels of each factor is different, go to S13;

[0053] S12: find the standard orthogonal table with the same number of levels, greater than the number of factors and the minimum difference with the number of factors; find the standard orthogonal table with the same number of factors, greater than the number of levels and the minimum difference with the number of levels; find the standard orthogonal table greater than the number of factors, greater than the number of levels, and the minimum difference with the number of factors and the number of levels; match the corresponding standard orthogonal table; compare the table level number in the standard orthogonal table with the number of levels, and compare the table factor number with the number of factors;

[0054] S13: find the known mixed level orthogonal table with the same number of factors and levels, match the corresponding mixed level orthogonal table if it exists, otherwise go to S14;

[0055] S14: if the number of levels of each factor is only two, determine whether the number of factors and the number of levels meet the conditions of the n 1 ×2 n order Hadamard matrix, if it meets, match the n order Hadamard matrix, and construct a new orthogonal table L 2n (n 1 ×2 n ) as the matched orthogonal table, otherwise go to S15; if the number of levels of each factor is greater than two, go to S15;

[0056] S15: find the standard orthogonal table with the same number of factors and the maximum number of levels, match the corresponding standard orthogonal table if it exists, otherwise go to S16; construct a new orthogonal table according to the quasi-level method; construct a new orthogonal table according to the modified two-level orthogonal table method; construct a new orthogonal table according to the combined factor method; match the corresponding orthogonal table;

[0057] S16: find the standard orthogonal table with the same number of levels, greater than the number of factors and the minimum difference of the number of factors; find the standard orthogonal table with the same number of factors, greater than the maximum number of levels and the minimum difference of the number of levels; find the standard orthogonal table greater than the number of factors, greater than the maximum number of levels, and the minimum difference with the number of factors and the number of levels; match the corresponding standard orthogonal table.

[0058] Furthermore, the pseudo-level method is to virtualize one or several levels for factors with fewer levels, making the number of levels of each factor equal to the corresponding number of levels in the orthogonal table, for matching to the orthogonal table, arranging the factors with fewer levels on the columns of the factors with more levels in the orthogonal table, and realizing the arrangement of factors with fewer levels on the multi-level orthogonal table. When the input factor level data cannot be matched to a just-suitable orthogonal table, the pseudo-level method needs to be used for orthogonal table matching. The process of constructing an orthogonal table using the pseudo-level method is as follows:

[0059] S111: Search for a standard orthogonal table with the same number of factors and the maximum number of levels respectively;

[0060] Virtualize the number of levels less than the maximum number of levels in the input to the maximum number of levels, and fill the corresponding number of levels of the factors to the maximum number of levels with 0; Search for a standard orthogonal table with the same number of virtualized factors and levels;

[0061] S112: Sort the factors according to the number of levels, combine the factors with adjacent different numbers of levels, virtualize the smaller number of levels in the combination to the larger number of levels, and match the same mixed-level orthogonal table according to the virtualized number of factors and levels.

[0062] Furthermore, the method of transforming the two-level orthogonal table is to merge any x columns in the 2 n type orthogonal table into one column x < n. If the merged column is a new column with two levels, then transform the two-level orthogonal table into a mixed-level orthogonal table, realizing the arrangement of multi-level factors in a less-level orthogonal table. For example: Merge any two columns in the 2 x type orthogonal table to form a 4 n × two i type orthogonal table. i represents the number of merges. Because each column in the two-level orthogonal table has only two values, 0 and 1, there are four combination methods between any two columns: (0, 0), (0, 1), (1, 0), (1, 1). When merging two columns into one column: (0, 0) is replaced by 0, (0, 1) is replaced by 1, (1, 0) is replaced by 2, and (1, 1) is replaced by 3. The merged column is a new column with four levels; Merge any three columns in the 2 n-2i type orthogonal table to form an 8 n × two i type orthogonal table. The process of constructing an orthogonal table using the method of transforming the two-level orthogonal table is as follows: n-3i

[0063] S121: Screen out all factors whose number of levels is not equal to two;

[0064] S122: Virtualize the number of levels corresponding to the screened factors to multiples of 2 and then split them into several two-level factors;

[0065]

[0065] S123: Match the same standard orthogonal table according to the virtualized number of factors and levels.

[0066] Further, the combination factor method is a design method of combining two factors with less levels into one factor with more levels arranged in the multi-level orthogonal table. The principle of level combination of two factors is to combine two levels of different factors with each other, and the number of combined levels must be less than the maximum level number, otherwise it cannot be combined. The process of constructing an orthogonal table by using the combination factor method is:

[0067] S131: Select the maximum level number of each factor;

[0068] S132: Combine two factors with the same level number with each other, and the number of combined levels is the product of the two original level numbers. When the number of combined levels is greater than the maximum level number, terminate the corresponding combination;

[0069] S133: According to the combined factors and the corresponding level number, match the orthogonal table by using the quasi-level method.

[0070] Further, record the orthogonal table data in the matched orthogonal table in S2, and generate test cases.

[0071] Further, the process of selecting the optimal test case set according to the preset evaluation rule is:

[0072] S31: Sort the test case sets according to the test case number from small to large, select the second test case set, and take the test case number corresponding to the selected test case set as the lower limit num_min. And set the acceptable test case number gap value num_gap;

[0073] S32: From all test case sets, select test case sets with test case numbers less than or equal to num_min+num_gap;

[0074] S33: Calculate the evaluation index corresponding to the selected test case set;

[0075] S34: According to the evaluation index and the preset index constraint, select the optimal test case set.

[0076] Further, the evaluation index includes whether the test case covers each level of each factor, the uniformity of the level distribution of each corresponding factor, the average weight of the test case, and the case reduction rate.

[0077] Further, the case reduction rate is calculated by using the evaluation selection model, and the test case set with a large case reduction rate is selected, and the expression is:

[0078]

[0079] Wherein, OrthogonalNum represents the number of test cases generated; CompleteNum represents the number of test cases generated by using the comprehensive test method; and the test case reduction rate represents the test burden reduced by the test case generation method based on the orthogonal table.

[0080] In one embodiment, for a data with four two-level factors, two three-level factors and one four-level factor, the data can be expressed by a mathematical formula as 2 4 ×3 2 ×4. The process of matching the quasi-level method to the orthogonal table is shown in Figure 2 , and includes the following steps:

[0081] S1: Find the maximum level number 4 and the factor number 7 in the data 2 4 ×3 2 ×4. Use the quasi-equal level algorithm to virtually level the factors with a level less than 4 to 4, convert the original data to seven four-level factors, and express the data by a mathematical formula as 4 7 . Match the seven four-level factors to the standard orthogonal table L 32 4 9 .

[0082] S2: Combine the factors corresponding to the two groups of different level numbers, and traverse the combinations in sequence to virtually level the different factors to different numbers to match the appropriate mixed-level orthogonal table. When the three-level factors and the four-level factors are combined, virtually level the two three-level factors to two four-level factors, and the original data 2 4 ×3 2 ×4 becomes 2 4 ×4 3 , and is matched to the mixed-level orthogonal table L8(2 4 ×4 3 ). When the two-level factors and the three-level factors are combined, virtually level the four two-level factors to four three-level factors, and the original data 2 4 ×3 2 ×4 becomes 3 6 ×4, and is matched to the mixed-level orthogonal table L 18 (3 6 ×4).

[0083] In one embodiment, for a data 2 4 ×3 2 ×4 with four two-level factors, two three-level factors and one four-level factor, the process of matching the quasi-level method to the orthogonal table is shown in Figure 3 , and the specific steps include:

[0084] S1: Find the maximum level number 4 in the data 2 4 ×3 2 ×4.

[0085] S2: After traversing analysis, two horizontal factors do not need to be combined and listed, each three horizontal factor and four horizontal factor need to combine two columns, the horizontal number is virtual as two horizontal, so 2 4 ×3 2 ×4 total 10 columns are needed;

[0086] S3: Match to two horizontal standard orthogonal table L 12 2 11 .

[0087] In one specific embodiment, for a data with four two horizontal factors, two three horizontal factors and one four horizontal factor, 2 4 ×3 2 ×4, the modified combination factor method is used to match the orthogonal table as shown in Figure 4 , which specifically includes the following steps:

[0088] S1: Find the maximum horizontal number in 2 4 ×3 2 ×4 data is 4;

[0089] S2: Two two horizontal factors are combined, the horizontal number after combination is the product of the original two groups of horizontal numbers, two four horizontal factors are obtained by combination;

[0090] S3: Since the horizontal number after combination of two three horizontal factors is greater than the maximum horizontal number, the combination of three horizontal factors is terminated;

[0091] S4: The original data 2 4 ×3 2 ×4 is converted to 3 2 ×4 2 after combination;

[0092] S4: The factor number and horizontal number after combination are matched using the idea of quasi-horizontal method, and matched to standard orthogonal table L 16 4 5 .

[0093] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts between each embodiment can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the related parts can be referred to the method part.

[0094] The foregoing description of the disclosed embodiments enables a person skilled in the art to make or use the application. Modifications of these embodiments will occur to persons of skill in the art, and that the appended claims are intended to cover all such modifications that do not depart from the true spirit and scope of the application. Therefore, the application is not limited to the embodiments shown but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A test case automatic generation method based on orthogonal table, characterized in that, The method comprises the following steps: Step 1: receiving inputted factors and corresponding levels, and matching an orthogonal table according to the factors and the levels by using an orthogonal table matching strategy; Step 2: generating a plurality of test case sets according to data mapping of the matched orthogonal table; Step 3: evaluating the test case sets, and selecting an optimal test case set according to a preset evaluation rule; The orthogonal table matching strategy comprises: Step 11: if the levels of the factors are the same, searching for a known standard orthogonal table with the same factor number and level number, matching the corresponding standard orthogonal table if the standard orthogonal table exists, or entering step 12; if the levels of the factors are different, entering step 13; Step 12: searching for a standard orthogonal table with the same level number, greater than the factor number and the minimum difference with the factor number; searching for a standard orthogonal table with the same factor number, greater than the level number and the minimum difference with the level number; searching for a standard orthogonal table greater than the factor number and greater than the level number, and the minimum difference with the factor number and the level number; matching the corresponding standard orthogonal table; comparing the table level number in the standard orthogonal table with the level number, and comparing the table factor number in the standard orthogonal table with the factor number; Step 13: searching for a known mixed level orthogonal table with the same factor number and level number, matching the corresponding mixed level orthogonal table if the mixed level orthogonal table exists, or entering step 14; Step 14: If the number of levels of each factor is only two, judge whether the number of factors and the number of levels satisfy the condition of the orthogonal table of type If the condition is satisfied, match the Hadamard matrix of order n, and construct a new orthogonal table according to the Hadamard matrix As a matched orthogonal table, otherwise go to step 15; If the number of levels of each factor is greater than two, go to step 15; Step 15: searching for a standard orthogonal table with the same factor number and maximum level number, matching the corresponding standard orthogonal table if the standard orthogonal table exists, or entering step 16; constructing a new orthogonal table according to a quasi-level method; constructing a new orthogonal table according to a modified two-level orthogonal table method; constructing a new orthogonal table according to a combined factor method; and matching the corresponding orthogonal table; Step 16: searching for a standard orthogonal table with the same maximum level number, greater than the factor number and the minimum difference with the factor number; searching for a standard orthogonal table with the same factor number, greater than the maximum level number and the minimum difference with the level number; searching for a standard orthogonal table greater than the factor number and greater than the maximum level number, and the minimum difference with the factor number and the level number; and matching the corresponding standard orthogonal table; The process of constructing the orthogonal table by using the modified two-level orthogonal table method is as follows: Step 121: screening out all factors with a level number not equal to two; Step 122: virtually splitting the screened factors into a plurality of two-level factors after the level numbers of the screened factors are virtually multiplied by two; Step 123: matching the same standard orthogonal table according to the virtual factor number and the level number.

2. The method for automatically generating test cases based on orthogonal table according to claim 1, wherein, The process of constructing the orthogonal table by using the quasi-level method is as follows: Step 111: searching for a standard orthogonal table with the same factor number and maximum level number; Step 112: sorting the factors according to the level number, combining the factors with adjacent different level numbers, virtually increasing the small level number in the combination to a large level number, and matching the same mixed level orthogonal table according to the virtual factor number and the level number.

3. The method of claim 1, wherein the orthogonal array based test case automatic generation method is characterized by, The process of constructing the orthogonal table by using the combined factor method is as follows: Step 131: selecting the maximum level number in the factors; Step 132: combining the factors with the same level number two by two, and the level number after the combination is the product of the original level numbers of the two groups, and the combination is terminated when the level number after the combination is greater than the maximum level number; Step 133: match the orthogonal table according to the combined factors and the corresponding level number using the quasi-level method.

4. The method of claim 1, wherein the orthogonal array based test case automatic generation method is characterized by, Record the orthogonal table data in the matched orthogonal table in step 2, and generate test cases.

5. The method of claim 1, wherein the orthogonal array based test case automatic generation method is characterized by, The process of selecting the optimal test case set according to the preset evaluation rule is: Step 31: sort the test case sets according to the number of test cases from small to large, select the second test case set, and set the number of test cases corresponding to the selected test case set as the lower limit num_min; and set the acceptable test case number gap value num_gap; Step 32: filter out the test case set with a test case number less than or equal to num_min+num_gap from all test case sets; Step 33: calculate the evaluation index corresponding to the filtered test case set; Step 34: select the optimal test case set according to the evaluation index and the preset index constraint.

6. The method of claim 5, wherein the orthogonal array based test case automatic generation method is characterized by, The evaluation index includes whether each level of each factor is covered by the test case, the uniformity of the level distribution of each corresponding factor, the average weight of the test case, and the case reduction rate.

7. The method of claim 6, wherein the orthogonal array based test case automatic generation method is characterized by, The case reduction rate is calculated using the evaluation selection model, and the expression is: Wherein, OrthogonalNum represents the number of generated test cases; CompleteNum represents the number of test cases generated by using the comprehensive test method.

Images