Software test case priority ranking method and system based on multiple factors

Abstract

The invention discloses a software test case priority ranking method and system based on multiple factors, and the method comprises the steps: obtaining defect evaluation results of all entities of tested software; obtaining a test case based on the tested software; obtaining code coverage information and diversity of the test cases based on the test cases; performing linear weighted integration on the basis of the defect evaluation result, the code coverage information and the diversity of the test cases to obtain a priority ranking objective function; and sorting the test cases by adopting a greedy algorithm based on the priority sorting target function to obtain a priority sorting result. And meanwhile, the test cases are subjected to priority ranking based on three factors of defect evaluation, code coverage and test case diversity, so that the software defects are detected as quickly as possible, and the software defect detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and more particularly to a method and system for prioritizing software testing cases based on multiple factors. Background Art

[0002] Software test case prioritization is a method of rearranging the order of software test cases according to a certain standard, and achieving the goal of software testing by executing the ordered test cases, such as detecting defects as quickly as possible. The traditional test case prioritization method is a code coverage-based test case prioritization method using a greedy algorithm, namely total code coverage (Total) and additional code coverage (Additional). Both methods are based on the premise that "test cases with high coverage are more likely to detect faults". Since the traditional method assumes that defects are evenly distributed in the software, it does not conform to the "80 / 20 rule" (that is, 80% of the defects in the software exist in 20% of the software units). Therefore, a code coverage-based method considering defect assessment is proposed to introduce the factor of defect distribution.

[0003] The existing software test case prioritization methods based on code coverage that consider defect assessment are mainly divided into two categories: one is to directly integrate the defect assessment results and code coverage to form a variant of the software test case prioritization method based on code coverage, such as the quality-aware test case prioritization method QETP proposed by Wang et al. This method has the problem that when the defect assessment result is 0, the contribution of code coverage is also offset, resulting in a large number of ties in the case priority sorting, and the real case ranking cannot be obtained; the other is to fully trust the defect assessment results, take defect assessment as the main goal and code coverage as the secondary goal, and sort test cases based on code coverage based on the defect assessment results, such as Paterson et al. proposed a TCP evaluation comparison strategy G-clef. This method has the problem that when the defect assessment is inaccurate, the priority sorting based on this benchmark will inevitably have a large deviation. In addition, diverse test cases can make the test more comprehensive and more likely to expose defects, while the existing methods considering defect assessment do not consider the diversity of test cases.

[0004] Therefore, how to prioritize test cases based on three factors, defect assessment, code coverage, and test case diversity, and thus improve the efficiency of software defect detection is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0005] In view of this, the present invention provides a software test case prioritization method and system based on multiple factors, which prioritizes test cases based on three factors: defect assessment, code coverage, and test case diversity, so as to detect as many software defects as possible and as quickly as possible, thereby improving the efficiency of software defect detection.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A software test case prioritization method based on multiple factors, including:

[0008] Obtain defect assessment results for all entities of the software under test;

[0009] Acquire test cases based on the software under test;

[0010] Obtaining code coverage information and test case diversity based on the test cases;

[0011] Performing linear weighted integration based on the defect assessment results, the code coverage information, and the test case diversity to obtain a priority sorting objective function;

[0012] Based on the priority sorting objective function, a greedy algorithm is used to sort the test cases to obtain a priority sorting result.

[0013] Preferably, the defect assessment result acquisition method is:

[0014] Obtaining historical defect data and current version measurement metadata of all entities of the tested software;

[0015] Based on the historical defect data as a training data set input into the machine learning algorithm, a defect detection model is constructed;

[0016] The defect assessment result is obtained by inputting the measurement metadata based on the current version into the defect detection model.

[0017] Preferably, the code coverage information is obtained based on the execution status of historical test cases;

[0018] When the historical test case covers a certain entity, it is set to 1, and when it does not cover, it is set to 0.

[0019] Preferably, the test case diversity TCD (TC j ) specifically:

[0020]

[0021] Among them, TC jrepresents the j-th test case, m represents the number of entities of the tested software, q represents the number of test cases, Cover(j,l) represents the coverage of the i-th entity by the l-th test case, which is 1 if covered and 0 if not covered. Cover(j,i) represents the coverage of the i-th entity by the j-th test case, which is 1 if covered and 0 if not covered.

[0022] Preferably, the priority ranking objective function f(TC j ) specifically:

[0023]

[0024] Among them, n i represents the number of entities covered by test cases, P(TC j ) represents the defect assessment result, C(TC j ) represents the code coverage information, w1, w2 and w3 represent the defect assessment result weighting coefficient, the code coverage information weighting coefficient and the test case diversity weighting coefficient respectively.

[0025] Preferably, the relationship between the defect assessment result weighting coefficient w1, the code coverage information weighting coefficient w2 and the test case diversity weighting coefficient w3 satisfies the following formula:

[0026] w1+w2+w3=1.

[0027] Preferably, obtaining the priority sorting result specifically includes:

[0028] The greedy algorithm includes Total and Additional;

[0029] Based on the priority sorting objective function, the greedy algorithms Total and Additional are respectively used to obtain a first sorting result and a second sorting result as the priority sorting result;

[0030] The first priority sorting result acquisition method is:

[0031] Based on the priority sorting objective function, the greedy algorithm Total is used as a sorting strategy to obtain a first sorting value for each of the test cases;

[0032] Based on the sizes of all the first sorting values, descending sorting is performed to obtain a first sorting result.

[0033] Preferably, a greedy algorithm Additional is used as a sorting strategy based on the priority sorting objective function;

[0034] Obtaining the test case corresponding to the maximum value in the first sorting result as the first test case;

[0035] Based on the first test case, code coverage information of other test cases is updated, and second ranking values ​​of other test cases other than the first test case are calculated;

[0036] Selecting a test case corresponding to the largest second ranking value as a second test case;

[0037] Based on the second test case, code coverage information of test cases other than the first test case and the second test case is updated, and a third ranking value of the corresponding test case is calculated accordingly;

[0038] Selecting a test case corresponding to the largest third ranking value as a third test case;

[0039] Repeat the above process until the last test case is obtained;

[0040] The order in which the test cases are screened is used as a sorting result, as the second sorting result.

[0041] Preferably, the method further comprises: evaluating the first sorting result and the second sorting result:

[0042] Execute all the test cases based on the first sorting result and the second sorting result respectively, and obtain a first execution result and a second execution result of each test case accordingly;

[0043] A priority metric is selected to evaluate the first execution result and the second execution result respectively.

[0044] A software test case priority ranking system based on multiple factors, comprising: a data acquisition module, an objective function construction module and a ranking result output module;

[0045] The data acquisition module is used to obtain defect assessment results of all entities of the tested software; obtain test cases based on the tested software; and obtain code coverage information and test case diversity based on the test cases;

[0046] The objective function construction module is used to perform linear weighted integration based on the defect assessment result, the code coverage information and the test case diversity to obtain a priority sorting objective function;

[0047] The sorting result output module is used to sort the test cases using a greedy algorithm based on the priority sorting objective function to obtain a priority sorting result.

[0048] Through the above technical solution, it can be known that compared with the prior art, the present invention discloses a method and system for prioritizing software test cases based on multiple factors, and at the same time, based on the three factors of defect assessment, code coverage and test case diversity, combined with the traditional Total strategy and Additional strategy, the test case priority is sorted, avoiding the unreasonable assumption that defects are evenly distributed in the software, and considering multiple factors that affect the priority order of test cases. Through linear weighted integration, the test case priority sorting problem is converted into a single-objective problem, and the detection rate of software defects is improved; the present invention linearly weighted integrates the three factors to obtain the objective function, so that any deviation in any of the three factors will not affect the influence of the other two factors on the test case priority sorting, breaking the assumption that defects are evenly distributed in the software: the larger the comprehensive value of code coverage, case diversity, and defect probability, the higher the probability of the test case detecting a defect. By considering multiple factors that affect the priority sorting of test cases, the defect detection efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0050] Figure 1 A flowchart of a software test case priority sorting method based on multiple factors provided by the present invention.

[0051] Figure 2 This is a flow chart of the second sorting result obtaining method provided by the present invention.

[0052] Figure 3 A schematic diagram of the structure of a software test case priority sorting system based on multiple factors provided by the present invention.

[0053] Figure 4 This is a structural block diagram of the computer device provided by the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] Example 1

[0056] like Figure 1 As shown, an embodiment of the present invention discloses a software test case priority sorting method based on multiple factors, including:

[0057] Obtain defect assessment results for all entities of the software under test;

[0058] Get test cases based on the software under test;

[0059] Get code coverage information and test case diversity based on test cases;

[0060] Based on the defect assessment results, code coverage information and test case diversity, a linear weighted integration is performed to obtain the priority sorting objective function;

[0061] Based on the priority sorting objective function, a greedy algorithm is used to sort the test cases to obtain the priority sorting result.

[0062] Example 2

[0063] The embodiment of the present invention discloses a method for prioritizing software test cases based on multiple factors, comprising:

[0064] Get defect assessment results for all entities of the tested software:

[0065] Preferably, the defect assessment result acquisition method is:

[0066] Obtain historical defect data and current version measurement metadata for all entities of the tested software;

[0067] Based on the historical defect data as the training data set input into the machine learning algorithm, a defect detection model is constructed;

[0068] The measurement metadata based on the current version is input into the defect detection model to obtain the defect assessment result.

[0069] Preferably, the historical defect data includes historical measurement metadata and corresponding defect labels; and the defect assessment result is a defect label prediction result corresponding to the measurement metadata of the current version.

[0070] Preferably, in this embodiment, the machine learning algorithm includes but is not limited to various types of support regression or classification algorithms such as logistic regression, support vector machine, and ensemble learning.

[0071] Preferably, entities are divided into at least different levels such as method level, unit level, statement level, etc.

[0072] Get test cases based on the software under test.

[0073] Preferably, the design is performed based on the software under test to obtain multiple test cases with known output results.

[0074] Get code coverage information and test case diversity based on test cases:

[0075] Preferably, code coverage information is obtained based on the execution status of historical test cases; when a historical test case covers an entity, it is set to 1, and when it does not cover an entity, it is set to 0.

[0076] Preferably, the code granularity covered by the test cases has different levels such as method level, unit level, statement level, etc. When the statistical execution status is at the statement level, the focus is on the statements covered by the test cases and the statements not covered by the test cases; when the statistical execution information is at the method level, the focus is on the methods covered by the test cases and the methods not covered by the test cases.

[0077] Taking method-level coverage as an example, among all methods, when a test case covers a method, it is set to 1, and when a method is not covered, it is set to 0. Thus, the method-level code coverage information of all test cases is obtained.

[0078] Preferably, test case diversity TCD (TC j ) specifically:

[0079]

[0080] Among them, TC j represents the jth test case, m represents the number of entities of the tested software, q represents the number of test cases, Cover(j,l) represents the coverage of the i-th entity by the l-th test case, which is 1 if covered and 0 if not covered. Cover(j,i) represents the coverage of the i-th entity by the j-th test case, which is 1 if covered and 0 if not covered. It represents the ratio of the number of test cases covering the i-th entity to the total number of test cases q.

[0081] Preferably, based on the above formula, TCD(TC j ) value, the larger the test case's priority is.

[0082] Preferably, the test case diversity is a new test case diversity measurement method proposed by the present invention, which assumes that the higher the proportion of code covered by a test case is covered by other test cases, the lower the priority of the test case is, and the diversity of test cases is ensured by covering different codes with different test cases as much as possible. j ) represents the test case diversity, and its value range is [0, 1].

[0083] Based on the linear weighted integration of defect assessment results, code coverage information and test case diversity, the priority sorting objective function is obtained:

[0084] Preferably, the priority ranking objective function f(TC j ) specifically:

[0085]

[0086] Among them, n i represents the number of entities covered by test cases, P(TC j ) represents the defect assessment result, C(TC j ) represents the code coverage information, w1, w2 and w3 represent the defect assessment result weighting coefficient, the code coverage information weighting coefficient and the test case diversity weighting coefficient respectively.

[0087] Preferably, the relationship between the defect assessment result weighting coefficient w1, the code coverage information weighting coefficient w2 and the test case diversity weighting coefficient w3 satisfies the following formula:

[0088] w1+w2+w3=1.

[0089] Preferably, this embodiment does not discuss the influence of the three factors on the test case priority sorting, and therefore, it is set to w1=w2=w3=1 / 3.

[0090] Based on the priority sorting objective function, a greedy algorithm is used to sort the test cases and the priority sorting result is obtained:

[0091] Preferably, obtaining the priority sorting result specifically includes:

[0092] Greedy algorithms include Total and Additional;

[0093] Based on the priority sorting objective function, the greedy algorithms Total and Additional are respectively used to obtain the first sorting result and the second sorting result as the priority sorting result;

[0094] Among them, the method for obtaining the first priority sorting result is:

[0095] Based on the priority sorting objective function, the greedy algorithm Total is used as the sorting strategy to obtain the first sorting value of each test case;

[0096] Based on the sizes of all the first sorting values, descending sorting is performed to obtain a first sorting result.

[0097] Preferably, the Total method arranges the test cases in descending order according to the total coverage number to obtain the test case priority ranking result; the Additional method arranges the test cases in descending order according to the total number of coverage and uncovered test cases to obtain the test case priority ranking result.

[0098] Preferably, when the greedy algorithm Total is selected as the sorting strategy, the calculation method of the three factors in the priority sorting objective function is determined, and the specific calculation formula is as follows:

[0099] Defect evaluation results P(TC j )for:

[0100]

[0101] Among them, Prob(j,i) represents the defect prediction value of the i-th method covered by the j-th test case, which is output by the defect detection model.

[0102] Code coverage information C(TC j )for:

[0103]

[0104] Among them, C(TC j ) by test case TC j The sum of the number of covered methods is obtained.

[0105] Based on the above formula and test case diversity TCD (TC j ) formula into the priority sorting objective function f(TC j ), w1=w2=w3=1 / 3, and get the Total objective function f(TC j ) Total :

[0106]

[0107] Based on the Total objective function f(TC j ) Total The corresponding first ranking value of each test case is obtained.

[0108] Preferably, the greedy algorithm Additional is used as a sorting strategy based on the priority sorting objective function;

[0109] like Figure 2 As shown, the test case corresponding to the maximum value in the first sorting result is obtained as the first test case;

[0110] Update code coverage information of other test cases based on the first test case, and calculate second ranking values ​​of other test cases other than the first test case;

[0111] Select the test case corresponding to the largest second ranking value as the second test case;

[0112] Update the code coverage information of the test cases other than the first test case and the second test case based on the second test case, and calculate the third ranking value of the corresponding test case accordingly;

[0113] Select the test case corresponding to the largest third ranking value as the third test case;

[0114] Repeat the above process until the last test case is obtained;

[0115] The order of screening the test cases is used as the sorting result, and as the second sorting result.

[0116] Preferably, when the greedy algorithm Additional is selected as the sorting strategy, the calculation method of the three factors in the priority sorting objective function is determined, and the specific calculation formula is as follows:

[0117] Defect evaluation results P(TC j )for:

[0118]

[0119] Among them, Prob(j,i) represents the defect prediction value of the i-th entity covered by the j-th test case, which refers to the defect probability of the entity covered by the remaining test cases that have not been executed. Prob(j',i) represents the defect prediction value of the i-th entity covered by the j'-th test case, which refers to the defect probability of the entity covered by the sorted test cases. The above two prediction values ​​are output by the defect detection model.

[0120] Code coverage information C(TC j )for:

[0121]

[0122] Among them, Cover(j,i) indicates the coverage of the i-th entity by the j-th test case, which refers to whether the remaining unexecuted test cases cover the entity, and is 1 if covered, and 0 if not covered; Cover(j',i) indicates the coverage of the i-th entity by the j'-th test case, which refers to whether the sorted test cases cover the entity, and is 1 if covered, and 0 if not covered.

[0123] Based on the above formula and test case diversity TCD (TC j ) formula into the priority sorting objective function f(TC j ), w1=w2=w3=1 / 3, and get the Additional objective function f(TC j ) Additionall :

[0124]

[0125] Based on the Additional objective function f(TC j ) Additionall Correspondingly, the second ranking value, the third ranking value, and finally the last ranking value of each test case are obtained.

[0126] Preferably, when n i =0, f(TC j )=0; when f(TC j ) are all 0, similar to the processing method of the Additional method, the coverage of all test cases is recalculated according to the original coverage.

[0127] Preferably, the method further includes: evaluating the first sorting result and the second sorting result:

[0128] Execute all test cases based on the first sorting result and the second sorting result respectively, and obtain the first execution result and the second execution result of each test case accordingly;

[0129] The priority metric is selected to evaluate the first execution result and the second execution result respectively.

[0130] Preferably, the priority metrics include: weighted average percentage of faults detected (APFD) and relative average percentage of faults detected (RAPFD).

[0131] Example 3

[0132] The present invention is verified based on the entity selection method:

[0133] This embodiment selects a software, including 6 methods and 5 test cases, wherein the 6 methods M={M1, M2, M3, M4, M5, M6}, the defect prediction evaluation results corresponding to the methods P={0.86, 0.06, 0.56, 0.12, 0.03, 0.21}, the 5 test cases T={TC1, TC2, TC3, TC4, TC5}, the execution results corresponding to the test cases are {pass, fail, pass, pass, pass}, and the relevant information of the tested software is shown in Table 1:

[0134] Table 1 Relevant information of the tested software in the specific implementation case

[0135]

[0136]

[0137] Among them, the second line indicates that test case TC1 covers method M1, method M3 and method M5; the third line indicates that test case TC2 covers method M2; the fourth line indicates that test case TC3 covers method M1, method M2, method M3 and method M6; the fifth line indicates that test case TC4 covers method M5 and method M6; the sixth line indicates that test case TC5 covers method M1, method M4 and method M5;

[0138] The prediction probability of method M1 is 0.86, the prediction probability of method M2 is 0.06, the prediction probability of method M3 is 0.56, the prediction probability of method M4 is 0.12, the prediction probability of method M5 is 0.03, and the prediction probability of method M6 is 0.21. The larger the defect prediction probability, the greater the possibility of defect.

[0139] Based on the information in Table 1, execute the greedy algorithms Total and Additional respectively;

[0140] Execute the greedy algorithm Total, denoted as MFTCP Total :

[0141] Substitute the information in Table 1 into the Total objective function f(TC j ) Total , corresponding to:

[0142]

[0143]

[0144]

[0145]

[0146]

[0147] in,

[0148]

[0149]

[0150]

[0151]

[0152]

[0153] Based on the above calculation results, descending sorting is performed to obtain the first sorting result: TC2-TC1-TC3-TC5-TC4.

[0154] Execute the greedy algorithm Additional, denoted as MFTCP Additional :

[0155] In the first round, the test case TC2 corresponding to the maximum value in the first sorting result is used as the first test case;

[0156] In the second round, the methods in other test cases that are the same as those covered by the first test case are updated and set to 0, resulting in Table 2:

[0157] Table 2 Second round update coverage results

[0158] <![CDATA[M1(0.86)]]> <![CDATA[M2(0.06)]]> <![CDATA[M3(0.56)]]> <![CDATA[M4(0.12)]]> <![CDATA[M5(0.03)]]> <![CDATA[M6(0.21)]]> Output <![CDATA[TC1]]> 1 0 1 0 1 0 pass <![CDATA[TC2]]> 0 1 0 0 0 0 Not passed <![CDATA[TC3]]> 1 0 1 0 0 1 pass <![CDATA[TC4]]> 0 0 0 0 1 1 pass <![CDATA[TC5]]> 1 0 0 1 1 0 pass

[0159] Among them, bold indicates the sorted test cases, and underline indicates the updated coverage according to the first test case;

[0160] Substitute the unsorted information in Table 2 into the Additional objective function f(TC j ) Additionall , corresponding to:

[0161]

[0162]

[0163]

[0164]

[0165] Based on the above calculation results, the test case TC3 corresponding to the maximum value is selected as the second test case;

[0166] In the third round, the methods in other test cases (excluding the first test case TC2) that are the same as those covered by the second test case are updated to 0, and Table 3 is obtained:

[0167] Table 3. Coverage results of the third round of updates

[0168] <![CDATA[M1(0.86)]]> <![CDATA[M2(0.06)]]> <![CDATA[M3(0.56)]]> <![CDATA[M4(0.12)]]> <![CDATA[M5(0.03)]]> <![CDATA[M6(0.21)]]> Output <![CDATA[TC1]]> 0 0 0 0 1 0 pass <![CDATA[TC2]]> 0 1 0 0 0 0 Not passed <![CDATA[TC3]]> 1 0 1 0 0 1 pass <![CDATA[TC4]]> 0 0 0 0 1 0 pass TC5 0 0 0 1 1 0 pass

[0169] Among them, bold indicates the sorted test cases, and underline indicates the updated coverage according to the second test case;

[0170] Substitute the unsorted information in Table 3 into the Additional objective function f(TC j ) Additionall , corresponding to:

[0171]

[0172]

[0173]

[0174] Since both f(TC1) and f(TC4) are 0.477, the largest one in the Total strategy is selected. Since f(TC1)>f(TC4) (0.428>0.373), TC1 is selected as the third test case;

[0175] In the fourth round, the methods in other test cases (excluding the first test case TC2 and the second test case TC3) that are the same as those covered by the third test case are updated to 0, and Table 4 is obtained:

[0176] Table 4. Coverage results of the fourth round of updates

[0177] <![CDATA[M1(0.86)]]> <![CDATA[M2(0.06)]]> <![CDATA[M3(0.56)]]> <![CDATA[M4(0.12)]]> <![CDATA[M5(0.03)]]> <![CDATA[M6(0.21)]]> Output <![CDATA[TC1]]> 0 0 0 0 1 0 pass <![CDATA[TC2]]> 0 1 0 0 0 0 Not passed <![CDATA[TC3]]> 1 0 1 0 0 1 pass <![CDATA[TC4]]> 0 0 0 0 0 0 pass <![CDATA[TC5]]> 0 0 0 1 0 0 pass

[0178] Among them, bold indicates the sorted test cases, and underline indicates the updated coverage according to the third test case;

[0179] Substitute the unsorted information in Table 4 into the Additional objective function f(TC j ) Additionall , corresponding to:

[0180] Since n4=0, according to the method of the present invention, f(TC4)=0;

[0181]

[0182] Based on the above calculation results, the test case TC5 corresponding to the maximum value is selected as the fourth test case; the test case TC4 is selected as the fifth test case;

[0183] The test cases are sorted based on the order of the test cases obtained above to obtain a second sorting result: TC2-TC3-TC1-TC5-TC4.

[0184] This application MFTCP Total The ranking results obtained by the method are: TC2-TC1-TC3-TC5-TC4;

[0185] This application MFTCP Additional The ranking results obtained by the method are: TC2-TC3-TC1-TC5-TC4;

[0186] According to the traditional Total method, the total method coverage value of each test case is calculated as {3, 1, 4, 2, 3}. If the coverage values ​​are the same, the first test case is selected, and the priority ranking result is: TC3-TC1-TC5-TC4-TC2;

[0187] According to the traditional Additional method, the total method additional coverage value of each test case is calculated. When the coverage values ​​are the same, the test case with the largest total coverage is selected. The priority sorting result is:

[0188] TC3-TC5-TC1-TC2-TC4.

[0189] Based on the sorting results obtained by the above method, all test cases are executed respectively, and the execution results of each test case are obtained accordingly;

[0190] Based on the execution results of the test cases, APFD is used to evaluate the performance of the above four methods:

[0191]

[0192] Among them, TF k It indicates the position of the first test case that finds defect k in the sorting strategy. The value range of k is [0, a]. a indicates the number of defects contained in the test case. b indicates the total number of test cases. The value range of APFD is [0, 1]. The larger the value, the better the performance of the sorting method.

[0193] According to the sorting results of the four methods, we can calculate

[0194] It can be seen that MFTCP Additional Methods and MFTCP Total Comparable and superior to the traditional Additional and Total methods.

[0195] Example 4

[0196] like Figure 3 As shown, a software test case priority ranking system based on multiple factors includes: a data acquisition module, an objective function construction module and a ranking result output module;

[0197] The data acquisition module is used to obtain defect assessment results of all entities of the software under test; obtain test cases based on the software under test; obtain code coverage information and test case diversity based on the test cases;

[0198] An objective function building module is used to perform linear weighted integration based on defect assessment results, code coverage information, and test case diversity to obtain a priority ranking objective function;

[0199] The sorting result output module is used to sort the test cases using a greedy algorithm based on the priority sorting objective function to obtain the priority sorting result.

[0200] Preferably, the above-mentioned software test case prioritization system based on multiple factors is applied to any of the software test case prioritization methods based on multiple factors such as Examples 1-3, which will not be described one by one here.

[0201] Example 5

[0202] Based on the same inventive concept, the present invention also provides a computer device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0203] Memory, used to store computer programs;

[0204] The processor, when used to execute the program stored in the memory, can implement a software test case priority sorting method based on multiple factors as in Example 1 or 2.

[0205] like Figure 4 As shown, the electronic device may include: a processor 41, a communication interface 42, a memory 43 and a communication bus 44, wherein the processor 41, the communication interface 42 and the memory 43 communicate with each other via the communication bus 44. The processor 41 may call the logic instructions in the memory 43 to execute a software test case priority sorting method based on multiple factors in Embodiment 1 or 2.

[0206] In addition, the logic instructions in the above-mentioned memory 43 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0207] The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0208] It can be seen from the above technical solution that the present invention discloses a software test case priority sorting method and system based on multiple factors, and sorts the priorities of test cases based on three factors: defect assessment, code coverage and test case diversity, combined with traditional Total strategy and Additional strategy, avoiding the unreasonable assumption that defects are evenly distributed in the software, and taking into account multiple factors affecting the priority order of test cases. Through linear weighted integration, the test case priority sorting problem is converted into a single-objective problem, thereby improving the detection rate of software defects.

[0209] The present invention integrates the three factors into a linear weighted integration to obtain an objective function, so that any deviation in any of the three factors will not affect the impact of the other two factors on the test case priority sorting, breaking the assumption that defects are evenly distributed in the software: the larger the comprehensive value of code coverage, test case diversity, and defect probability, the higher the probability of the test case detecting a defect. By considering multiple factors that affect the test case priority sorting, the defect detection efficiency is improved.

[0210] The present invention proposes a test case priority sorting method that considers three factors, namely, dynamic code coverage of test cases, defect prediction results, and diversity of test cases, to break the assumption of traditional test case priority sorting methods based on dynamic code coverage that "defects are evenly distributed in software"; in terms of test case diversity measurement, it is assumed that "the greater the diversity of test cases, the greater the probability of detecting defects", and a new measurement method is proposed based on the coverage of the code by test cases, which measures the diversity of use cases by the proportion of code covered by use cases, and integrates it into the sorting objective function to achieve the optimization of the sorting method; a target function construction method based on linear weighted integration is proposed to convert the multi-objective problem of three factors into a single-objective problem, while avoiding the problem that any one of the three factors returning to zero will affect the influence of other factors on the sorting; through the above key technical points, the sorting method proposed by the present invention can detect software defects as much as possible and as quickly as possible, thereby improving the software defect detection rate.

[0211] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0212] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A software test case prioritization method based on multiple factors, characterized in that: include: Obtain defect assessment results for all entities of the software under test; Acquire test cases based on the software under test; Obtaining code coverage information and test case diversity based on the test cases; Performing linear weighted integration based on the defect assessment results, the code coverage information, and the test case diversity to obtain a priority sorting objective function; Based on the priority sorting objective function, a greedy algorithm is used to sort the test cases to obtain a priority sorting result.

2. A software test case prioritization method based on multiple factors according to claim 1, characterized in that: The defect assessment result acquisition method is: Obtaining historical defect data and current version measurement metadata of all entities of the tested software; Based on the historical defect data as a training data set input into the machine learning algorithm, a defect detection model is constructed; The defect assessment result is obtained by inputting the measurement metadata based on the current version into the defect detection model.

3. A software test case prioritization method based on multiple factors according to claim 1, characterized in that: The code coverage information is obtained based on the execution status of historical test cases; When the historical test case covers a certain entity, it is set to 1, and when it does not cover, it is set to 0.

4. The method for prioritizing software test cases based on multiple factors according to claim 1, characterized in that: The test case diversity TCD (TC j ) Specifically: Among them, TC j represents the j-th test case, m represents the number of entities of the tested software, q represents the number of test cases, Cover(j,l) represents the coverage of the i-th entity by the l-th test case, which is 1 if covered and 0 if not covered. Cover(j,i) represents the coverage of the i-th entity by the j-th test case, which is 1 if covered and 0 if not covered.

5. A software test case prioritization method based on multiple factors according to claim 4, characterized in that: The priority sorting objective function f(TC j ) Specifically: Among them, n i represents the number of entities covered by test cases, P(TC j ) represents the defect assessment result, C(TC j ) represents the code coverage information, w1, w2 and w3 represent the defect assessment result weighting coefficient, the code coverage information weighting coefficient and the test case diversity weighting coefficient respectively.

6. A software test case prioritization method based on multiple factors according to claim 5, characterized in that: The relationship between the defect assessment result weighting coefficient w1, the code coverage information weighting coefficient w2 and the test case diversity weighting coefficient w3 satisfies the following formula: w1+w2+w3=1.

7. A software test case prioritization method based on multiple factors according to claim 1, characterized in that: The priority sorting results include: The greedy algorithm includes Total and Additional; Based on the priority sorting objective function, the greedy algorithms Total and Additional are respectively used to obtain a first sorting result and a second sorting result as the priority sorting result; The first priority sorting result acquisition method is: Based on the priority sorting objective function, the greedy algorithm Total is used as a sorting strategy to obtain a first sorting value for each of the test cases; Based on the sizes of all the first sorting values, descending sorting is performed to obtain a first sorting result.

8. A method for prioritizing software test cases based on multiple factors according to claim 7, characterized in that: Based on the priority sorting objective function, the greedy algorithm Additional is used as a sorting strategy; Obtaining the test case corresponding to the maximum value in the first sorting result as the first test case; Based on the first test case, code coverage information of other test cases is updated, and second ranking values ​​of other test cases other than the first test case are calculated; Selecting a test case corresponding to the largest second ranking value as a second test case; Based on the second test case, code coverage information of test cases other than the first test case and the second test case is updated, and a third ranking value of the corresponding test case is calculated accordingly; Selecting a test case corresponding to the largest third ranking value as a third test case; Repeat the above process until the last test case is obtained; The order in which the test cases are screened is used as a sorting result, as the second sorting result.

9. The method for prioritizing software test cases based on multiple factors according to claim 7, characterized in that: Also includes: Evaluate the first sorting result and the second sorting result: Execute all the test cases based on the first sorting result and the second sorting result respectively, and obtain a first execution result and a second execution result of each test case accordingly; A priority metric is selected to evaluate the first execution result and the second execution result respectively.

10. A software test case prioritization system based on multiple factors, applied to a software test case prioritization method based on multiple factors as claimed in any one of claims 1 to 9, characterized in that: include: Data acquisition module, objective function construction module and ranking result output module; The data acquisition module is used to obtain defect assessment results of all entities of the tested software; Acquire test cases based on the software under test; Obtaining code coverage information and test case diversity based on the test cases; The objective function construction module is used to perform linear weighted integration based on the defect assessment result, the code coverage information and the test case diversity to obtain a priority sorting objective function; The sorting result output module is used to sort the test cases using a greedy algorithm based on the priority sorting objective function to obtain a priority sorting result.

Images