A test case priority calculation method, system, electronic device, and medium

Abstract

The application discloses a test case priority calculation method, system, electronic equipment and medium. The method specifically comprises the following steps: step S1, calculating test case coverage and test case relevance to obtain the test case priority of the first round of testing; step S2, performing the first round of testing according to the test case priority of the first round of testing obtained in step S1, recording the effectiveness and defects of the test case, and then determining the test case priority of the second round of testing according to the defect detection capability of the test case; and step S3, performing the second round of testing according to the test case priority of the second round of testing obtained in step S2. The method can significantly improve the defect discovery rate.

Description

Technical Field

[0001] This application relates to the field of software testing technology, and more specifically, to a test case priority calculation method, system, electronic device, and medium. Background Technology

[0002] With the increasing size of current software systems, the importance of software testing is becoming increasingly prominent. However, the software industry currently experiences rapid updates and iterations with short release cycles, making software testing increasingly demanding. At the same time, the execution efficiency of test cases, which are highly dependent on each other during the software testing process, has not improved. The main reason is that during the test case generation process, the priority of test cases is generally determined subjectively by the tester, lacking a clear priority calculation method. Furthermore, due to the lack of priority, test cases are usually executed in the order they are generated, resulting in low efficiency during test case execution and a low defect discovery rate.

[0003] Therefore, the software testing industry needs standardized methods for calculating test case priorities to improve the efficiency of test case execution. Summary of the Invention

[0004] In view of this, the present invention proposes a test case priority calculation method, system, electronic device, and medium.

[0005] To achieve the above technical objectives, the technical solution of the present invention is as follows: A first aspect of the present invention provides a method for calculating test case priority, the method specifically including the following steps:

[0006] Step S1: Calculate test case coverage and test case correlation to obtain the test case priority for the first round of testing;

[0007] Step S2: Execute the first round of tests according to the test case priority obtained in step S1, record the validity and defects of the test cases, and then determine the test case priority for the second round of tests based on the defect detection capability of the test cases.

[0008] Step S3: Execute the second round of tests according to the test case priority obtained in step S2.

[0009] Furthermore, step S1 specifically includes the following sub-steps:

[0010] S101, Calculate test case coverage: The test case coverage is obtained by comparing the statements covered by the test cases with all statements involved in the new and optimized requirements during the software iteration.

[0011] S102, Calculate test case correlation: Based on test case coverage, calculate the correlation between each test case and all other test cases, and sum them up to obtain the overall test case correlation of each test case;

[0012] S103, Obtain the priority of test cases in the first round of testing: Weight the test case coverage and test case relevance to obtain the priority of test cases in the first round.

[0013] Furthermore, the process of calculating the correlation between each test case and all other test cases, and then summing them up to obtain the overall test case correlation for each test case, includes:

[0014] The ratio of the intersection of the statements covered by the first test case and the statements covered by the second test case to the union of the statements covered by the first test case and the statements covered by the second test case is used as the correlation between the first test case and the second test case. This process is repeated to obtain the overall correlation of the first test case. The above steps are repeated to obtain the overall correlation of each test case.

[0015] Furthermore, recording the validity and defects of test cases includes:

[0016] Record the defects found during the execution of test cases, the correspondence between defects and test cases, evaluate the effectiveness of test cases, and record the modified and removed test cases.

[0017] Furthermore, the process of evaluating the effectiveness of test cases and documenting modified or removed test cases includes:

[0018] If a test case is not invalid, it does not need to be modified; if a test case is invalid, it needs to be removed; if a test case is partially valid, it needs to be modified according to the test scenario and / or test function.

[0019] Furthermore, the defect detection capability of the test cases is determined by the number of defects found when executing the test cases, the ratio of statements involved before and after defect fixing, and the number of verifications after defect fixing.

[0020] Furthermore, the priority of test cases in the second round of testing is determined based on their defect detection capabilities, including:

[0021] The priority of test cases in the first round is determined by weighting the priority of test cases, the validity of test cases, and the defect detection capability of test cases.

[0022] A second aspect of this invention provides a test case priority calculation system for implementing the above-described test case priority calculation method, the system comprising:

[0023] The first round of testing uses a test case priority determination unit to calculate test case coverage and test case correlation to obtain the test case priority for the first round of testing.

[0024] The first round of test execution unit executes the first round of tests according to the priority of the test cases in the first round, records the validity and defects of the test cases, and then determines the priority of the test cases in the second round of tests based on the defect detection capability of the test cases;

[0025] The second round of test execution unit executes the second round of tests according to the priority of the test cases in the second round.

[0026] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-described test case priority calculation method.

[0027] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the above-described test case priority calculation method.

[0028] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention proposes a test case priority calculation method. The priority of test cases in the first round of testing is determined by test case coverage and test case correlation. During the execution of the first round of testing, the validity and defects of the test cases are recorded. Then, the priority of test cases in the second round of testing is determined based on the defect detection capability of the test cases, and finally, the second round of testing is performed. By comparing the APFD (Advanced Processing Defect Finding Rate) after executing test cases with the default sorting, with the APFD after priority calculation and execution of test cases in priority order, it can be found that the defect finding rate of the method of this invention is significantly improved after priority calculation and execution of test case sequences ordered according to the calculation results. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is an exemplary flowchart of a test case priority calculation method according to some embodiments of this specification.

[0031] Figure 2This is an exemplary block diagram of a test case priority calculation method according to some embodiments of this specification;

[0032] Figure 3 This is an exemplary block diagram illustrating the execution process of the first round of test cases according to some embodiments of this specification;

[0033] Figure 4 This is an exemplary block diagram illustrating the execution process of the second round of test cases according to some embodiments of this specification;

[0034] Figure 5 This is an exemplary flowchart of a test case priority calculation system according to some embodiments of this specification;

[0035] Figure 6 This is a schematic diagram of an electronic device according to some embodiments of this specification. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0037] This invention proposes a test case priority calculation method. Based on the different objectives of the two rounds of testing, test case priorities are calculated separately for each round. In the first round of testing, since the version functionality is initially transferred to testing, the completion rate of requirements and the extent of defects are still unclear. The test cases executed in the first round need to cover the iteration requirements as quickly as possible, while also improving the defect discovery rate so that developers have sufficient time to fix defects. Therefore, the test case priority in the first round of testing is calculated through test case coverage and test case relevance. Test case coverage refers to the statement coverage of the test cases for the current iteration functionality. Test cases with a wider coverage range need to be executed first, thereby improving the defect discovery rate. Test case relevance refers to the degree of similarity between test cases. Test cases with a higher degree of similarity have a higher scope of verification and higher consistency of results, meaning they are more likely to cause duplicate verification. Therefore, test cases with low relevance need to be executed first to improve testing efficiency. In the second round of testing, after the first round of testing, the completion status of requirements has been understood and some defects have been submitted. The purpose of the second round of testing is to verify whether the defects and their impact range discovered in the first round have been fixed and usable, to further confirm the existing normal functionality, and to improve test cases that need modification and remove invalid test cases. Therefore, the priority of test cases in this round of testing needs to be calculated by adding test case defect detection capability and test case validity to the priority of test cases based on the first round of testing.

[0038] like Figure 1 and Figure 2 As shown, this invention proposes a method for calculating test case priority, the method comprising the following steps:

[0039] Step S1: Calculate test case coverage and test case correlation to obtain the test case priority for the first round of testing.

[0040] Step S1 specifically includes the following sub-steps:

[0041] S101, Calculate test case coverage: The test case coverage is obtained by comparing the statements covered by test cases with all statements involved in new or optimized requirements during software iterations; the calculation formula is as follows:

[0042]

[0043] Among them, Cover(n i ) represents test case coverage, i represents the number of test cases, and Sentence n Sentence represents the statement covered by test case n. all This represents all statements involved in the new and optimized requirements during the i-th iteration.

[0044] S102, Calculate test case relevance: Based on test case coverage, calculate the relevance between each test case and other test cases, and sum them to obtain the overall test case relevance (n) for each test case. i ).

[0045] For example, the statement covered by the first test case n1 is denoted as Sentence. n1 Let the statement covered by the second test case n2 be denoted as Sentence. n2 Calculate the relevance between the first test case n1 and the second test case n2. n1n2 The formula is as follows:

[0046]

[0047] Then calculate the correlation between the first test case n1 and other test cases, and sum them up to obtain the overall test case correlation Relevance(n1) of the first test case n1.

[0048] S103, Obtain the priority of test cases in the first round of testing: Weight the test case coverage and test case relevance to obtain the priority of test cases in the first round.

[0049] The calculation formula is as follows:

[0050] Priority1(n i)=α1Cover(n i )+α2Relevance(n i )

[0051] Where α1 is the weighted system corresponding to test case coverage, and α2 is the weighted coefficient corresponding to test case correlation.

[0052] Step S2: Execute the first round of tests according to the test case priority obtained in step S1, record the validity and defects of the test cases, and then determine the test case priority for the second round of tests based on the defect detection capability of the test cases.

[0053] like Figure 3 As shown, the step S2 of recording the validity and defects of test cases includes: executing the first round of tests according to the priority of the test cases obtained in step S1, recording the defects found during the execution of test cases, including the correspondence between defects and test cases, recording the modified and removed test cases, and evaluating the validity of test cases.

[0054] The validity of the test cases (Validity(n)) i The expression for ) is as follows:

[0055]

[0056] Specifically, based on the execution results of the first round of test cases, some test case scenarios were modified and improved, and invalid test cases were removed. Invalid test cases do not need to be executed again in the second round of testing. For modified test cases, there may be functional implementations that do not conform to the original design, or test scenarios that were omitted in the original test cases have been supplemented. In the subsequent second round, the modified test cases need to be monitored.

[0057] Furthermore, the defect detection capability of the test cases is determined by the number of defects found during test case execution, the ratio of statements involved before and after defect fixing, and the number of verifications performed after defect fixing. The defect detection capability of the test cases is Detection(n). i The expression for ) is as follows:

[0058]

[0059] Where m represents the number of defects found when executing the i-th test case, and Sentence bug Sentences that indicate the statements involved in defect repair. used This indicates the statements involved before the defect was fixed, and k indicates the number of verifications after the defect was fixed.

[0060] Determining the priority of test cases for the second round of testing based on their defect detection capabilities involves weighting the priority of the test cases from the first round, their effectiveness, and their defect detection capabilities to determine the priority of the test cases for the second round. The expression is as follows:

[0061]

[0062] Where ε1 is the priority of the first round of test cases. The corresponding weighting coefficient, ε2, represents the validity (n) of the test cases. i The corresponding weight coefficient, ε3, represents the defect detection capability of the test case Detection(n). i The corresponding weighting coefficient.

[0063] Step S3: Execute the second round of tests according to the test case priority obtained in step S2.

[0064] like Figure 4 As shown, step S3 specifically involves: executing the second round of testing on the test cases modified and removed in step S2 according to the test case priority obtained in step S2. During the execution of the second round of testing, defects are discovered and recorded, and verification is performed after the defects are fixed.

[0065] Example 1

[0066] Step S1: Calculate test case coverage and test case correlation to obtain the test case priority for the first round of testing.

[0067] The test case set T output in a certain iteration contains 6 test cases, T = {t1, t2, t3, t4, t5, t6}. After calculating the statement coverage of each test case, the data in Table 1 is obtained:

[0068] Table 1: Statement Coverage Table for Each Test Case

[0069] Use Cases / Coverage <![CDATA[t1]]> <![CDATA[t2]]> <![CDATA[t3]]> <![CDATA[t4]]> <![CDATA[t5]]> <![CDATA[t6]]> <![CDATA[Cover(t i )]]> 0.07 0.19 0.23 0.26 0.175 0.092

[0070] After calculating the correlation and number of correlations between each use case and other use cases, the data in Table 2 is obtained:

[0071] Table 2: Relationships between each use case and other use cases

[0072] Use Cases / Relevance <![CDATA[t1]]> <![CDATA[t2]]> <![CDATA[t3]]> <![CDATA[t4]]> <![CDATA[t5]]> <![CDATA[t6]]> <![CDATA[Relevance(t i )]]> 0.54 0.27 0.61 0.177 0.35 0.217

[0073] The test case priorities for the first round of testing were calculated, resulting in the data shown in Table 3:

[0074] Table 3: Test Case Table for the First Round of Testing

[0075] Use Cases / Priorities <![CDATA[t1]]> <![CDATA[t2]]> <![CDATA[t3]]> <![CDATA[t4]]> <![CDATA[t5]]> <![CDATA[t6]]> <![CDATA[Priority1(t i )]]> 0.22 0.23 0.36 0.24 0.23 0.13

[0076] Based on the priority calculation results, the priority order of the first round of test cases is t3-t4-t2-t5-t1-t6.

[0077] Step S2: Execute the first round of tests according to the test case priority obtained in step S1, record the validity and defects of the test cases, and then determine the test case priority for the second round of tests based on the defect detection capability of the test cases.

[0078] After the first round of testing, test case t2 was invalid, t3 found 1 defect, and t4 found 2 defects. The validity of each test case in test case set T is shown in Table 4 below:

[0079] Table 4: Validity Table for Each Test Case

[0080] Use Cases / Validity <![CDATA[t1]]> <![CDATA[t2]]> <![CDATA[t3]]> <![CDATA[t4]]> <![CDATA[t5]]> <![CDATA[t6]]> <![CDATA[Validity(t i )]]> 1 0 1 1 1 1

[0081] The defect detection capabilities of each test case in test case set T are shown in Table 5 below:

[0082] Table 5: Defect Detection Capability of Each Test Case

[0083] Use case / defect detection capabilities <![CDATA[t1]]> <![CDATA[t2]]> <![CDATA[t3]]> <![CDATA[t4]]> <![CDATA[t5]]> <![CDATA[t6]]> <![CDATA[Detection(t i )]]> 0 0 0.33 0.72 0 0

[0084] To compare the defect detection rate of the priority-calculated test case sequence with the default sequence, the APFD values ​​for the two test case sequences were calculated separately. APFD (Average Percentage of Fault Detection) is a metric used to measure defect detection rate, and its calculation formula is as follows:

[0085]

[0086] Where n represents the number of test cases, m represents the number of defects, and TF i This indicates the position of the first test case that detects the i-th defect in the test case sequence. The closer the result is to 100%, the faster the defect detection rate.

[0087] The APFD calculation result of the default sequence is 42%, while the APFD calculation result after priority calculation and sorting is 81%. Therefore, the defect rate is significantly improved by using the test case sequence after priority calculation.

[0088] Based on the effectiveness of the test cases and the defect detection capability of the first round of testing, the priority of the test cases for the second round of testing is calculated as shown in Table 6 below:

[0089] Table 6: Test Case Priority Table for the Second Round of Testing

[0090]

[0091]

[0092] The order of the second round of test cases after priority sorting is as follows: t4-t3-t5-t1-t6-t2, where test case t2 is invalid and does not need to be executed.

[0093] Step S3: Execute the second round of tests according to the test case priority obtained in step S2.

[0094] After the second round of testing, t3 found one defect. By comparing the APFD of the second round execution order with that of the default sorting, it was calculated that the APFD value of the default sorting was 58%, while the APFD value of the second round execution sequence was 75%. The defect discovery rate of the test case sequence after priority sorting was significantly higher than that of the test case sequence with default sorting.

[0095] Therefore, the method of the present invention can effectively improve the defect discovery rate and thus improve the testing efficiency by calculating the priority of test cases to improve the execution sequence of test cases.

[0096] like Figure 5 As shown, this embodiment of the invention also provides a test case priority calculation system, which includes:

[0097] The first round of testing uses a test case priority determination unit to calculate test case coverage and test case correlation to obtain the test case priority for the first round of testing.

[0098] The first round of test execution unit executes the first round of tests according to the priority of the test cases in the first round, records the validity and defects of the test cases, and then determines the priority of the test cases in the second round of tests based on the defect detection capability of the test cases;

[0099] The second round of test execution unit executes the second round of tests according to the priority of the test cases in the second round.

[0100] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0101] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0102] Accordingly, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the test case priority calculation method as described above. Figure 6 The diagram shown is a hardware structure diagram of any device with data processing capabilities where the test case priority calculation method provided in this embodiment of the invention is located, except... Figure 6 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0103] Accordingly, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the test case priority calculation method described above. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.

[0104] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the full scope of the claims.

Claims

1. A method for calculating test case priority, characterized in that, The method specifically includes the following steps: Step S1: Calculate test case coverage and test case correlation to obtain the test case priority for the first round of testing; Specifically, step S1 includes the following sub-steps: S101, Calculate test case coverage: The test case coverage is obtained by comparing the statements covered by the test cases with all statements involved in the new and optimized requirements during the software iteration. S102, Calculate test case correlation: Based on test case coverage, calculate the correlation between each test case and all other test cases, and sum them up to obtain the overall test case correlation of each test case; The process of calculating the correlation between each test case and all other test cases, and then summing them to obtain the overall test case correlation for each test case, includes: The ratio of the intersection of the statements covered by the first test case and the statements covered by the second test case to the union of the statements covered by the first test case and the statements covered by the second test case is used as the correlation between the first test case and the second test case. This process is repeated to obtain the overall correlation of the first test case. The above steps are repeated to obtain the overall correlation of each test case. S103, Obtain the priority of test cases in the first round of testing: Weight the test case coverage and test case relevance to obtain the priority of test cases in the first round; Step S2: Execute the first round of tests according to the test case priority obtained in step S1, record the validity and defects of the test cases, and then determine the test case priority for the second round of tests based on the defect detection capability of the test cases. Among them, determining the priority of test cases for the second round of testing based on their defect detection capabilities includes: The priority of the test cases in the first round is determined by weighting the priority of the test cases, the validity of the test cases, and the defect detection capability of the test cases; Step S3: Execute the second round of tests according to the test case priority obtained in step S2.

2. The test case priority calculation method according to claim 1, characterized in that, Recording the validity and defects of test cases includes: Record the defects found during the execution of test cases, the correspondence between defects and test cases, evaluate the effectiveness of test cases, and record the modified and removed test cases.

3. The test case priority calculation method according to claim 2, characterized in that, The process of evaluating the effectiveness of test cases and recording modified or removed test cases includes: If a test case is not invalid, it does not need to be modified; if a test case is invalid, it needs to be removed; if a test case is partially valid, it needs to be modified according to the test scenario and / or test function.

4. The test case priority calculation method according to claim 1, characterized in that, The defect detection capability of the test cases is determined by the number of defects found when executing the test cases, the ratio of statements involved before and after defect fixing, and the number of verifications after defect fixing.

5. A test case priority calculation system, used to implement the test case priority calculation method according to any one of claims 1-4, characterized in that, The system includes: The first round of testing uses a test case priority determination unit to calculate test case coverage and test case correlation to obtain the test case priority for the first round of testing. The first round of test execution unit executes the first round of tests according to the priority of the test cases in the first round, records the validity and defects of the test cases, and then determines the priority of the test cases in the second round of tests based on the defect detection capability of the test cases; The second round of test execution unit executes the second round of tests according to the priority of the test cases in the second round.

6. An electronic device comprising a memory and a processor, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the test case priority calculation method according to any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the test case priority calculation method as described in any one of claims 1-4.

Images