Software test case test method and system

Abstract

The invention belongs to the field of software testing, and discloses a software test case testing method and system, and the method comprises the steps: S1, obtaining a set U of test cases for testing, and S11, calculating a risk value of a module based on a change record of a code of a module of software; s12, calculating a risk coverage weight of the test case based on the risk value; s13, calculating a defect prediction weight and a space-time efficiency weight of the test case based on the historical test record; s14, calculating a screening value of the test case based on the risk coverage weight, the defect prediction weight and the space-time efficiency weight; s15, acquiring a set U of test cases based on the screening values; and S2, using the test case in the set U to test a module needing to be tested, and obtaining a test record. According to the invention, the test priority of the high-risk module can be ensured, and a more effective test result can be obtained.

Description

Technical Field

[0001] The present invention relates to the field of software testing, and particularly to a method and system for testing software test cases. Background Art

[0002] A test case refers to a description of the test tasks for a specific software product, reflecting the test plan, methods, techniques, and strategies. During the software development process, test cases are usually used for multiple rounds of testing. After each round of testing, the discovered defects are modified in the code according to the test results, so as to obtain the code for the next round of testing.

[0003] In order to improve the testing efficiency within a limited testing time, the prior art usually automatically screens test cases so that as many effective tests as possible can be carried out within a limited time. During the screening process, the prior art usually only considers fixed attributes such as the coverage degree of test cases and the correlation between test cases, and does not dynamically combine the code changes for screening, resulting in insufficient testing effectiveness and possibly failing to prioritize the testing of high-risk code files. Summary of the Invention

[0004] The purpose of the present invention is to disclose a method and system for testing software test cases, and solve the technical problems pointed out in the background art.

[0005] In order to achieve the above purpose, the present invention adopts the following technical solutions: On the one hand, the present invention provides a method for testing software test cases, including: S1. Obtain a set U of test cases for testing, including: S11. Calculate the risk value of a module based on the change record of the code of the software module; S12. Calculate the risk coverage weight of the test case based on the risk value; S13. Calculate the defect prediction weight and the space-time efficiency weight of the test case based on the historical test record; S14. Calculate the screening value of the test case based on the risk coverage weight, the defect prediction weight, and the space-time efficiency weight; S15. Obtain the set U of test cases based on the screening value; S2. Use the test cases in the set U to test the module to be tested, and obtain the test record.

[0006] Further, before S1, it further includes: Establish a test case database.

[0007] Further, after S2, it further includes: Statistically analyze the test records and generate a test report.

[0008] Further, S11 includes: Express the total number of historical tests as K. Then, when the software is tested for the (K + 1)-th time, the calculation process of the risk value includes: Calculate the dynamic statistics count; Based on the dynamic statistics count, obtain the change count and change time density of the code for each module of the software respectively; Calculate the risk value for each module based on the change count and change time density respectively.

[0009] Further, S12 includes: Based on the dependency relationship between modules, correct the risk value of the module to obtain a risk correction value; Obtain the set of modules covered by each test case respectively; Calculate the risk coverage weight of the test case based on the set of modules covered by the test case and the risk correction value.

[0010] Further, the historical test records include the number of the test case, the type of the defect detected by the test case, and the module where the defect is located.

[0011] Further, calculating the defect prediction weight of the test case based on the historical test records includes: Obtain the set HF of historical test records based on the dynamic statistics count; Calculate the defect density of each module based on HF respectively; Obtain the set of hot modules based on the defect density; Calculate the defect prediction weight of each test case based on the set of hot modules respectively.

[0012] Further, the historical test records also include the running duration of the test case and the peak memory occupancy.

[0013] Further, calculating the space-time efficiency weight of the test case based on the historical test records includes: Calculate the space-time efficiency weight of the test case based on the running duration and peak memory occupancy of the test case in HF.

[0014] Further, calculating the screening value of the test case based on the risk coverage weight, defect prediction weight, and space-time efficiency weight includes: Perform weighted summation on the risk coverage weight, defect prediction weight, and space-time efficiency weight to obtain the screening value of the test case.

[0015] On the other hand, the present invention provides a test system for software test cases, including an acquisition module and a test module; The acquisition module is used to acquire a set U of test cases for testing, including: Calculating the risk value of the module based on the change record of the code of the software-based module; Calculating the risk coverage weight of the test case based on the risk value; Calculating the defect prediction weight and the space-time efficiency weight of the test case based on the historical test record; Calculating the screening value of the test case based on the risk coverage weight, the defect prediction weight, and the space-time efficiency weight; Acquiring the set U of test cases based on the screening value; The test module is used to test the module to be tested using the test cases in the set U to obtain test records.

[0016] Beneficial effects: Different from the prior art, the present invention calculates the risk value through the change records of each module of the software to be tested, then calculates the risk coverage weight based on the risk value, and then combines the risk coverage weight, the defect prediction weight, and the space-time efficiency weight to comprehensively calculate the screening value, so that the process of screening test cases not only considers the fixed attributes of the test cases, but also considers the factor of the code change frequency, thereby ensuring the priority of testing high-risk modules and obtaining more effective test results. Description of the drawings

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a schematic diagram of the test method for the software test cases of the present invention.

[0019] Figure 2 It is a schematic diagram of the method for acquiring the set U of test cases for testing of the present invention. Detailed implementation manners

[0020] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention described and illustrated herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0021] Embodiment 1: As Figure 1 shown in an embodiment, the present invention provides a test method for software test cases, including: S1. Obtain a set U of test cases for testing.

[0022] Preferably, as Figure 2 , S1 includes: S11. Calculate the risk value of the module based on the change record of the code of the module of the software.

[0023] Preferably, S11 includes: Express the total number of historical tests as K. Then, when the software is tested for the (K + 1)-th time, the calculation process of the risk value includes: Calculate the dynamic statistics count; Based on the dynamic statistics count, obtain the change count and change time density of the code of each module of the software respectively; Based on the change count and change time density, calculate the risk value of each module respectively.

[0024] Preferably, calculating the dynamic statistics count includes: Calculate the dynamic statistics count using the following formula:

[0025] and are the dynamic statistics counts used when the software is tested for the K-th and (K + 1)-th times respectively; and are the total numbers of defects obtained after the software is tested for the K-th and (K - 1)-th times respectively; max represents obtaining and the larger value between them, and N is a preset positive integer;

[0026] and They are respectively a preset first quantity threshold and a second quantity threshold.

[0027] When calculating the risk value of the calculation module in the present invention, the statistical period of the number of changes in the code of the module is not fixed. Because a fixed statistical period cannot discard test results that are too far from the current time in a timely manner when the code changes frequently, the calculated risk value cannot effectively represent the risk level of the module. In the present invention, when calculating the dynamic statistical number, it is calculated based on the change trend of the total number of defects obtained from the previous two tests. If the number of defects shows an increasing trend, the present invention will reduce the value of the dynamic statistical number, and the reduction amplitude is positively correlated with the change amplitude of the trend, which can maintain the effectiveness of the obtained number of code changes, so as to obtain a more accurate risk value, which is conducive to subsequent selection of more test cases with test significance for testing.

[0028] Preferably, the value of N is 10.

[0029] Preferably, the preset first quantity threshold and second quantity threshold are 5 and 20 respectively.

[0030] Preferably, based on the dynamic statistical number, the number of code changes and the change time density of each module of the software are respectively obtained, including: Respectively obtain the total number of code changes of each module after the most recent times of testing; Take the total number of code changes of the module within the most recent one month as the change time density of the code of the module.

[0031] For example, if the value of is 5, then the total number of code changes of the module after the most recent 5 times of testing the software is counted. The value range of the total number of code changes is [0, 5]. If the code changes after each test, the total number of code changes is 5.

[0032] Preferably, based on the number of code changes and the change time density, the risk value of each module is respectively calculated, including: For module b, represent its risk value as ,

[0033] is the total number of code changes of module b after the most recent times of testing, is the total number of times of testing the software within the most recent one month, is the total number of code changes of module b after testing the software within the most recent one month.

[0034] The risk value of the present invention has the characteristic of dynamic change. Therefore, when calculating the risk coverage weight subsequently, it is no longer calculated based on fixed data, and the calculation result can be made more effective.

[0035] S12. Calculate the risk coverage weight of the test case based on the risk value.

[0036] Preferably, S12 includes: Modify the risk value of the module based on the dependency relationship between modules to obtain a risk modification value; Obtain the set of modules covered by each test case respectively; Calculate the risk coverage weight of the test case based on the set of modules covered by the test case and the risk modification value.

[0037] Since S11 does not consider the dependency relationship between modules when calculating the risk value, in S12, the present invention modifies the risk value and calculates the risk coverage weight through the risk modification value, so that the test case can be screened more with reference to the risk degree of the module, and the selected test case is more effective and more targeted.

[0038] The inter-module dependency relationship of software refers to the mutual dependence and association between various modules or components in the software system. This dependency relationship can be direct or indirect. Specifically, when a module needs to call the function of another module, use its data or interact with it, an inter-module dependency relationship is formed. The determination of the dependency relationship can be achieved through the dependency relationship diagram between modules.

[0039] Preferably, modifying the risk value of the module based on the dependency relationship between modules to obtain a risk modification value includes: For module b, represent the total number of modules that directly depend on module b and indirectly depend on module b as Then the risk modification value corresponding to module b is:

[0040] is the risk modification value corresponding to module b, represents the total number of modules.

[0041] The present invention modifies the risk modification value of module b by the total number of other modules that depend on module b, so that when the value of is larger, the larger the value of, the more important module b is and the higher the risk. This is beneficial to making the calculated risk coverage weight more accurately represent the importance degree of the test case.

[0042] In the present invention, a module directly dependent on module b refers to a module that needs to perform operations based on the operation result of module b. A module indirectly dependent on module b refers to a module that is separated from module b by at least one other module in the dependency graph.

[0043] For example, in the dependency chain b -> c -> e, e is a module indirectly dependent on b. c operates based on the operation result of b, and e operates based on the operation result of c.

[0044] Preferably, calculating the risk coverage weight of test cases based on the set of modules covered by the test cases and the risk correction value includes: For test case t, use to represent the set of modules covered by test case t; Then the calculation formula for the risk coverage weight of test case t is:

[0045] is the risk coverage weight of test case t, Ubk is the set of test cases, is the set of modules covered by test case j, and are the risk correction values of modules i and k respectively.

[0046] The risk coverage weight of the present invention is calculated based on the risk correction value of the set of modules covered by test case t. Therefore, if the cumulative sum of the risk correction values of the test cases in is higher, it means that the risk coverage degree of test case t is higher, and it is more appropriate to perform testing based on test case t.

[0047] S13, calculating the defect prediction weight and the space-time efficiency weight of test cases based on historical test records.

[0048] Preferably, the historical test records include the number of test cases, the types of defects detected by the test cases, and the modules where the defects are located.

[0049] Specifically, the types of defects include memory leaks, null pointers, login timeouts, etc.

[0050] Preferably, calculating the defect prediction weight of test cases based on historical test records includes: Obtaining the set HF of historical test records based on dynamic statistical times; Calculating the defect density of each module based on HF; Obtaining the set of hot modules based on the defect density; Calculate the defect prediction weight for each test case based on the set of hot modules respectively.

[0051] S13 of the present invention is mainly used to obtain the set of hot modules with frequently occurring defects. After that, the defect prediction weight is calculated based on the total number of modules in the set of hot modules covered by the test case, so that the modules with potential defects can be predicted based on the historical defect data, making the screening result of the test case more effective.

[0052] Preferably, obtain the set HF of historical test records based on the dynamic statistical times, including: Take the test records obtained from the most recent

[0053] tests as the historical test records and store them in the set HF.

[0054] Since the historical test records are obtained based on the dynamic statistical times, the present invention can make the historical test records more effectively represent the latest state of the software and avoid using the test records that are too far from the current time for testing.

[0054] Preferably, calculate the defect density of each module based on HF, including: For module b, the defect density of module b is calculated by the formula:

[0055] is the total number of defects of module b existing in the test records of HF, is the total number of lines of code included in module b after the most recent tests.

[0056] The defect density is used to measure the code quality. The higher the density, the greater the defect risk hidden in each line of code.

[0057] Example: Number of lines of module code: 2000 lines; Number of defects: 15; Defect density = 15 / 2000 = 0.0075 defects / line.

[0058] Preferably, obtain the set of hot modules based on the defect density, including: Sort all the modules according to the defect density, and store the top modules with the largest defect density in the set of hot modules, where Z is a preset first ratio. Nbk represents the total number of modules.

[0059] Preferably, the value of Z is 0.2.

[0060] Preferably, the defect prediction weights of each test case are calculated based on the set of hot modules, including: For test case t, the defect prediction weight of t is calculated using the following formula :

[0061] and are respectively the total number of modules belonging to the set of hot modules in the modules covered by test cases t and j.

[0062] The calculation of the defect prediction weight in the present invention is different from the traditional coverage calculation. The traditional coverage calculation is obtained by dividing the total number of modules covered by the test cases by the total number of modules. The value calculated in this way is a fixed value and it is difficult to represent the change trend of defects in different modules. Therefore, the present invention calculates the defect prediction weight by dynamically counting the number of times, so that the test case with a higher number of defective modules identified in the next test process obtains a higher weight, further improving the effectiveness of the selected test cases.

[0063] Preferably, the historical test records also include the running duration of the test case and the peak memory occupancy.

[0064] Preferably, the space-time efficiency weights of the test cases are calculated based on the historical test records, including: The space-time efficiency weights of the test cases are calculated based on the running duration and the peak memory occupancy of the test cases in HF.

[0065] Specifically, for test case t, the set of running durations of t in the historical test records is denoted as TU; the set of peak memory occupancies of t in the historical test records is denoted as MU; Each element in TU and MU is normalized respectively to obtain sets TUA and MUA; Then the calculation formula for the space-time efficiency weight is:

[0066] is the space-time efficiency weight of test case t, and are respectively the running durations in sets TU and ; is the set of running durations of test case j in the historical test records, and are respectively the peak memory occupancies in sets MU and ; and are respectively the running duration weight and the peak memory occupancy weight.

[0067] By calculating the space-time efficiency weight, test cases with shorter running time and lower peak memory occupancy can have a greater space-time efficiency weight, so that low-consumption test cases can be preferentially executed and the utilization rate of the test environment can be improved.

[0068] The peak memory occupancy refers to the maximum value of the memory occupancy ratio during the execution of the test case, and the value range is [0, 100%].

[0069] Preferably, the running time weight and the peak memory occupancy weight are 0.7 and 0.3 respectively.

[0070] S14, calculate the screening value of the test case based on the risk coverage weight, defect prediction weight, and space-time efficiency weight.

[0071] Preferably, S14 includes: Perform weighted summation on the risk coverage weight, defect prediction weight, and space-time efficiency weight to obtain the screening value of the test case.

[0072] Specifically, use the following formula to calculate the screening value:

[0073] is the screening value of test case t, 、 and are respectively 、 and weights.

[0074] Preferably, 、 and weights are 0.5, 0.3, and 0.2 respectively.

[0075] S15, obtain the set U of test cases based on the screening value.

[0076] Specifically, store the top test cases with the largest screening value into the set U.

[0077] Nts is the total number of test cases, and Y is a preset second ratio.

[0078] Preferably, the value of Y is 0.3.

[0079] S2, use the test cases in the set U to test the module to be tested and obtain the test record.

[0080] Specifically, the module to be tested refers to the module covered by the test cases in the set U.

[0081] The modules covered by the test case can be determined by the tags set when the test case is designed. For example, in a test case of type void pointer, tags such as login module and payment module can be attached during design to indicate the coverage of the test case.

[0082] In addition, you can also use code coverage tools (such as JaCoCo, Coverage.py, Istanbul) to record the code modules that are actually covered during test execution.

[0083] Execution steps: Run the test suite and generate a coverage report.

[0084] Analysis report covers each test case in the code file, function or class.

[0085] Map code modules to functional modules (e.g. payment_api.py belongs to the “payment module”).

[0086] Furthermore, before S1, it also includes: Create a test case database.

[0087] The test case database can be established by a tester, and the tester stores the established test cases in the test case database. When the present invention is implemented, it is performed based on the test cases in the test case database.

[0088] Furthermore, after S2, it also includes: Perform statistics and analysis on the test records and generate test reports.

[0089] According to the test results: Count the pass rate of the test cases, that is, the number of passed cases divided by the total number of cases. This can be calculated in a spreadsheet using a formula such as =COUNTIF(result column, "pass") / COUNTA(result column)), or in a database using a SQL query such as SELECT COUNT(*)100.0 / (SELECT COUNT() FROM TestExecutions) AS PassRate FROM TestExecutions WHERE Result = 'pass'.

[0090] Count the number of failed and blocked test cases, and their distribution in different modules and priorities. For example, you can use a pivot table in Excel, put "Module" or "Priority" in the row label area, put "Execution Result" in the column label area, and count and summarize.

[0091] Statistics by module: Calculate metrics such as the number of test cases, passed cases, and failed cases for each module. This helps understand the quality status of each module. For example, in a database, a query statement like SELECT Module, COUNT(*)AS TotalCases, SUM(CASE WHEN Result = 'Passed' THEN 1 ELSE 0 END)AS PassedCases, SUM(CASE WHEN Result = 'Failed' THEN 1 ELSE 0 END)AS FailedCases Test FROM Executions GROUP BY Module can be executed.

[0092] Data analysis: Trend analysis: Observe the trend of test results over time. If the test is executed multiple times, a line chart can be drawn with the test execution date on the horizontal axis and metrics such as the pass rate or the number of failures on the vertical axis to analyze whether the product quality is gradually improving, deteriorating, or remaining stable. For example, in Excel, the test dates and the corresponding pass rate data can be selected and a line chart can be inserted to visually display the trend.

[0093] Analyze the trend of defect discovery to understand whether defect discovery is concentrated in a certain stage or module during the testing process, which helps optimize the testing strategy.

[0094] Defect analysis: Analyze the defects associated with the failed test cases. Statistically analyze information such as the type of defects (such as functional defects, performance defects, interface defects, etc.), severity (such as critical, major, minor), and priority.

[0095] Analyze the distribution of defects to find out which modules or functional areas are high-incidence areas of defects, which can help the development team focus on fixing them.

[0096] Embodiment 2: The present invention provides a test system for software test cases, including an acquisition module and a test module; The acquisition module is used to acquire a set U of test cases for testing, including: Calculating the risk value of the module based on the change record of the code of the software module; Calculating the risk coverage weight of the test case based on the risk value; Calculating the defect prediction weight and the space-time efficiency weight of the test case based on the historical test record; Calculate the screening value of test cases based on the risk coverage weight, defect prediction weight, and spatio-temporal efficiency weight; Obtain the set U of test cases based on the screening value; The test module is used to test the module to be tested with the test cases in the set U to obtain test records.

[0097] As mentioned above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A test method for software test cases, characterized in that, Including: S1. Obtain a set U of test cases for testing, including: S11. Calculate the risk value of a module based on the change record of the code of the module of the software; S12. Calculate the risk coverage weight of a test case based on the risk value; S13. Calculate the defect prediction weight and the space-time efficiency weight of a test case based on the historical test record; S14. Calculate the screening value of a test case based on the risk coverage weight, the defect prediction weight, and the space-time efficiency weight; S15. Obtain the set U of test cases based on the screening value; S2. Use the test cases in the set U to test the module to be tested, and obtain the test record.

2. The testing method for software test cases according to claim 1, characterized in that Before S1, it also includes: Establish a test case database.

3. The testing method for software test cases according to claim 1, characterized in that, S11 includes: Express the total number of historical tests as K. Then, when testing the software for the (K + 1)-th time, the calculation process of the risk value includes: Calculate the dynamic statistical count; Obtain the change count and the change time density of the code of each module of the software respectively based on the dynamic statistical count; Calculate the risk value of each module respectively based on the change count and the change time density.

4. The testing method of software test cases according to claim 1, wherein S12 includes: Revise the risk value of the module based on the dependency relationship between modules to obtain the risk correction value; Obtain the set of modules covered by each test case respectively; Calculate the risk coverage weight of a test case based on the set of modules covered by the test case and the risk correction value.

5. The testing method of software test cases according to claim 1, characterized in that The historical test record includes the number of the test case, the type of the defect detected by the test case, and the module where the defect is located.

6. The test method for software test cases according to claim 1, characterized in that, Calculating the defect prediction weight of a test case based on the historical test record includes: Obtain the set HF of historical test records based on the dynamic statistical count; Calculate the defect density of each module respectively based on HF; Obtain the set of hot modules based on the defect density; Calculate the defect prediction weight of each test case respectively based on the set of hot modules.

7. The testing method for software test cases according to claim 1, characterized in that, The historical test record also includes the running duration of the test case and the peak memory occupancy.

8. The test method for software test cases according to claim 1, characterized in that, Calculating the space-time efficiency weight of a test case based on the historical test record includes: Calculate the space-time efficiency weight of a test case based on the running duration and the peak memory occupancy of the test case in HF.

9. The testing method for software test cases according to claim 1, characterized in that, Calculating the screening value of a test case based on the risk coverage weight, the defect prediction weight, and the space-time efficiency weight includes: Perform weighted summation on the risk coverage weight, the defect prediction weight, and the space-time efficiency weight to obtain the screening value of the test case.

10. A test system for software test cases, characterized in that, Including an acquisition module and a test module; The acquisition module is used to obtain the set U of test cases for testing, including: Calculate the risk value of a module based on the change record of the code of the module of the software; Calculate the risk coverage weight of a test case based on the risk value; Calculate the defect prediction weight and the space-time efficiency weight of a test case based on the historical test record; Calculate the screening value of a test case based on the risk coverage weight, the defect prediction weight, and the space-time efficiency weight; Obtain the set U of test cases based on the screening value; The test module is used to test the module to be tested with the test cases in the set U and obtain the test record.

Images