Testing method and system for software test cases

Abstract

The present invention belongs to the field of software testing and discloses a testing method and system for software test cases. The method includes S1, obtaining a set U of test cases for testing, including: S11, calculating the risk value of the module based on the change record of the code of the software module; S12, calculating the risk coverage weight of the test case based on the risk value; S13, calculating the defect prediction weight and time-space efficiency weight of the test case based on historical test records; S14, calculating the screening value of the test case based on the risk coverage weight, defect prediction weight, and time-space efficiency weight; S15, obtaining the set U of test cases based on the screening value; S2, using the test cases in the set U to test the modules to be tested and obtain test records. The present invention can ensure the priority of testing high-risk modules and obtain more effective test results.

Description

Technical Field

[0001] The present invention relates to the field of software testing, and in particular to a testing method and system for software test cases. Background Art

[0002] A test case is a description of the testing tasks for a specific software product, embodying the test plan, methods, techniques, and strategies. During the software development process, test cases are typically used to conduct multiple rounds of testing. After each round of testing, the code is modified based on the test results to address any defects found, resulting in code that is suitable for the next round of testing.

[0003] To improve testing efficiency within a limited testing timeframe, existing technologies typically perform automated test case screening, enabling the most effective testing possible within the limited timeframe. During this screening process, existing technologies typically only consider fixed attributes such as test case coverage and the correlation between test cases, without dynamically factoring in code changes. This results in insufficient test effectiveness and may prevent prioritization of high-risk code files. Summary of the Invention

[0004] The purpose of the present invention is to disclose a testing method and system for software test cases and to solve the technical problems pointed out in the background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In one aspect, the present invention provides a method for testing a software test case, comprising:

[0007] S1, obtain a set U of test cases for testing, including:

[0008] S11, calculating the risk value of the module based on the change record of the module code of the software;

[0009] S12, calculating the risk coverage weight of the test case based on the risk value;

[0010] S13, calculating the defect prediction weight and time-space efficiency weight of the test case based on historical test records;

[0011] S14, calculating the screening value of the test case based on the risk coverage weight, defect prediction weight and time and space efficiency weight;

[0012] S15, obtaining a set U of test cases based on the screening value;

[0013] S2, use the test cases in set U to test the modules that need to be tested and obtain test records.

[0014] Furthermore, before S1, it also includes:

[0015] Create a test case database.

[0016] Furthermore, after S2, it also includes:

[0017] Perform statistics and analysis on test records and generate test reports.

[0018] Furthermore, S11 includes:

[0019] The total number of historical tests is expressed as K. When the software is tested for the K+1th time, the risk value calculation process includes:

[0020] Calculate dynamic statistics times;

[0021] Based on dynamic statistical times, the number of code changes and change time density of each module of the software are obtained respectively;

[0022] The risk value of each module is calculated based on the number of changes and the change time density.

[0023] Furthermore, S12 includes:

[0024] The risk value of the module is modified based on the dependency relationship between the modules to obtain a risk correction value;

[0025] Get the set of modules covered by each test case separately;

[0026] The risk coverage weight of a test case is calculated based on the set of modules covered by the test case and the risk modifier.

[0027] Furthermore, the historical test records include the number of the test case, the type of defect detected by the test case, and the module where the defect is located.

[0028] Furthermore, the defect prediction weight of the test case is calculated based on the historical test records, including:

[0029] Get the set HF of historical test records based on dynamic statistical times;

[0030] The defect density of each module is calculated based on HF;

[0031] Get hotspot module set based on defect density;

[0032] The defect prediction weight of each test case is calculated based on the hotspot module set.

[0033] Furthermore, historical test records also include the running time of the test cases and the peak memory usage.

[0034] Furthermore, the time and space efficiency weights of the test cases are calculated based on historical test records, including:

[0035] The time and space efficiency weights of the test cases are calculated based on the runtime and peak memory usage of the test cases in HF.

[0036] Furthermore, the screening value of the test case is calculated based on the risk coverage weight, defect prediction weight, and time and space efficiency weight, including:

[0037] The risk coverage weight, defect prediction weight, and time-space efficiency weight are weighted and summed to obtain the screening value of the test case.

[0038] On the other hand, the present invention provides a testing system for software test cases, including an acquisition module and a testing module;

[0039] The acquisition module is used to obtain a set U of test cases for testing, including:

[0040] Calculate the risk value of the module based on the change record of the module code;

[0041] Calculate the risk coverage weight of the test case based on the risk value;

[0042] Calculate the defect prediction weight and time-space efficiency weight of the test case based on historical test records;

[0043] Calculate the screening value of test cases based on risk coverage weight, defect prediction weight and time and space efficiency weight;

[0044] Get a set U of test cases based on the filter value;

[0045] The test module is used to test the modules that need to be tested using the test cases in the set U and obtain test records.

[0046] Beneficial effects:

[0047] Different from the existing technology, the present invention calculates the risk value by recording the changes of each module of the software to be tested, and then calculates the risk coverage weight based on the risk value. Finally, the risk coverage weight, defect prediction weight and time-space efficiency weight are combined to comprehensively calculate the screening value. In this way, in the process of screening test cases, not only the fixed attributes of the test cases are considered, but also the frequency of code changes is considered, thereby ensuring the priority of testing high-risk modules and obtaining more effective test results. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 Schematic diagram of the testing method of the software test case of the present invention.

[0050] Figure 2 A schematic diagram of a method for obtaining a set U of test cases for testing according to the present invention. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0052] Example 1:

[0053] like Figure 1 In one embodiment shown, the present invention provides a method for testing a software test case, comprising:

[0054] S1, obtain a set U of test cases for testing.

[0055] Preferably, Figure 2 , S1 includes:

[0056] S11, calculating the risk value of the module based on the change record of the code of the software module.

[0057] Preferably, S11 includes:

[0058] The total number of historical tests is expressed as K. When the software is tested for the K+1th time, the risk value calculation process includes:

[0059] Calculate dynamic statistics times;

[0060] Based on dynamic statistical times, the number of code changes and change time density of each module of the software are obtained respectively;

[0061] The risk value of each module is calculated based on the number of changes and the change time density.

[0062] Preferably, calculating the dynamic statistical number includes:

[0063] Use the following formula to calculate the dynamic statistics:

[0064]

[0065] and The dynamic statistical times used when testing the software for the Kth and K+1th times respectively; and The total number of defects obtained after testing the software for the Kth and K-1th times respectively; max means obtaining and The larger value between , N is a preset positive integer;

[0066]

[0067] and The first and second quantity thresholds are preset respectively.

[0068] When calculating the risk value of a module, the present invention does not use a fixed statistical period for the number of code changes of the module, because a fixed statistical period cannot discard test results that are too far away from the current time in time to calculate the risk value when the code changes frequently, so that the calculated risk value cannot effectively represent the risk level of the module. The present invention calculates the dynamic statistical number based on the changing trend of the total number of defects obtained from the first two tests. If the number of defects shows an increasing trend, the present invention will reduce the value of the dynamic statistical number, and the magnitude of the reduction is positively correlated with the magnitude of the trend change, which can maintain the validity of the obtained code change count, thereby obtaining a more accurate risk value, which is also conducive to the subsequent selection of test cases with more test significance for testing.

[0069] Preferably, the value of N is 10.

[0070] Preferably, the preset first quantity threshold and the second quantity threshold are 5 and 20 respectively.

[0071] Preferably, obtaining the number of code changes and the change time density of each module of the software based on the dynamic statistical number includes:

[0072] Get the latest The total number of code changes for each module after each test;

[0073] The total number of module code changes in the last month is taken as the module code change time density.

[0074] For example, if If the value is 5, then the total number of times the module code has been changed after the last five tests of the software is counted. The value range of the total number of changes is [0,5]. If the code is changed after each test, the total number of changes is 5.

[0075] Preferably, the risk value of each module is calculated based on the number of changes and the change time density, including:

[0076] For module b, its risk value is expressed as ,

[0077]

[0078] For the recent The total number of code changes for module b after each test, The total number of software tests performed in the last month. The total number of code changes for module b after the software was tested in the last month.

[0079] The risk value of the present invention has the characteristic of dynamic change. Therefore, when subsequently calculating the risk coverage weight, the calculation is no longer based on fixed data, which can make the calculation result more effective.

[0080] S12, calculate the risk coverage weight of the test case based on the risk value.

[0081] Preferably, S12 includes:

[0082] The risk value of the module is modified based on the dependency relationship between the modules to obtain a risk correction value;

[0083] Get the set of modules covered by each test case separately;

[0084] The risk coverage weight of a test case is calculated based on the set of modules covered by the test case and the risk modifier.

[0085] Since S11 does not take into account the dependencies between modules when calculating the risk value, the present invention corrects the risk value in S12 and calculates the risk coverage weight through the risk correction value, so that the test cases can be screened with more reference to the risk level of the modules, making the selected test cases more effective and more targeted.

[0086] Inter-module dependencies in software refer to the interdependencies and relationships between modules or components within a software system. These dependencies can be direct or indirect. Specifically, an inter-module dependency occurs when one module calls functionality, uses data, or interacts with another module. Dependency graphs can be used to identify these dependencies.

[0087] Preferably, the risk value of the module is modified based on the dependency relationship between the modules to obtain the risk correction value, including:

[0088] For module b, the total number of modules that directly depend on module b and indirectly depend on module b is expressed as , then the risk correction value corresponding to module b is:

[0089]

[0090] is the risk correction value corresponding to module b, Indicates the total number of modules.

[0091] The present invention modifies the risk correction value of module b by adjusting the total number of other modules that rely on module b, so that The larger the value of The larger the value is, the more important module b is and the higher the risk is. This helps the calculated risk coverage weight to more accurately represent the importance of the test case.

[0092] In the present invention, a module that directly depends on module b refers to a module that needs to perform operations based on the running results of module b;

[0093] A module that indirectly depends on module b refers to a module that is separated from module b by at least one other module in the dependency graph.

[0094] For example, in the dependency chain b->c->e, e is a module that indirectly depends on b. c depends on the result of b to run, and e depends on the result of c to run.

[0095] Preferably, calculating the risk coverage weight of a test case based on the set of modules covered by the test case and the risk correction value includes:

[0096] For test case t, use Represents the set of modules covered by test case t;

[0097] The calculation formula for the risk coverage weight of test case t is:

[0098]

[0099] is the risk coverage weight of test case t, Ubk is the set of test cases, is the set of modules covered by test case j, and are the risk correction values ​​for modules i and k, respectively.

[0100] The risk coverage weight of the present invention is calculated based on the risk correction value of the set of modules covered by test case t. Therefore, if The higher the cumulative sum of the risk correction values ​​of the test cases in , the higher the risk coverage of test case t is, and the more testing should be performed based on test case t.

[0101] S13, calculating the defect prediction weight and time-space efficiency weight of the test case based on historical test records.

[0102] Preferably, the historical test record includes the number of the test case, the type of defect detected by the test case, and the module where the defect is located.

[0103] Specifically, the types of defects include memory leaks, null pointers, login timeouts, etc.

[0104] Preferably, calculating the defect prediction weight of a test case based on historical test records includes:

[0105] Get the set HF of historical test records based on dynamic statistical times;

[0106] The defect density of each module is calculated based on HF;

[0107] Get hotspot module set based on defect density;

[0108] The defect prediction weight of each test case is calculated based on the hotspot module set.

[0109] S13 of the present invention is mainly used to obtain a set of hotspot modules where defects occur frequently. Then, the defect prediction weight is calculated based on the total number of modules in the hotspot module set covered by the test case, so that modules with potential defects can be predicted based on historical defect data, thereby making the screening results of the test case more effective.

[0110] Preferably, obtaining a set HF of historical test records based on dynamic statistical times includes:

[0111] will be recently The test records obtained from the tests are stored in the set HF as historical test records.

[0112] Since the historical test records are obtained based on dynamic statistical times, the present invention can make the historical test records more effectively represent the latest status of the software, and avoid using test records that are too far away from the current time for testing.

[0113] Preferably, the defect density of each module is calculated based on HF, including:

[0114] For module b, the defect density of module b The calculation formula is:

[0115]

[0116] is the total number of defects in module b in the test records of HF, For the recent The total number of lines of code contained in module b after the tests.

[0117] Defect density is used to measure code quality. A higher density indicates a greater risk of defects per line of code.

[0118] Example:

[0119] Module code lines: 2000 lines;

[0120] Number of defects: 15;

[0121] Defect density = 15 / 2000 = 0.0075 defects / row.

[0122] Preferably, obtaining a hotspot module set based on defect density includes:

[0123] Sort all modules by defect density and put the modules with the largest defect density first. Modules are stored in the hotspot module set, Z is the preset first ratio, and Nbk represents the total number of modules.

[0124] Preferably, the value of Z is 0.2.

[0125] Preferably, the defect prediction weight of each test case is calculated based on the hotspot module set, including:

[0126] For test case t, use the following formula to calculate the defect prediction weight of t :

[0127]

[0128] and are the total number of modules belonging to the hotspot module set among the modules covered by test cases t and j, respectively.

[0129] The calculation of defect prediction weights in the present invention is different from the traditional coverage calculation. The traditional coverage calculation is based on the total number of modules covered by the test case divided by the total number of modules. The calculated value is a fixed value, which is difficult to represent the changing trend of defects in different modules. Therefore, the present invention calculates the defect prediction weights through dynamic statistical times, so that the test cases with a higher number of defective modules identified in the next test process will obtain higher weights, further improving the effectiveness of the screened test cases.

[0130] Preferably, the historical test records also include the running time of the test case and the peak memory usage.

[0131] Preferably, calculating the time and space efficiency weights of test cases based on historical test records includes:

[0132] The time and space efficiency weights of the test cases are calculated based on the runtime and peak memory usage of the test cases in HF.

[0133] Specifically, for test case t, the set of its running time in historical test records is denoted as TU; the set of its memory usage peak values ​​in historical test records is denoted as MU;

[0134] Normalize each element in TU and MU respectively to obtain the sets TUA and MUA;

[0135] The calculation formula for the space-time efficiency weight is:

[0136]

[0137] is the time and space efficiency weight of test case t, and are the sets TU and The runtime in is the set of running time of test case j in historical test records, and are the sets MU and Peak memory usage in ; and They are respectively the runtime weight and the peak memory usage weight.

[0138] By calculating the time and space efficiency weights, test cases with shorter runtimes and lower peak memory usage can be given larger time and space efficiency weights, so that low-consumption test cases can be executed first and the test environment utilization rate can be improved.

[0139] The peak memory usage refers to the maximum memory usage during the test case execution, and its value range is [0,100%].

[0140] Preferably, the runtime weight and the peak memory usage weight are 0.7 and 0.3 respectively.

[0141] S14, calculates the screening value of the test case based on the risk coverage weight, defect prediction weight and time and space efficiency weight.

[0142] Preferably, S14 includes:

[0143] The risk coverage weight, defect prediction weight, and time-space efficiency weight are weighted and summed to obtain the screening value of the test case.

[0144] Specifically, the following formula is used to calculate the screening value:

[0145]

[0146] is the screening value of test case t, 、 and They are 、 and The weight of .

[0147] Preferably, 、 and The weights are 0.5, 0.3 and 0.2 respectively.

[0148] S15, obtaining a set U of test cases based on the filter value.

[0149] Specifically, the largest value will be filtered The test cases are stored in set U.

[0150] Nts is the total number of test cases, and Y is the preset second ratio.

[0151] Preferably, the value of Y is 0.3.

[0152] S2, use the test cases in set U to test the modules that need to be tested and obtain test records.

[0153] Specifically, the modules that need to be tested refer to the modules covered by the test cases in the set U.

[0154] The modules covered by a test case can be determined by the tags set during test case design. For example, in a void pointer test case, tags such as login module and payment module can be attached during design to indicate the test case's coverage.

[0155] In addition, you can also use code coverage tools (such as JaCoCo, Coverage.py, Istanbul) to record the code modules that are actually covered during test execution.

[0156] Execution steps:

[0157] Run the test suite and generate a coverage report.

[0158] The code files, functions, or classes covered by each test case in the analysis report.

[0159] Map code modules to functional modules (e.g. payment_api.py belongs to the “payment module”).

[0160] Furthermore, before S1, it also includes:

[0161] Create a test case database.

[0162] The test case database can be established by a tester, and the tester stores the established test cases in the test case database. When the present invention is implemented, it is performed based on the test cases in the test case database.

[0163] Furthermore, after S2, it also includes:

[0164] Perform statistics and analysis on test records and generate test reports.

[0165] Statistics by test results:

[0166] Calculate the pass rate of test cases by dividing the number of passed cases by the total number of cases. This can be calculated in a spreadsheet using a formula such as =COUNTIF(result column, "pass") / COUNTA(result column) or in a database using a SQL query such as SELECT COUNT(*)100.0 / (SELECT COUNT() FROM TestExecutions) AS PassRate FROM TestExecutions WHERE Result = 'pass'.

[0167] Count the number of failed and blocked test cases, and their distribution across modules and priorities. For example, in Excel, you can use a pivot table with "Module" or "Priority" as the row label and "Execution Result" as the column label, and perform a count summary.

[0168] Statistics by module:

[0169] Calculate metrics such as the number of test cases, passing cases, and failing cases for each module. This helps understand the quality status of each module. For example, in the database, you can execute a query like SELECT Module, COUNT (*) AS TotalCases, SUM (CASE WHEN Result = 'Passed' THEN 1 ELSE 0 END) AS PassedCases, SUM (CASE WHEN Result = 'Failed' THEN 1 ELSE 0 END) AS FailedCases Test FROM Executions GROUP BY Module.

[0170] Data Analysis:

[0171] Trend analysis:

[0172] Observe the trend of test results over time. If you run tests multiple times, you can create a line chart with the test execution date on the horizontal axis and metrics such as pass rate or number of failures on the vertical axis. This allows you to analyze whether product quality is gradually improving, deteriorating, or remaining stable. For example, in Excel, you can select the test date and the corresponding pass rate data and insert a line chart to visually display the trend.

[0173] Analyze the defect discovery trend and understand whether the defect discovery is concentrated in a certain stage or module during the testing process, which helps optimize the testing strategy.

[0174] Defect analysis:

[0175] Analyze the defects associated with failed test cases. Collect information such as defect type (e.g., functional defect, performance defect, interface defect), severity (e.g., severe, general, minor), and priority.

[0176] Analyzing the distribution of defects and finding out which modules or functional areas are high-incidence areas of defects can help the development team focus on fixing them.

[0177] Example 2:

[0178] The present invention provides a testing system for software test cases, comprising an acquisition module and a testing module;

[0179] The acquisition module is used to obtain a set U of test cases for testing, including:

[0180] Calculate the risk value of the module based on the change record of the module code;

[0181] Calculate the risk coverage weight of the test case based on the risk value;

[0182] Calculate the defect prediction weight and time-space efficiency weight of the test case based on historical test records;

[0183] Calculate the screening value of test cases based on risk coverage weight, defect prediction weight and time and space efficiency weight;

[0184] Get a set U of test cases based on the filter value;

[0185] The test module is used to test the modules that need to be tested using the test cases in the set U and obtain test records.

[0186] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A testing method for a software test case, characterized in that: include: S1, obtain a set U of test cases for testing, including: S11, calculating the risk value of the module based on the change record of the module code, including: The total number of historical tests is expressed as K. When the software is tested for the K+1th time, the risk value calculation process includes: Calculate dynamic statistics, including: Use the following formula to calculate the dynamic statistics: ； and The dynamic statistical times used when testing the software for the Kth and K+1th times respectively; and The total number of defects obtained after testing the software for the Kth and K-1th times respectively; max means obtaining and The larger value between , N is a preset positive integer; ； and are respectively a preset first quantity threshold and a second quantity threshold; Based on dynamic statistics, the number of code changes and change time density of each module of the software are obtained, including: Get the latest The total number of code changes for each module after each test; The total number of module code changes in the last month is used as the module code change time density; The risk value of each module is calculated based on the number of changes and the change time density, including: For module b, its risk value is expressed as , ； For the recent The total number of code changes for module b after each test, is the total number of software tests performed in the last month. The total number of code changes in module b after the software was tested in the last month; S12, calculating the risk coverage weight of the test case based on the risk value; S13, calculating the defect prediction weight and time-space efficiency weight of the test case based on historical test records; S14, calculating the screening value of the test case based on the risk coverage weight, defect prediction weight and time and space efficiency weight; S15, obtaining a set U of test cases based on the screening value; S2, use the test cases in set U to test the modules that need to be tested and obtain test records.

2. The software test case testing method according to claim 1, wherein: Before S1, it also included: Create a test case database.

3. The software test case testing method according to claim 1, wherein: S12 includes: Based on the dependency relationship between modules, the risk value of the module is modified to obtain a risk correction value; Get the set of modules covered by each test case separately; The risk coverage weight of a test case is calculated based on the set of modules covered by the test case and the risk modifier.

4. The software test case testing method according to claim 1, wherein: Historical test records include the test case number, the type of defect detected by the test case, and the module where the defect is located.

5. The software test case testing method according to claim 1, wherein: Calculate the defect prediction weight of the test case based on historical test records, including: Get the set HF of historical test records based on dynamic statistical times; The defect density of each module is calculated based on HF; Get hotspot module set based on defect density; The defect prediction weight of each test case is calculated based on the hotspot module set.

6. The software test case testing method according to claim 1, wherein: Historical test records also include the running time of the test case and the peak memory usage.

7. The software test case testing method according to claim 1, wherein: Calculate the time and space efficiency weights of test cases based on historical test records, including: The time and space efficiency weights of the test cases are calculated based on the runtime and peak memory usage of the test cases in HF.

8. The software test case testing method according to claim 1, wherein: Calculate the screening value of test cases based on risk coverage weight, defect prediction weight, and time and space efficiency weight, including: The risk coverage weight, defect prediction weight, and time-space efficiency weight are weighted and summed to obtain the screening value of the test case.

9. A testing system for software test cases, characterized in that: Includes acquisition module and test module; The acquisition module is used to obtain a set U of test cases for testing, including: The risk value of a software module is calculated based on the change record of the module's code, including: The total number of historical tests is expressed as K. When the software is tested for the K+1th time, the risk value calculation process includes: Calculate dynamic statistics, including: Use the following formula to calculate the dynamic statistics: ； and The dynamic statistical times used when testing the software for the Kth and K+1th times respectively; and The total number of defects obtained after testing the software for the Kth and K-1th times respectively; max means obtaining and The larger value between , N is a preset positive integer; ； and are respectively a preset first quantity threshold and a second quantity threshold; Based on dynamic statistics, the number of code changes and change time density of each module of the software are obtained, including: Get the latest The total number of code changes for each module after each test; The total number of module code changes in the last month is used as the module code change time density; The risk value of each module is calculated based on the number of changes and the change time density, including: For module b, its risk value is expressed as , ； For the recent The total number of code changes for module b after each test, The total number of software tests performed in the last month. The total number of code changes in module b after the software was tested in the last month; Calculate the risk coverage weight of the test case based on the risk value; Calculate the defect prediction weight and time-space efficiency weight of the test case based on historical test records; Calculate the screening value of test cases based on risk coverage weight, defect prediction weight and time and space efficiency weight; Get a set U of test cases based on the filter value; The test module is used to test the modules that need to be tested using the test cases in the set U and obtain test records.

Images