Software automation accurate test analysis platform

Abstract

The invention discloses a software automation accurate test analysis platform, and relates to the technical field of software testing, the software automation accurate test analysis platform comprises a case management module, the case management module is used for importing and adjusting test cases according to different test project requirements; the software test module is used for testing the software in sequence based on the imported test cases; a coverage rate collection module; the monitoring module is used for monitoring whether each row of codes of the software are executed or not when the software is tested by each test case, and obtaining a test case code row execution set; and executing set integration calculation based on the test case code line to obtain software test code line coverage rate data, and evaluating the current software test based on the software test code line coverage rate data. According to the method, the defect code lines are accurately positioned, a user is helped to quickly position key program codes, the user can conveniently and accurately analyze and modify the code quality, and developers and testers are helped to optimize the code quality and improve the test efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a software automated precision testing and analysis platform. Background Art

[0002] Software testing is the process of executing a program to find software errors and defects (collectively referred to as bugs). It is an important part of software development and is used to confirm whether the software's functions or performance meet the requirements before development. Before the software is put into operation, software testing can be used to conduct a final review of the software's requirements analysis, design specifications, and coding to ensure the quality of the software.

[0003] For example, the patent with application publication number CN108628729A, application publication date 2018.10.09, and name “A software testing method and software testing client” specifically includes negotiating with the server to generate a software testing configuration file; calling the interface function in the software testing configuration file to generate a software testing program; requesting software testing parameters from the server according to the software testing configuration file; and executing the software testing program to test the software according to the software testing parameters obtained from the server.

[0004] Similar to the above application, in the existing software testing field, testing is performed by automated testing and manual testing. In both aspects of testing, due to the drawbacks of each method, testing work has become a burden. In the current manual testing, it is too much affected by subjective human factors: each time a version is released, the tester relies entirely on personal experience to determine the scope of impact of the change on the system. Usually, either the test scope is set too small, resulting in missed tests, or the test scope is too large, the cost is too high, resulting in delayed delivery;

[0005] In automated testing, as the software is continuously iterated, the number of automated use cases will accumulate more and more, from hundreds to tens of thousands. At this time, due to the need to pay attention to test stability, running time, and a large number of repeated test scenarios and codes, the overall test return rate does not increase with the increase in the number of use cases. Instead, maintenance and troubleshooting become a burden on testers' daily work. They are overwhelmed and have no energy to invest their time in more useful exploratory testing and analysis work, which leads to frequent bugs. The entire team loses trust in automation until it is abandoned. These are also a major bottleneck that prevents most companies from implementing agile testing. Summary of the invention

[0006] The purpose of the present invention is to provide a software automated precision testing and analysis platform to solve the above-mentioned deficiencies in the prior art.

[0007] In order to achieve the above-mentioned purpose, the present invention provides the following technical solution: a software automated precision testing and analysis platform, which is used to ensure the functional correctness of the software, comprising:

[0008] Use case management module: Use the use case management module to import and adjust test cases according to different test project requirements;

[0009] A software testing module, wherein the software testing module sequentially tests the software based on the imported test cases;

[0010] Coverage collection module; it is used to monitor whether each line of software code is executed when each test case tests the software, and obtain the test case code line execution set;

[0011] Based on the test case code lines, a collection integration calculation is performed to obtain the software test code line coverage data, and the current software test is evaluated based on the software test code line coverage data;

[0012] The test analysis module is used to collect the running data of each test case when testing the software and perform correlation and integration analysis to obtain the test data set, accurately locate the defective code line based on the test data set, and provide feedback display;

[0013] The regression test module is used to identify the modified software code to obtain the difference features, associate the test case code line execution set based on the difference features, and perform traceability association to determine the scope of the regression test case.

[0014] As a further description of the above technical solution: the use case management module includes a use case management unit and a use case library unit; the use case library unit stores all test cases, including historical cases, and users can search, filter and reuse test cases in the library to improve testing efficiency; the use case management unit is used for users to create, edit, delete and manage test cases, and import and adjust test cases according to different test projects.

[0015] As a further description of the above technical solution: the step of obtaining the test case code line execution set specifically includes the following steps:

[0016] The code of the software to be tested is numbered based on the line number sequence to obtain code line number data, and marked as H1, H2, H3...Hn respectively; wherein Hn represents the nth line of program code of the software to be tested;

[0017] Each imported test case is marked as C1, C2, C3...Cm; Cm represents the mth test case used to test the software;

[0018] The execution status of the test software code is collected when each test case tests the test software in turn, and the test case code line number running set Uc1, Uc2, Uc3..Ucm is obtained, where Ucm represents the line number set of the software code running when the mth test case tests the software.

[0019] As a further description of the above technical solution: obtaining software test code line coverage data and evaluating the current software test based on the software test code line coverage specifically includes the following steps:

[0020] Retrieve the test case code line execution set to obtain the number of executed code lines and the total number of software code lines;

[0021] The number of executed code lines ÷ the total number of code lines × 100% to obtain the software test code line coverage;

[0022] Set a comparison threshold, compare the code line coverage with the comparison threshold, issue an alarm when the code line coverage is less than the comparison threshold, and re-import the test cases to test the software in sequence.

[0023] As a further description of the above technical solution: collecting the running data of each test case during the software test and performing correlation integration analysis to obtain the test data set specifically includes the following steps:

[0024] Collect the running data of each test case when testing the software, where the running data includes all input information and response information during the test;

[0025] Map and combine input information and response information and associate test cases to form a complete test data pair, and compare it with the preset response information;

[0026] When there is a deviation between the preset response information and the actual measured response information, the current test data pair is marked as an abnormal data pair;

[0027] When the preset response information is consistent with the actual measured response information, the current test data pair is marked as a normal data pair;

[0028] The test data set includes a defect test data set and a normal test data set. All abnormal data pairs are integrated to obtain the defect test data set, and all normal data pairs are integrated to obtain the normal test data set.

[0029] As a further description of the above technical solution: accurately locating defective code lines based on the test data set and providing feedback display specifically includes the following steps:

[0030] An abnormal data pair is randomly retrieved from the defect test data set, and all running code line number information contained in the corresponding test case code line number running set is retrieved based on the test case associated with the abnormal data pair, and marked as the defect code line number set;

[0031] Traverse all the test case code execution sets in the normal test data set, identify the running code line numbers contained in the test case code execution set, select a test case code execution set whose running code line numbers overlap the most with the defect code line number set, obtain all the running code line number information contained therein, and mark it as the comparison code line number set;

[0032] The different code line numbers between the defective code line number set and the comparison code line number set are marked as defective code line numbers, and feedback is provided.

[0033] As a further description of the above technical solution: it also includes a coverage optimization module, which is used to call the test case code line execution set, and perform correlation integration analysis and optimization to select test cases based on the currently imported test cases.

[0034] As a further description of the above technical solution: performing correlation integration analysis and optimizing the selection of test cases based on the currently imported test cases specifically includes the following steps:

[0035] S1: Retrieve the test case code line execution set to obtain the executed code line number data, and integrate to obtain the code line number set

[0036] S2: traverse the test case code execution set to identify the running code line number contained in the test case code execution set, select a test case code execution set with the largest number of running code lines and mark it as the first test case;

[0037] S3: Based on the line numbers included in the code execution set in the first test case, mark the covered line number data in the code line number set;

[0038] S4: Then, from the remaining test case code execution sets, select the next software trial case code execution set that can cover the most line number data in the uncovered code line number set and mark it as the second test case;

[0039] S5: Repeat step S4 until all line number data in the code line number set are covered, that is, all code lines in the code line number set are triggered, and all selected test cases are integrated to obtain optimized test cases.

[0040] As a further description of the above technical solution: the modified software code is identified to obtain difference features, the test case code line execution set is associated based on the difference features, and the scope of the regression test case is determined by tracing and associating:

[0041] Compare the modified software code with the software code before modification to determine the modified code line number data;

[0042] Based on the modified code line number data, the test case code line execution set is associated with the modified code line number to determine which test cases are associated with, and the test case scope of the regression test is obtained through integration.

[0043] As a further description of the above technical solution: based on the modified code line number data associated with the test case code line execution set, determine which test cases are associated with the modified code line number, and integrate to obtain the use case range of the regression test as follows: obtain the code line number set where the modifications have occurred, traverse the test case code execution set to identify the test case with the most modified code line numbers in the test case code execution set, and mark it as the first regression test case; then select the next test case code execution set that can cover the most uncovered line number data in the modified code line number set from the remaining test case code execution sets and mark it as the second regression test case, repeat the above operation until all line number data in the modified code line number set are covered, and integrate all the selected regression test cases to obtain the use case range of the regression test.

[0044] In the above technical scheme, the present invention provides a software automated precision testing and analysis platform, which first monitors whether each line of software code is executed when each test case tests the software, obtains the test case code line execution set, and then integrates and calculates to obtain the software test code line coverage data, and evaluates the current software test based on the software test code line coverage data to determine whether the program code coverage of this test meets the test requirements, thereby helping the tester to more accurately evaluate the test risk and form a test accuracy standard.

[0045] Secondly, by collecting the running data of each test case during software testing and performing correlation and integration analysis to obtain the defective test data set and the normal test data set, the abnormal data in the defective test data set is correlated with the associated defective code line number set and the normal test data set for correlation and comparison analysis, so as to accurately locate the defective code line, help users quickly locate key program code, facilitate users to accurately analyze and modify the code quality, and help developers and testers optimize code quality and improve testing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0047] Figure 1 A schematic diagram of a software automation and precision testing and analysis platform module provided by an embodiment of the present invention;

[0048] Figure 2 A schematic diagram of the workflow of a software automated precision testing and analysis platform provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0050] Embodiment 1:

[0051] See also Figure 1-2 The embodiment of the present invention provides a technical solution: a software automated precision testing and analysis platform, which is used to ensure the functional correctness of the software, including:

[0052] The case management module is used to import and adjust test cases according to different test project requirements; the specific case management module includes a case management unit and a case library unit; the case library unit stores all test cases, and users can search, filter and reuse all test cases in the library to improve the efficiency of testing; the case management unit is used for users to create, edit, delete and manage test cases, and import and adjust test cases according to different test projects;

[0053] A software testing module, wherein the software testing module tests the software in sequence based on the imported test cases; wherein the software testing module first verifies the format and integrity of the imported test cases to ensure that the test cases can be executed correctly, and then configures the test environment according to the test requirements. During the execution process, the software testing module simulates user operations, inputs test data, captures the software's responses, and records log information of the test process;

[0054] Coverage collection module; it is used to monitor whether each line of software code is executed when each test case tests the software, and obtain the test case code line execution set;

[0055] Based on the test case code lines, a collection integration calculation is performed to obtain the software test code line coverage data, and the current software test is evaluated based on the software test code line coverage data;

[0056] The test analysis module is used to collect the running data of each test case when testing the software and perform correlation and integration analysis to obtain the test data set, accurately locate the defective code line based on the test data set, and provide feedback display;

[0057] The regression test module is used to identify the modified software code to obtain the difference features, associate the test case code line execution set based on the difference features, and perform traceability association to determine the scope of the regression test case.

[0058] Specifically, by monitoring whether each line of software code is executed when each test case tests the software, the test case code line execution set is obtained, and then the software test code line coverage data is integrated and calculated. Based on the software test code line coverage data, the current software test is evaluated to determine whether the program code coverage of this test meets the test requirements, thereby helping the tester to more accurately assess the test risk and form a test accuracy standard.

[0059] In another embodiment provided by the present invention, obtaining the test case code line execution set specifically includes the following steps:

[0060] The code of the software to be tested is numbered based on the line number sequence to obtain code line number data, and marked as H1, H2, H3...Hn respectively; wherein Hn represents the nth line of program code of the software to be tested;

[0061] Each imported test case is marked as C1, C2, C3...Cm; Cm represents the mth test case used to test the software;

[0062] The execution status of the test software code is collected when each test case tests the test software in turn, and the test case code line number running sets Uc1, Uc2, Uc3..Ucm are obtained, where Ucm represents the line number set of the software code running when the mth test case tests the software, that is, the test case code line number running set contains multiple line number data, and the same line number data exists in each test case code line number running set.

[0063] In another embodiment provided by the present invention, obtaining software test code line coverage data and evaluating the current software test based on the software test code line coverage specifically includes the following steps:

[0064] Retrieve the test case code line execution set to obtain the number of executed code lines and the total number of software code lines. When performing statistical analysis on the number of executed code lines, repeated software code line number data is only counted once.

[0065] The number of executed code lines ÷ the total number of code lines × 100% to obtain the software test code line coverage;

[0066] Set a comparison threshold, where the comparison threshold is the code line coverage data threshold preset to meet software testing requirements. Compare the code line coverage with the comparison threshold. When the code line coverage is less than the comparison threshold, an alarm is issued, indicating that there are defects in the software testing integrity and the test cases need to be re-determined to test the software.

[0067] In another embodiment provided by the present invention, collecting the running data of each test case when testing the software and performing correlation integration analysis to obtain the test data set specifically includes the following steps:

[0068] Collect the running data of each test case when testing the software, where the running data includes all input information and response information during the test;

[0069] Map and combine input information and response information and associate test cases to form a complete test data pair, and compare it with the preset response information;

[0070] When there is a deviation between the preset response information and the actual measured response information, the current test data pair is marked as an abnormal data pair;

[0071] When the preset response information is consistent with the actual measured response information, the current test data pair is marked as a normal data pair;

[0072] The test data set includes a defect test data set and a normal test data set. All abnormal data pairs are integrated to obtain the defect test data set, and all normal data pairs are integrated to obtain the normal test data set.

[0073] In another embodiment provided by the present invention, accurately locating defective code lines based on a test data set and providing feedback display specifically includes the following steps:

[0074] An abnormal data pair is randomly retrieved from the defect test data set, and all running code line number information contained in the corresponding test case code line number running set is retrieved based on the test case associated with the abnormal data pair, and marked as the defect code line number set;

[0075] Traverse all the test case code execution sets in the normal test data set, identify the running code line numbers contained in the test case code execution set, select a test case code execution set whose running code line numbers overlap the most with the defect code line number set, obtain all the running code line number information contained therein, and mark it as the comparison code line number set;

[0076] The different code line numbers between the defective code line number set and the comparison code line number set are marked as defective code line numbers, and feedback is provided.

[0077] The present embodiment provides a software automated precision testing and analysis platform, which obtains defective test data sets and normal test data sets by collecting the running data of each test case during software testing and performing correlation and integration analysis, and correlates and compares the defective code line number set associated with the abnormal data in the defective test data set with the normal test data set, so as to accurately locate the defective code lines, help users quickly locate key program codes, facilitate users to accurately analyze and modify code quality, and help developers and testers optimize code quality and improve testing efficiency.

[0078] Embodiment 2:

[0079] It also includes a coverage optimization module, which is used to call a test case code line execution set and perform correlation integration analysis and optimization to select test cases based on the currently imported test cases.

[0080] In another embodiment provided by the present invention, performing correlation integration analysis and optimizing the selection of test cases based on the currently imported test cases specifically includes the following steps:

[0081] S1: Retrieve the test case code line execution set to obtain the executed code line number data, and integrate to obtain the code line number set

[0082] S2: traverse the test case code execution set to identify the running code line number contained in the test case code execution set, select a test case code execution set with the largest number of running code lines and mark it as the first test case;

[0083] S3: Based on the line numbers included in the code execution set in the first test case, mark the covered line number data in the code line number set;

[0084] S4: Then, from the remaining test case code execution sets, select the next software trial case code execution set that can cover the most line number data in the uncovered code line number set and mark it as the second test case;

[0085] S5: Repeat step S4 until all line number data in the code line number set are covered, that is, all code lines in the code line number set are triggered, and all selected test cases are integrated to obtain optimized test cases.

[0086] It should be noted that when ensuring the functional correctness test of the software, when all functional requirements have been verified by functional test cases, if other test cases have triggered all code lines, there is no need to add additional test cases. The present embodiment provides a software automated precision testing and analysis platform, which selects a test case code execution set with the largest number of running code lines by calling the test case code line execution set and marks it as the first test case, and then selects the next software trial case code execution set that can cover the most line number data in the uncovered code line number set from the remaining test case code execution set and marks it as the second test case; repeats until all line number data in the code line number set are covered, thereby achieving the selection of the least number of test cases to complete the test of the software, realizing the automation and standardization of the testing process, significantly improving the efficiency of software testing, and avoiding testers relying solely on personal experience to determine the scope of impact of changes on the system. Usually, either the test scope is set too small, resulting in missed tests, or the test scope is too large, and the cost is too high.

[0087] Embodiment three:

[0088] Identify the modified software code to obtain the difference features, associate the test case code line execution set based on the difference features, and perform source tracing and association to determine the scope of the regression test case:

[0089] Compare the modified software code with the software code before modification to determine the modified code line number data;

[0090] Based on the modified code line number data, the test case code line execution set is associated with the modified code line number to determine which test cases are associated with, and the test case scope of the regression test is obtained through integration.

[0091] In another embodiment provided by the present invention, based on the modified code line number data associated with the test case code line execution set, it is determined which test cases are associated with the modified code line number, and the test case range of the regression test is integrated as follows: the code line number set where the modification occurs is obtained, the test case code execution set is traversed to identify the test case with the most modified code line numbers in the test case code execution set, and it is marked as the first regression test case; then, the next test case code execution set that can cover the most uncovered line number data in the modified code line number set is selected from the remaining test case code execution sets and marked as the second regression test case, and the above operation is repeated until all line number data in the modified code line number set is covered, and all the selected regression test cases are integrated to obtain the test case range of the regression test.

[0092] This embodiment provides a software automated precision testing and analysis platform, which determines which test cases are associated with the modified code line numbers by associating the test case code line execution set based on the modified code line number data, thereby establishing an association between the test case and the code level. By establishing a binding relationship from code to use case, we can perform two-way traceability. Once the developer modifies the code, we can trace back to the corresponding test case through the associated link, and integrate the related test cases to obtain the use case scope of the regression test, thereby helping testers to effectively shorten the test scope, accurately find the functions and use cases that need to be tested, and accurately define the use case scope that needs regression testing, effectively narrow the scope of regression testing, and improve testing efficiency.

[0093] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A software automated precision testing and analysis platform, which is used to ensure the functional correctness of the software, characterized in that: include: Use case management module: Use the use case management module to import and adjust test cases according to different test project requirements; A software testing module, wherein the software testing module sequentially tests the software based on the imported test cases; Coverage collection module; It is used to monitor whether each line of software code is executed when each test case tests the software, and obtain the test case code line execution set; Based on the test case code lines, a collection integration calculation is performed to obtain the software test code line coverage data, and the current software test is evaluated based on the software test code line coverage data; The test analysis module is used to collect the running data of each test case when testing the software and perform correlation and integration analysis to obtain the test data set, accurately locate the defective code line based on the test data set, and provide feedback display; The regression test module is used to identify the modified software code to obtain the difference features, associate the test case code line execution set based on the difference features, and perform traceability association to determine the scope of the regression test case.

2. A software automated precision testing and analysis platform according to claim 1, characterized in that: The use case management module includes a use case management unit and a use case library unit; the use case library unit stores all test cases; the use case management unit is used for users to create, edit, delete and manage test cases, and import and adjust test cases according to different test projects.

3. A software automated precision testing and analysis platform according to claim 1, characterized in that: The method of obtaining the test case code line execution set specifically includes the following steps: The code of the software to be tested is numbered based on the line number sequence to obtain code line number data, and marked as H1, H2, H3...Hn respectively; wherein Hn represents the nth line of program code of the software to be tested; Each imported test case is marked as C1, C2, C3...Cm; Cm represents the mth test case used to test the software; The execution status of the test software code is collected when each test case tests the test software in turn, and the test case code line number running set Uc1, Uc2, Uc3..Ucm is obtained, where Ucm represents the line number set of the software code running when the mth test case tests the software.

4. A software automated precision testing and analysis platform according to claim 3, characterized in that: Obtaining software test code line coverage data and evaluating current software testing based on the software test code line coverage specifically includes the following steps: Retrieve the test case code line execution set to obtain the number of executed code lines and the total number of software code lines; The number of executed code lines ÷ the total number of code lines × 100% to obtain the software test code line coverage; Set a comparison threshold, compare the code line coverage with the comparison threshold, issue an alarm when the code line coverage is less than the comparison threshold, and re-import the test cases to test the software in sequence.

5. A software automated precision testing and analysis platform according to claim 3, characterized in that: Collecting the running data of each test case during software testing and performing correlation and integration analysis to obtain the test data set specifically includes the following steps: Collect the running data of each test case when testing the software, where the running data includes all input information and response information during the test; Map and combine input information and response information and associate test cases to form a complete test data pair, and compare it with the preset response information; When there is a deviation between the preset response information and the actual measured response information, the current test data pair is marked as an abnormal data pair; When the preset response information is consistent with the actual measured response information, the current test data pair is marked as a normal data pair; The test data set includes a defect test data set and a normal test data set. All abnormal data pairs are integrated to obtain the defect test data set, and all normal data pairs are integrated to obtain the normal test data set.

6. A software automated precision testing and analysis platform according to claim 5, characterized in that: Accurately locate the defective code line based on the test data set and provide feedback display, which specifically includes the following steps: An abnormal data pair is randomly retrieved from the defect test data set, and all running code line number information contained in the corresponding test case code line number running set is retrieved based on the test case associated with the abnormal data pair, and marked as the defect code line number set; Traverse all the test case code execution sets in the normal test data set, identify the running code line numbers contained in the test case code execution set, select a test case code execution set whose running code line numbers overlap the most with the defect code line number set, obtain all the running code line number information contained therein, and mark it as the comparison code line number set; The different code line numbers between the defective code line number set and the comparison code line number set are marked as defective code line numbers, and feedback is provided.

7. A software automated precision testing and analysis platform according to claim 6, characterized in that: It also includes a coverage optimization module, which is used to call a test case code line execution set and perform correlation integration analysis and optimization to select test cases based on the currently imported test cases.

8. A software automated precision testing and analysis platform according to claim 7, characterized in that: The following steps are used to analyze and optimize the test cases based on the currently imported test cases: S1: Retrieve the test case code line execution set to obtain the executed code line number data, and integrate to obtain the code line number set; S2: traverse the test case code execution set to identify the running code line number contained in the test case code execution set, select a test case code execution set with the largest number of running code lines and mark it as the first test case; S3: Based on the line numbers included in the code execution set in the first test case, mark the covered line number data in the code line number set; S4: Then, from the remaining test case code execution sets, select the next software trial case code execution set that can cover the most line number data in the uncovered code line number set and mark it as the second test case; S5: Repeat step S4 until all line number data in the code line number set are covered, that is, all code lines in the code line number set are triggered, and all selected test cases are integrated to obtain optimized test cases.

9. The software automated precision testing and analysis platform according to claim 1, characterized in that: Identify the modified software code to obtain the difference features, associate the test case code line execution set based on the difference features, and perform source tracing and association to determine the scope of the regression test case: Compare the modified software code with the software code before modification to determine the modified code line number data; Based on the modified code line number data, the test case code line execution set is associated with the modified code line number to determine which test cases are associated with, and the test case scope of the regression test is obtained through integration.

10. A software automated precision testing and analysis platform according to claim 9, characterized in that: Based on the modified code line number data, the test case code line execution set is associated with which test cases are determined, and the test case range of the regression test is obtained by integrating: obtaining the modified code line number set, traversing the test case code execution set to identify the test case in the test case code execution set that contains the most modified code line numbers, and marking it as the first regression test case; Then, from the remaining test case code execution sets, select the next test case code execution set that can cover the most line number data in the uncovered modified code line number set and mark it as the second regression test case. Repeat the above steps until all line number data in the modified code line number set are covered, and integrate all the selected regression test cases to obtain the test case scope of the regression test.

Images