A testing method for automatically tracing software bugs

Abstract

The present invention proposes a testing method for automatically tracing software bugs. By querying historical submissions, a binary method combined with code log analysis is used to preliminarily confirm a commit ID number set that may introduce bugs. Then, an automated testing framework module is used to generate a test plan according to a bug tracing analysis tool module, and a regression test case set is used to perform automated testing on a software product to be tested, thereby realizing tracing of code submissions that introduce bugs. The present invention can be executed quickly and automatically, is on duty 24 hours a day, has high reliability, high execution efficiency, eliminates artificial unstable factors, and improves testing efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field related to software automated testing, and in particular to a testing method for automatically tracing software bugs. Background Art

[0002] In the software development process, as the code is continuously integrated, some bugs will be introduced. For large-scale software projects, there are a large number of software developers working together. When a large amount of code is integrated, various marginal effects will often occur, and bugs may occur after the software is integrated. At this time, it is necessary to use software testing methods to trace back the specific code changes from which the software bugs come. Development testers repeatedly roll back versions and test to locate which code submissions introduced the bugs. These backtracking tasks are generally completed manually by testers, which requires testers to have certain experience and is time-consuming and labor-intensive. If the amount of code submissions is relatively large, manual backtracking is both time-consuming and error-prone.

[0003] In view of the above-mentioned defects, there is an urgent need to design a testing method for automatically tracing software bugs to improve software testing efficiency. Summary of the invention

[0004] In order to solve the problems mentioned above, the present invention proposes a testing method for automatically tracing software bugs, which automatically traces back to the code submission where the bug was introduced by querying historical submissions and adopting an efficient algorithm, thereby improving software testing efficiency.

[0005] A testing method for automatically tracing software bugs, the tool comprising: an SCM code management system module, an automated testing framework module, a bug tracing analysis tool module and a test report management system module;

[0006] The SCM code management system module is used to record the commit id of each code submission;

[0007] The automated testing framework module is used to perform automated testing on the software product under test using a set of regression test cases according to a test plan;

[0008] The bug tracing analysis tool module is used to generate a test plan;

[0009] The test report management system module is used to record test results.

[0010] Further, obtaining the software product to be tested according to the SCM code management system module and the test report management system module;

[0011] Generate a test plan according to the test report management system module and the bug tracing analysis tool module;

[0012] The automated testing framework module is used to collect test cases that have failed testing to form a new regression test case set.

[0013] Further, the SCM code management system module is preferably Gitea;

[0014] The automated testing framework module is preferably a Pytest automated testing framework;

[0015] The test report management system module is preferably testlink.

[0016] Based on the above test tool, the present invention also provides a test method, characterized in that: the method comprises the following steps:

[0017] Step 1: After a test failure occurs, a commit ID number set of the software product to be tested is obtained according to the SCM code management system module and the test report management system module;

[0018] Step 2: Generate a software product to be tested according to a commit ID in the number set, and generate a test plan according to the test report management system module and the bug tracing analysis tool module;

[0019] Step 3: The automated testing framework module uses a regression test case set to perform automated testing on the software product under test according to the test plan;

[0020] Step 4: Repeat steps 2 and 3 until the commit ID that introduces the bug is determined.

[0021] Step 5: Automatically send the traceability results via email to complete the automatic traceability test.

[0022] Furthermore, the step 1 comprises:

[0023] Step 1-1, obtain the commit ID of the most recent successful test through the test report management system module;

[0024] Step 1-2: Obtain all commit IDs between the last successful test and the current test through the SCM code management system module;

[0025] Step 1-3: For all commit IDs obtained in step 1-2, use binary search combined with code log analysis to confirm the commit ID number set of the software product to be tested.

[0026] Furthermore, the binary search method rapidly reduces the scope of the code that introduces the bug by continuously dividing the range of commit IDs into two; the code log analysis reduces the scope of the code that introduces the bug by analyzing the specific changes in each code submission.

[0027] The beneficial effects of the present invention are:

[0028] The tool and method designed by the present invention query historical submissions, use the binary method combined with code log analysis to preliminarily confirm the commit id number set that may introduce bugs, then use the test plan generated by the automated testing framework module according to the bug tracing analysis tool module, and use the regression test case set to perform automated testing on the software product to be tested, thereby realizing the tracing of the code submission that introduces the bug. The present invention can be quickly and automatically executed, on duty 24 hours a day, with high reliability and high execution efficiency, and eliminates human unstable factors, thereby improving the testing efficiency; at the same time, the regression test case of the present invention can be updated according to the test case that failed the test, so that the case selection is accurate and the execution efficiency is high; and the email feedback is timely and effective, the process is recorded, and can be traced. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 FIG. 4 is a schematic diagram of the structure of the testing device of the present invention.

[0030] Figure 2 Schematic diagram of the testing method of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described below in conjunction with the embodiments.

[0032] The following examples are used to illustrate the present invention, but they cannot be used to limit the scope of protection of the present invention. The conditions in the examples can be further adjusted according to specific conditions. Simple improvements to the method of the present invention under the premise of the concept of the present invention belong to the scope of protection claimed in the present invention.

[0033] Example 1

[0034] like Figure 1 As shown, a testing method for automatically tracing software bugs, the tool includes: an SCM code management system module, an automated testing framework module, a bug tracing analysis tool module and a test report management system module.

[0035] The SCM code management system module is used to record the commit id of each code submission. The bug tracing analysis tool module is used to generate a test plan. The test report management system module is used to record test results.

[0036] The software product to be tested is acquired according to the SCM code management system module and the test report management system module; a test plan is generated according to the test report management system module and the bug tracing analysis tool module. The automated testing framework module is used to perform automated testing on the software product to be tested using a regression test case set according to the test plan.

[0037] Preferably, the automated testing framework module is used to collect test cases that have failed tests to form a new regression test case set.

[0038] Preferably, the SCM code management system module is preferably Gitea.

[0039] Preferably, the automated testing framework module is preferably a Pytest automated testing framework.

[0040] Preferably, the test report management system module is preferably testlink.

[0041] Preferably, the device also includes a mail server for sending the tracing results.

[0042] Preferably, the automated testing framework module in the device is connected to the testing cluster.

[0043] Example 2

[0044] like Figure 2 As shown, based on the above test tool, the present invention also provides a test method, which includes the following steps:

[0045] Step 1: After a test failure occurs, the commit id number set of the software product to be tested is obtained according to the SCM code management system module and the test report management system module.

[0046] The step 1 comprises:

[0047] Step 1-1, obtain the commit ID of the most recent successful test through the test report management system module;

[0048] Step 1-2: Obtain all commit IDs between the last successful test and the current test through the SCM code management system module;

[0049] Step 1-3: For all commit IDs obtained in step 1-2, the commit ID number set of the software product to be tested is confirmed by binary search combined with code log analysis. The binary search method quickly narrows down the scope of the code that introduces the bug by continuously dividing the scope of the commit ID into two; the code log analysis narrows down the scope of the code that introduces the bug by analyzing the specific changes in each code submission.

[0050] Step 2: Generate a software product to be tested according to a commit ID in the number set, and generate a test plan according to the test report management system module and the bug tracing analysis tool module.

[0051] Step 3: The automated testing framework module uses a regression test case set to perform automated testing on the software product under test according to the test plan.

[0052] Step 4. Repeat steps 2 and 3 until the commit ID that introduced the bug is determined.

[0053] Step 5: Automatically send the traceability results via email to complete the automatic traceability test.

[0054] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A testing method for automatically tracing software bugs, characterized by: The test method is run on a tool that automatically traces software bugs, and the tool includes: an SCM code management system module, an automated testing framework module, a bug tracing analysis tool module, and a test report management system module; The SCM code management system module is used to record the commit id of each code submission; The automated testing framework module is used to perform automated testing on the software product under test using a set of regression test cases according to a test plan; The bug tracing analysis tool module is used to generate a test plan; The test report management system module is used to record test results; Acquire the software product to be tested according to the SCM code management system module and the test report management system module; Generate a test plan according to the test report management system module and the bug tracing analysis tool module; The automated testing framework module is used to collect test cases that failed the test to form a new regression test case set; The testing method comprises the following steps: Step 1: After a test failure occurs, a commit ID number set of the software product to be tested is obtained according to the SCM code management system module and the test report management system module; Step 2: Generate a software product to be tested according to a commit ID in the number set, and generate a test plan according to the test report management system module and the bug tracing analysis tool module; Step 3: The automated testing framework module uses a regression test case set to perform automated testing on the software product under test according to the test plan; Step 4: Repeat steps 2 and 3 until the commit ID that introduces the bug is determined. Step 5: Automatically send the traceability results via email to complete the automatic traceability test; The step 1 comprises: Step 1-1, obtain the commit ID of the most recent successful test through the test report management system module; Step 1-2: Obtain all commit IDs between the last successful test and the current test through the SCM code management system module; Step 1-3: Use binary search and code log analysis to identify the commit ID number set of the software product to be tested. The binary search method quickly reduces the scope of the code that introduces the bug by continuously dividing the range of commit IDs into two; the code log analysis reduces the scope of the code that introduces the bug by analyzing the specific changes in each code submission.

2. A method for automatically tracing software bugs according to claim 1, characterized in that: The SCM code management system module is Gitea; The automated testing framework module is the Pytest automated testing framework; The test report management system module is testlink.

Images