Application program testing method, device, storage medium and program product

Abstract

The embodiment of the present invention provides an application testing method, device, storage medium and program product. The method includes: obtaining multiple test cases; executing multiple test cases in a first version of the application source code to determine the code line coverage information of the multiple test cases; in response to a change operation on the first version of the application source code, determining multiple lines of code corresponding to the change operation; determining a target test case to be executed based on the code line coverage information of the multiple test cases, the preset dependencies between the multiple test cases and the multiple lines of code, so as to execute the target test case in the changed second version of the application source code. In this solution, when the application source code is changed, there is no need to use multiple test cases to re-test the changed source code, and the testing efficiency is high.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to an application testing method, device, storage medium and program product. Background Art

[0002] Continuous Integration (CI) / Continuous Delivery (CD) is a method of frequently delivering applications to customers by introducing automation during the application development phase. In the current application development model, multiple developers are often required to collaborate on the development of an application. Generally, multiple developers write their own functional modules or some developers are jointly responsible for the development of the same functional module. Therefore, the code of each developer can be integrated into a shared repository through a version control system. Each developer can create a corresponding branch according to different requirements. After completing the requirements, each developer submits a merge to integrate the branch code developed by each developer and merge it into the main branch. CI can help developers merge code changes into the main branch more conveniently. Once the developer merges the modified code into the main branch, the system will automatically build the application and run test cases to verify these changes to ensure that these changes do not damage the application. In the development process of an application, version iteration, that is, code changes, is often involved. In the CI / CD development model, users can change the source code of the application through pull requests (PRs). After CI completes the automated processes of building and testing, CD can automatically deploy the verified code from the development environment to the enterprise's production environment.

[0003] Currently, testers prepare multiple test cases in advance to test the developed application. Moreover, when part of the code in the application source code is changed, these test cases are still used to retest the changed source code, which leads to low testing efficiency. Summary of the invention

[0004] The embodiments of the present invention provide an application testing method, device, storage medium and program product. When the application source code is changed, only the test cases related to the code lines affected by the changed behavior need to be used to test the changed source code, thereby achieving high testing efficiency.

[0005] In a first aspect, an embodiment of the present invention provides an application testing method, the method comprising:

[0006] Obtaining code line coverage information of the multiple test cases obtained after executing the multiple test cases in the first version of the application source code;

[0007] In response to a change operation on the current version of the application source code, determining a plurality of lines of code corresponding to the change operation relative to the first version of the application source code;

[0008] According to the code line coverage information of the multiple test cases, the preset dependencies between the multiple test cases and the multiple lines of code, the target test cases to be executed are determined to execute the target test cases in the changed second version of the application source code.

[0009] In a second aspect, an embodiment of the present invention provides an application testing device, the device comprising:

[0010] An acquisition module, configured to acquire code line coverage information of a plurality of test cases obtained after executing the plurality of test cases in the first version of the application source code;

[0011] a response module, configured to, in response to a change operation on a current version of the application source code, determine a plurality of lines of code corresponding to the change operation relative to the first version of the application source code;

[0012] A determination module is used to determine a target test case to be executed based on the code line coverage information of the multiple test cases, the preset dependencies between the multiple test cases and the multiple lines of code, so as to execute the target test case in the changed second version of the application source code.

[0013] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the application testing method as described in the first aspect.

[0014] In a fourth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor executes the application testing method as described in the first aspect.

[0015] In a fifth aspect, an embodiment of the present invention provides a computer program product, comprising: a computer program, which, when executed by a processor of an electronic device, causes the processor to execute the application testing method as described in the first aspect.

[0016] The application testing method provided by the embodiment of the present invention can determine the correspondence between test cases and code lines by obtaining the code line coverage information of multiple test cases obtained after executing multiple test cases in the first version of the application source code. Afterwards, in response to the change operation on the current version of the application source code, it is possible to determine the multiple lines of code corresponding to the change operation relative to the first version of the application source code, that is, the impact result of the change operation: the multiple lines of code in the second version of the application source code after the change are different from the first version of the application source code. Afterwards, based on the dependency relationship between different test cases and the correspondence between test cases and code lines, the target test case related to the multiple lines of code can be determined from multiple test cases, and then the target test case can be directly executed in the second version of the application source code after the change, without the need to use all the above multiple test cases to re-test the changed source code, thereby improving the testing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A flowchart of an application testing method provided by an embodiment of the present invention;

[0019] Figure 2 A flowchart for determining a target test case to be executed provided by an embodiment of the present invention;

[0020] Figure 3 An application schematic diagram of an application program testing method provided by an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of an application of a directed acyclic graph provided by an embodiment of the present invention;

[0022] Figure 5 Another flow chart for determining a target test case to be executed provided by an embodiment of the present invention;

[0023] Figure 6 A schematic diagram of an application of an updated directed acyclic graph provided by an embodiment of the present invention;

[0024] Figure 7 A flowchart for determining code line coverage information for multiple test cases provided by an embodiment of the present invention;

[0025] Figure 8A schematic diagram of the structure of an application testing system provided by an embodiment of the present invention;

[0026] Fig. 9 A schematic diagram of an application of a code coverage information collection unit provided in an embodiment of the present invention;

[0027] Fig.10 A schematic diagram of the structure of an application testing device provided by an embodiment of the present invention;

[0028] Fig.11 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the case where there is no conflict between the embodiments, the following embodiments and the features in the embodiments can be combined with each other. In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0032] First, the terms or concepts involved in the embodiments of the present invention are explained:

[0033] Test Case: refers to the description of a test task for a specific application, reflecting the test plan, method, technology and strategy. It can also be simply considered that a test case is a set of test inputs, execution conditions and expected results compiled for a special goal, used to verify whether a specific application requirement is met.

[0034] Directed Acyclic Graph (DAG): refers to a directed graph without loops, which consists of multiple nodes and multiple directed edges. In an embodiment of the present invention, multiple nodes are used to represent multiple test cases, each node is associated with code line coverage information of the corresponding test case, and the directed edges between nodes represent the dependency relationship between the test cases.

[0035] Continuous Integration (CI): is an application development practice, the core of which is that team members frequently integrate their work, usually each member integrates at least once a day, which also means that multiple integrations may occur every day. Each integration is verified by automated builds (including compilation, release, and automated testing) to discover integration errors as early as possible.

[0036] Continuous Delivery (CD): It is used to complete the application output process in a short cycle to ensure that the application can be stable and continuously maintained in a state that can be released at any time. Its goal is to make the compilation, testing and release of applications faster and more frequent, thereby reducing the cost and time of application development and reducing risks.

[0037] Pull Request (PR) is a notification mechanism. For example, when you modify someone else's code, you inform the original author of your changes and ask him to merge your changes. This process is called PR.

[0038] In the process of application development, version iteration, that is, code changes, is often involved. In the CI / CD development model, users can change the application source code through pull requests (PR). After completing the automated processes of building and testing through CI, CD can automatically deploy the verified code from the development environment to the enterprise's production environment. At present, testers will prepare multiple test cases in advance to test the developed applications. Moreover, after changing part of the code in the application source code, these test cases are still used to retest the changed source code, which is inefficient.

[0039] In view of this, the embodiment of the present invention provides the following ideas to solve the problem of low testing efficiency: first, obtain the code line coverage information of multiple test cases obtained after executing multiple test cases in the first version of the application source code, that is, determine which code lines in the application source code are covered by which test cases. Afterwards, when the current version of the application source code is changed, determine the multiple lines of code corresponding to the change operation relative to the first version of the application source code, select the target test cases related to these code lines from the multiple test cases, and then directly execute the target test cases corresponding to the multiple lines of code in the second version of the application source code after the change, without using the above multiple test cases to re-test the changed source code, thereby improving the testing efficiency.

[0040] The following describes the application program testing solution provided by the embodiment of the present invention in conjunction with the accompanying drawings.

[0041] Figure 1 A flowchart of an application testing method provided by an embodiment of the present invention, the method can be executed by a server, such as Figure 1 As shown, the method comprises the following steps:

[0042] 101. Obtain code line coverage information of multiple test cases obtained after executing the multiple test cases in the first version of the application source code.

[0043] 102. In response to a change operation on a current version of the application source code, determine a plurality of lines of code corresponding to the change operation relative to the first version of the application source code.

[0044] 103. Determine a target test case to be executed according to the code line coverage information of the multiple test cases, the preset dependencies between the multiple test cases, and the multiple lines of code, so as to execute the target test case in the changed second version of the application source code.

[0045] In actual applications, when it is necessary to test an application, multiple test cases corresponding to the application can be obtained first. Specifically, the steps for obtaining multiple tests are as follows: determine the test scenario and / or the functional modules included in the first version of the application source code; obtain multiple test cases according to the test scenario and / or the functional modules. In an embodiment of the present invention, the first version refers to the initial version of the application, and the current version can be the first version or another version different from the first version. In an embodiment of the present invention, the version obtained after the current version is changed is named the second version.

[0046] It should be understood that in order to ensure the integrity of the application test, multiple test cases can cover the functional modules included in the application source code. Specifically, as an optional implementation method, the functional modules included in the first version of the application source code can be determined first, and then the test cases corresponding to the functional modules can be obtained as the multiple test cases to be used in the end. For example, assuming that the functional modules included in the application source code include a data processing module, a data storage module, and a log module, then after determining the above-mentioned functional modules, multiple test cases corresponding to the data processing module, the data storage module, and the log module can be directly obtained as the multiple test cases to be used in the end. The acquisition of multiple test cases here refers to obtaining the test cases written by the test case writers for these functional modules.

[0047] As another optional implementation method, in order to ensure the comprehensiveness of application testing, multiple test cases can be determined based on the test scenario. For ease of understanding, for example, assuming that the application is a commodity sales application, then it should be understood that the number of user visits to the commodity sales application on ordinary weekdays or holidays is A (test scenario 1), and the number of user visits on commodity promotion days is B (test scenario 2), and B is much larger than A. In these two test scenarios, the requirements for the performance and stability of the application are also different. Therefore, different test cases can be selected for different test scenarios, so as to finally use multiple test cases.

[0048] As another optional implementation method, the above-mentioned functional modules and test scenarios can be combined. On the basis of ensuring that multiple test cases cover the functional modules included in the application source code, corresponding test cases can be selected as the multiple test cases to be used in the end for different test scenarios, so as to ensure the comprehensiveness of the application test while ensuring the smooth completion of the application test.

[0049] After obtaining multiple test cases, the multiple test cases are executed in the first version of the application source code. In the process of executing the multiple test cases, the detection tool can be used to determine the code line coverage information of the multiple test cases, that is, to determine which code lines in the application source code are covered by which test cases, which can also be simply understood as: determining the correspondence between code lines and test cases. At this time, the obtained code line coverage information of the multiple test cases will be stored.

[0050] Among them, the selection of the detection tool can be determined according to the programming language type of the application source code. Specifically, for example, assuming that the programming language type of the application source code is Python, the detection tool can be Coverage.py, which is a tool for measuring the code coverage of Python programs. It can help developers identify which code lines, statements or branches are not covered by tests, thereby guiding developers to write more test cases and improve code quality. Assuming that the programming language type of the application source code is C / C++, the detection tool can be gcov, which is a tool in GCC (GNU Compiler Collection). Assuming that the programming language type of the application source code is Java, the detection tool can be JaCoCo, etc., which are not listed here one by one. JaCoCo is a tool for calculating Java code coverage. It provides multiple ways to collect and report code coverage data to help developers understand which parts of the code their tests cover.

[0051] It should be noted that when the current version of the application source code is changed, in response to the change operation on the current version of the application source code, multiple lines of code corresponding to the change operation are determined relative to the first version of the application source code. The multiple lines of code are the multiple lines of code that have changed in the source code. For example, assuming that the current version of the application source code contains 500 lines of code, of which lines 200-300 have changed, then the above "multiple lines of code" refers to these lines 200-300. In this process, the multiple lines of code that have changed can be obtained by comparing the first version with the second version of the application source code obtained after the change.

[0052] If the current version is the first version, after obtaining the above-mentioned "multiple lines of code", the target test cases to be executed can be determined directly based on the code line coverage information of the above-mentioned multiple test cases, the dependencies between the preset multiple test cases and the multiple lines of code.

[0053] If the current version is a version other than the first version, the code line coverage information of the previously stored multiple test cases can be directly obtained, without having to determine the code line coverage information of the multiple test cases by executing the multiple test cases again, so as to improve work efficiency. Afterwards, the target test cases to be executed can be determined based on the code line coverage information of the multiple test cases, the preset dependencies between the multiple test cases, and the multiple lines of code.

[0054] The dependencies between the test cases are manually configured when generating these test cases. It should be understood that after multiple lines of code are changed relative to some lines of code in the source code, in order to ensure the accuracy of the application test, some test cases related to the multiple lines of code can be determined from multiple test cases. When determining these test cases, not only the correspondence between the test cases and the lines of code should be considered, but also the dependencies between the multiple test cases should be considered, that is, the target test cases to be executed are determined based on the code line coverage information of the multiple test cases, the preset dependencies between the multiple test cases and the multiple lines of code.

[0055] For ease of understanding, for example, assume that the "multiple lines of code" are the 200-300 lines of code in the first version of the application source code, and based on the code line coverage information of multiple test cases, it can be determined that the test cases corresponding to the multiple lines of code include test case A and test case B, and test case A needs to rely on test case C for execution, and test case B needs to rely on test case D for execution. Then, the new test cases finally determined, that is, the target test cases to be executed, include test case A, test case B, test case C and test case D at the same time.

[0056] Furthermore, after determining the target test case to be executed, the target test case can be directly executed in the changed second version of the application source code. Based on the above, the application testing method provided by the embodiment of the present invention can obtain the correspondence between test cases and code lines after obtaining the code line coverage information of multiple test cases obtained after executing multiple test cases in the first version of the application source code. Afterwards, when the current version of the application source code is changed, the target test case related to the multiple lines of code corresponding to the change operation relative to the first version of the application source code is selected from the multiple test cases, and then the target test case corresponding to the multiple lines of code is directly executed in the changed second version of the application source code, without the need to re-test the changed source code using the above multiple test cases, thereby improving the testing efficiency.

[0057] Figure 2 A flowchart for determining a target test case to be executed provided by an embodiment of the present invention, such as Figure 2 As shown, according to the code line coverage information of multiple test cases, the dependencies between multiple test cases and multiple lines of code, determining the target test cases to be executed specifically includes the following steps:

[0058] 201. Generate a directed acyclic graph based on the code line coverage information of multiple test cases and the dependency relationship between the multiple test cases, wherein the multiple test cases are determined as multiple nodes in the directed acyclic graph, each node is associated with the code line coverage information of the corresponding test case, and the directed edges between the nodes represent the dependency relationship between the test cases.

[0059] 202. Determine target test cases to be executed based on the directed acyclic graph and multiple lines of code.

[0060] For ease of understanding, the following Figure 3 The following example illustrates the code line coverage information of multiple test cases and the dependencies between multiple test cases:

[0061] Figure 3 There are 4 test cases and 4 different functional modules. The 4 test cases are test case A, test case B, test case C and test case D, and the 4 functional modules are functional module 1 (corresponding to 1-500 lines of code), functional module 2 (corresponding to 1-200 lines of code), functional module 3 (corresponding to 1-100 lines of code) and functional module 4 (corresponding to 1-300 lines of code). Test case A covers lines 200-300 of code in functional module 1, test case B covers lines 400-500 of code in functional module 1, and test case B depends on test case A. Test case C covers lines 20-100 of code in functional module 2 and lines 20-30 of code in functional module 3. Test case D covers lines 60-90 of code in functional module 3 and lines 200-300 of code in functional module 4, and test case D depends on test case B and test case C.

[0062] After obtaining the code line coverage information of the above multiple test cases and the dependency relationship between the multiple test cases, a directed acyclic graph can be generated based on this. The directed acyclic graph can be specifically referred to in Figure 4 .exist Figure 4There are 4 nodes in total, which are used to represent test case A, test case B, test case C and test case D respectively. Among them, the directed edge between test case A and test case B indicates that test case B depends on test case A. Similarly, the directed edge between test case B and test case D indicates that test case D depends on test case B. The directed edge between test case C and test case D indicates that test case D depends on test case C. The "Functional Module 1: 200-300" next to test case A indicates the code line coverage information associated with the node of test case A. The specific content of the code line coverage information is: test case A covers the 200th to 300th lines of code in function module 1. Similarly, the "Functional Module 1: 400-500" next to test case B indicates the code line coverage information associated with the node of test case B. The specific content of the code line coverage information is: test case B covers the 400th to 500th lines of code in function module 1. "Functional module 2: 20-100, Functional module 3: 20-30" next to test case C indicates the code line coverage information associated with the node of test case C. The specific content of the code line coverage information is: test case C covers lines 20-100 of code in function module 2 and lines 20-30 of code in function module 3. "Functional module 3: 60-90, Functional module 4: 200-300" next to test case D indicates the code line coverage information associated with the node of test case D. The specific content of the code line coverage information is: test case D covers lines 60-90 of code in function module 3 and lines 200-300 of code in function module 4. It should be noted that in Figure 4 In the graph, the two special nodes "start" and "end" are logical virtual nodes used to represent the head and tail of the directed acyclic graph.

[0063] After obtaining the above-mentioned directed acyclic graph, the target test case to be executed can be determined according to the directed acyclic graph and multiple lines of code. The specific steps are as follows: determine at least one test case covering multiple lines of code in the directed acyclic graph; determine the test cases that at least one test case depends on in the directed acyclic graph; determine that the target test case includes at least one test case and at least one test case that each test case depends on.

[0064] It should be understood that by generating a directed acyclic graph based on the code line coverage information of multiple test cases and the dependency relationship between multiple test cases, the visualization effect of the information is increased, and it is convenient to quickly and accurately find at least one test case that covers multiple lines of code, and the test cases that at least one test case depends on. In practical applications, the at least one test case and the test cases that at least one test case depends on can be directly determined as the final target test cases, thus ensuring the accuracy of the target test cases. It should be noted that when executing the target test case, it is necessary to first execute the test cases on the dependent side of the target test case, and then execute the test cases on the dependent side. For example, assuming that the target test case includes: test case A, test case B and test case D, wherein test case B depends on test case A, and test case D depends on test case B, when the target test case is specifically executed, it will be executed in the order of test case A, test case B, and test case D.

[0065] In actual applications, the change operations on the first version of the application source code may include: modifying, adding, deleting the code lines of an existing functional module, and adding a new functional module.

[0066] Based on this, in the embodiment of the present invention, the first version of the application source code includes codes corresponding to multiple functional modules, and the change operation includes: the operation of adding a functional module. Correspondingly, the multiple lines of code corresponding to the change operation include: the code of the added functional module. Figure 5 To perform the target test case determination process.

[0067] Figure 5 Another flow chart for determining a target test case to be executed provided by an embodiment of the present invention is as follows: Figure 5 As shown, determining the target test case to be executed specifically includes the following steps:

[0068] Get the new test cases generated for the newly added functional modules and the dependencies between the new test cases and the existing test cases.

[0069] 502. Execute the new test case in the changed second version of the application source code to determine code line coverage information of the new test case.

[0070] 503. Update the directed acyclic graph according to the dependency relationship between the new test case and the multiple test cases and the code line coverage information of the new test case.

[0071] 504. Determine new test cases covering the codes of the newly added functional modules in the updated directed acyclic graph.

[0072] 505. Determine the test cases on which the new test case depends in the updated directed acyclic graph.

[0073] 506. Determine that the target test cases include new test cases and test cases that the new test cases depend on.

[0074] It should be understood that in actual applications, if the change operation on the source code of the first version of the application is the operation of adding a new functional module, then the multiple lines of code corresponding to the change operation are the codes corresponding to the new functional module. In specific implementation, since there is a new functional module, a new test case corresponding to the new functional module should be generated, and the dependency relationship between the new test case and multiple test cases should be configured. For example, suppose in Figure 3 and Figure 4 A functional module 5 is newly added on the basis of the provided embodiment. Then, a test case E (i.e., a new test case) can be generated for the functional module 5, and the dependency relationships between the test case E and the test case A, test case B, test case C, and test case D can be determined respectively. For example, it is finally determined that the test case A depends on the test case E.

[0075] Afterwards, execute the new test case in the changed second version of the application source code to determine the code line coverage information of the new test case, and update the previous directed acyclic graph based on the dependency relationship between the new test case and multiple test cases and the code line coverage information of the new test case: for example, add a node corresponding to the new test case, modify the connection edges between the nodes, associate the corresponding code line coverage information for the newly added node, modify the code line coverage information associated with other existing nodes, and so on. Assuming that functional module 5 corresponds to 1-200 lines of code, and test case E covers lines 10-180 of code in functional module 5 and lines 50-90 of code in functional module 1, combined with the above dependency relationship "test case A depends on test case E", the updated directed acyclic graph can be obtained, which can be seen in detail. Figure 6 .

[0076] After obtaining the updated directed acyclic graph, the above-mentioned new test cases covering the code of the newly added functional module and the test cases on which the new test cases depend can be determined in the updated directed acyclic graph, and the new test cases and the test cases on which the new test cases depend can be used as the final target test cases. The specific implementation of this process can refer to the above-mentioned embodiment and will not be repeated here. That is to say, for the change operation of the newly added functional module, the target test cases finally determined include not only the new test cases designed for the newly added functional module, but also other test cases on which the new test cases depend, so that a comprehensive and accurate test can be performed on the result of the change operation.

[0077] Figure 7A flowchart for determining code line coverage information for multiple test cases provided by an embodiment of the present invention, such as Figure 7 As shown, determining the code line coverage information of multiple test cases specifically includes the following steps:

[0078] 701. Obtain at least one basic test case obtained by splitting multiple test cases and advanced test cases that are respectively dependent on at least one basic test case, wherein the test configuration information contained in a test case includes the test configuration information contained in the split basic test case and the test configuration information contained in the corresponding advanced test case, and each basic test case includes common test configuration information of at least two test cases among the multiple test cases.

[0079] 702. Obtain dependency relationships between the optimized test cases, wherein the optimized test cases include at least one basic test case and advanced test cases that are respectively dependent on the at least one basic test case.

[0080] 703. Obtain code line coverage information of multiple test cases obtained after executing the optimized multiple test cases in the first version of the application source code.

[0081] In actual applications, there may be such a situation: the initially generated multiple test cases may contain some common test configuration information. For example, the initially generated test cases F, G, and H all contain the same test environment configuration information, but the difference is that the test data, test steps, and other test configuration information in each test case are different. Then, in actual applications, if test case F, test case G, and test case H are executed separately, the same test configuration information will be repeatedly executed multiple times, increasing resource consumption and test time.

[0082] Therefore, in this embodiment, for the multiple test cases initially generated, the "same test configuration information" of all or part of the test cases can be extracted as a basic test case corresponding to these test cases, and the "difference test configuration information" is used as an advanced test case, thereby completing the optimization of the initial multiple test cases. For example, test case F is split into a basic test case a, and an advanced test case b that depends on the basic test case a, while test case G is split into a basic test case a, and an advanced test case c that depends on the basic test case a, and test case H is split into a basic test case a, and an advanced test case d that depends on the basic test case a. Therefore, the optimized multiple test cases include a basic test case a, and advanced test cases b, c, and d that depend on the basic test case a, respectively.

[0083] At this time, in the first version of the application source code, the basic test case a is executed first, and then the advanced test case b, advanced test case c and advanced test case d are executed respectively. That is, the test configuration information in the basic test case a only needs to be executed once, and there is no need to repeat it multiple times, which reduces resource consumption and testing time and improves testing efficiency.

[0084] In addition, the generation of the above-mentioned directed acyclic graph can also be generated based on these optimized test cases.

[0085] In order to more intuitively understand the application testing method provided by the embodiment of the present invention, a specific embodiment of an application testing system is provided below to illustrate the solution of the present invention:

[0086] Figure 8 A schematic diagram of the structure of an application testing system provided by an embodiment of the present invention. Figure 8 As shown, the system includes a code coverage information collection unit 81, a directed acyclic graph construction unit 82 and a test case selection optimization unit 83.

[0087] The code coverage information collection unit 81 includes a code coverage information collection tool and a data analysis engine. In practical applications, the code coverage information collection tool is used to collect code line coverage information of multiple test cases, while the data analysis engine is used to analyze the code line coverage information of multiple test cases to obtain a corresponding relationship table between the test cases and the code lines in the source code.

[0088] When implementing it, Fig. 9 As shown, the code coverage information collection unit 81 includes at least the following steps in actual application:

[0089] Determine the programming language type of the application source code;

[0090] If the programming language type is Python, Coverage.py is used as the code coverage information collection tool, or if the programming language type is Java, JaCoCo is used as the code coverage information collection tool, or if the programming language type is C / C++, GCC is used as the code coverage information collection tool, or if the programming language type is other languages, the corresponding detection tool is used as the code coverage information collection tool. It should be noted that in the specific implementation, after selecting the code coverage information collection tool, the code coverage information collection tool will be integrated into the entire application testing framework;

[0091] Execute multiple test cases in the first version of the application source code, during which a code coverage information collection tool is used to determine code line coverage information of the multiple test cases, and generate a code line coverage information analysis report, wherein the code line coverage information analysis report identifies which code lines are covered by which test cases;

[0092] The data analysis engine is used to analyze the code line coverage information analysis report to generate a correspondence table between test cases and code lines in the source code.

[0093] Based on the above, when facing different programming language types, the embodiment of the present invention can select different code coverage information collection tools, thereby adapting to various application environments, and has good scalability and practicality.

[0094] The directed acyclic graph construction unit 82 is used to perform node arrangement for multiple test cases, determine the dependency relationship between multiple test cases, and generate a directed acyclic graph based on the code line coverage information of multiple test cases (i.e., the correspondence table between the test cases and the code lines in the source code obtained by the code coverage information collection unit 81) and the dependency relationship between multiple test cases. Among them, performing node arrangement for multiple test cases means organizing and arranging multiple test cases according to a certain logic and sequence to ensure the orderly progress of the test work and the accuracy of the test results.

[0095] The test case selection optimization unit 83 is used to check the code change lines of the current PR and perform code change analysis to obtain multiple lines of code for this change, as well as the directed acyclic graph (including code line coverage information of multiple test cases, dependencies between multiple test cases) and multiple lines of code obtained by the directed acyclic graph construction unit 82, determine the target test cases to be executed, and execute the target test cases in the changed application source code.

[0096] Figure 8 The system shown can execute the steps executed by the application testing method in the aforementioned embodiment. The detailed execution process and technical effects are described in the aforementioned embodiment and will not be repeated here.

[0097] The following will describe in detail the application testing device of one or more embodiments of the present invention. Those skilled in the art will appreciate that these devices can be configured using commercially available hardware components through the steps taught in this solution.

[0098] Fig.10 A schematic diagram of the structure of an application testing device provided by an embodiment of the present invention is shown in FIG. Fig.10 As shown, the device includes: an acquisition module 11, a response module 12 and a determination module 13.

[0099] An acquisition module 11 is used to acquire code line coverage information of multiple test cases obtained after executing the multiple test cases in the first version of the application source code;

[0100] A response module 12, configured to respond to a change operation on the current version of the application source code and determine, relative to the first version of the application source code, a plurality of lines of code corresponding to the change operation;

[0101] The determination module 13 is used to determine the target test case to be executed based on the code line coverage information of the multiple test cases, the preset dependencies between the multiple test cases and the multiple lines of code, so as to execute the target test case in the changed second version of the application source code.

[0102] Optionally, the current version is the first version or another version different from the first version.

[0103] Among them, optionally, the determination module 13 is specifically used to: generate a directed acyclic graph according to the code line coverage information of the multiple test cases and the dependency relationship between the multiple test cases, and determine the target test cases to be executed according to the directed acyclic graph and the multiple lines of code, wherein the multiple test cases are determined as multiple nodes in the directed acyclic graph, each node is associated with the code line coverage information of the corresponding test case, and the directed edges between the nodes represent the dependency relationship between the test cases.

[0104] Among them, optionally, the determination module 13 is specifically used to: determine at least one test case covering the multiple lines of code in the directed acyclic graph; determine the test cases on which the at least one test case depends in the directed acyclic graph; determine that the target test case includes the at least one test case and the test cases on which the at least one test case depends.

[0105] Wherein, optionally, the first version of the application source code includes codes corresponding to multiple functional modules, the change operation includes: an operation of adding a new functional module, and correspondingly, the multiple lines of code include: the code of the newly added functional module. Based on this, the acquisition module 11 is also used to obtain a new test case generated for the newly added functional module, and the dependency relationship between the new test case and the multiple test cases.

[0106] The determination module 13 is used to execute the new test case in the changed second version of the application source code to determine the code line coverage information of the new test case, and to update the directed acyclic graph according to the dependency relationship between the new test case and the multiple test cases and the code line coverage information of the new test case.

[0107] Among them, optionally, the determination module 13 is also specifically used to: determine the new test case that covers the code of the newly added functional module in the updated directed acyclic graph; determine the test case that the new test case depends on in the updated directed acyclic graph; determine that the target test case includes the new test case and the test case that the new test case depends on.

[0108] Optionally, the acquisition module 11 is also used to obtain at least one basic test case obtained by splitting the multiple test cases and advanced test cases that are respectively dependent on the at least one basic test case, wherein the test configuration information contained in a test case includes the test configuration information contained in the split basic test case and the test configuration information contained in the corresponding advanced test case, and each basic test case includes common test configuration information in at least two test cases among the multiple test cases; and, obtain the dependency relationship between the optimized multiple test cases, wherein the optimized multiple test cases include the at least one basic test case and the advanced test cases that are respectively dependent on the at least one basic test case.

[0109] Therefore, the acquisition module 11 is specifically used to: obtain the code line coverage information of the multiple test cases obtained after executing the optimized multiple test cases in the first version of the application source code, and determine the detection tool to be used according to the programming language type of the application source code; determine the code line coverage information of the multiple test cases based on the detection tool.

[0110] Fig.10 The device shown can execute the steps provided in the aforementioned embodiments. For detailed execution process and technical effects, please refer to the description in the aforementioned embodiments, which will not be repeated here.

[0111] In one possible design, the above Fig.10 The structure of the application testing device shown can be implemented as an electronic device. Fig.11 As shown, the electronic device may include: a processor 21, a memory 22, and a communication interface 23. The memory 22 stores executable codes, and when the executable codes are executed by the processor 21, the processor 21 can at least implement the application program testing method provided in the above-mentioned embodiment.

[0112] In addition, an embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the application testing method provided in the aforementioned embodiment.

[0113] In addition, an embodiment of the present invention provides a computer program product, which includes: a computer program, when the computer program is executed by a processor of an electronic device, the processor can at least implement the application program testing method provided in the above embodiment.

[0114] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art may understand and implement the present invention without creative effort.

[0115] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and application programs. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An application testing method, characterized in that: include: Obtaining code line coverage information of multiple test cases obtained after executing the multiple test cases in the first version of the application source code, wherein the multiple test cases are determined according to the test scenarios and / or the functional modules included in the first version of the application source code; In response to a change operation on the current version of the application source code, determining a plurality of lines of code corresponding to the change operation relative to the first version of the application source code; Determine a target test case to be executed according to the code line coverage information of the multiple test cases, the preset dependency relationship between the multiple test cases and the multiple lines of code, so as to execute the target test case in the changed second version of the application source code; The method of determining target test cases to be executed based on the code line coverage information of the multiple test cases, the dependencies between the multiple test cases and the multiple lines of code includes: generating a directed acyclic graph based on the code line coverage information of the multiple test cases and the dependencies between the multiple test cases, wherein the multiple test cases are determined as multiple nodes in the directed acyclic graph, each node is associated with the code line coverage information of the corresponding test case, and the directed edges between the nodes represent the dependencies between the test cases; and determining the target test cases to be executed based on the directed acyclic graph and the multiple lines of code.

2. The method according to claim 1, characterized in that The current version is the first version or another version different from the first version.

3. The method according to claim 1, characterized in that Determining a target test case to be executed according to the directed acyclic graph and the multiple lines of code includes: Determining at least one test case covering the plurality of lines of code in the directed acyclic graph; Determining in the directed acyclic graph the test cases on which each of the at least one test case depends; It is determined that the target test case includes the at least one test case and the test cases on which the at least one test case depends.

4. The method according to claim 1, characterized in that: The first version of the application source code includes codes corresponding to the plurality of functional modules, the change operation includes: an operation of adding a new functional module, and correspondingly, the plurality of lines of code include: codes of the newly added functional module; The method further comprises: Acquire a new test case generated for the newly added functional module, and a dependency relationship between the new test case and the multiple test cases; Execute the new test case in the changed second version of the application source code to determine code line coverage information of the new test case; The directed acyclic graph is updated according to the dependency relationship between the new test case and the multiple test cases and the code line coverage information of the new test case.

5. The method according to claim 4, characterized in that Determining a target test case to be executed according to the directed acyclic graph and the multiple lines of code includes: Determine the new test case covering the code of the newly added functional module in the updated directed acyclic graph; Determining, in the updated directed acyclic graph, the test cases on which the new test case depends; It is determined that the target test case includes the new test case and the test case on which the new test case depends.

6. The method according to claim 1, characterized in that The method further comprises: Obtain at least one basic test case obtained by splitting the multiple test cases and advanced test cases that are respectively dependent on the at least one basic test case, wherein the test configuration information contained in a test case includes the test configuration information contained in the split basic test case and the test configuration information contained in the corresponding advanced test case, and each basic test case includes common test configuration information in at least two test cases among the multiple test cases; Obtaining a dependency relationship between the optimized multiple test cases, wherein the optimized multiple test cases include the at least one basic test case and advanced test cases that are respectively dependent on the at least one basic test case; The acquiring of code line coverage information of the multiple test cases obtained after executing the multiple test cases in the first version of the application source code includes: Code line coverage information of the multiple test cases obtained after executing the optimized multiple test cases in the first version of the application source code is obtained.

7. The method according to any one of claims 1 to 6, characterized in that The acquiring of code line coverage information of the multiple test cases obtained after executing the multiple test cases in the first version of the application source code: Execute multiple test cases on the first version of the application source code; Determine the detection tool to be used according to the programming language type of the source code of the application program; Code line coverage information of the multiple test cases is determined based on the detection tool.

8. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the application testing method as described in any one of claims 1 to 7.

9. A non-transitory machine-readable storage medium, characterized in that: The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the application testing method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that include: A computer program, when executed by a processor of an electronic device, causes the processor to execute the application testing method according to any one of claims 1 to 7.

Images