A method, device, equipment and storage medium for determining test coverage

Abstract

The present invention discloses a method, device, equipment and storage medium for determining test coverage. The method comprises: obtaining the chain proportions corresponding to each execution chain in the code to be tested; determining the execution chains in the code to be tested that are respectively covered by each test case for the code to be tested during the execution process, and obtaining the test case coverage results; for each execution chain in the code to be tested, determining the number of test cases covering the current execution chain according to the test case coverage results, and determining the use case coverage ratio corresponding to the current execution chain; generating the test coverage determination result according to the use case coverage ratio and chain proportion corresponding to each execution chain. Through the technical solution of the present invention, the coverage of the test case can be evaluated in a standard, scientific and quantitative manner, and the evaluation accuracy of the coverage can be improved, so as to achieve the effect of selecting more comprehensive test cases according to the evaluation results to ensure the comprehensiveness of the test scope.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular to a method, apparatus, device and storage medium for determining test coverage. Background Art

[0002] With the development of Internet + distributed systems, there are more and more projects and the relationships between systems are complex. Whenever there is a change in requirements, all affected functions need to be tested regardless of the number of software systems involved.

[0003] There are currently two main testing methods: one is to collect real online traffic as test cases according to the interface dimension, and use the collected test cases to test the corresponding interfaces; the other is to manually write test cases according to requirements and use the test cases for testing.

[0004] In the process of implementing the present invention, it is found that there are at least the following problems in the prior art: in order to ensure the comprehensiveness of the test scope and avoid online problems, it is necessary to determine the test coverage of the test cases, but there is currently no standard, scientific and quantifiable method for determining the test coverage. Summary of the invention

[0005] The embodiments of the present invention provide a method, apparatus, device and storage medium for determining test coverage, so as to achieve a standard, scientific and quantitative evaluation of test case coverage and improve the accuracy of coverage evaluation.

[0006] In a first aspect, an embodiment of the present invention provides a method for determining test coverage, comprising:

[0007] Obtaining the chain weights corresponding to the execution chains in the code to be tested; wherein the chain weights represent the ratio of the number of execution nodes included in the corresponding execution chain to the total number of execution nodes included in the code to be tested;

[0008] Determine the execution chains in the code to be tested that are respectively covered by each test case for the code to be tested during the execution process, and obtain the test case coverage result;

[0009] For each execution chain in the code to be tested, determine the number of test cases covering the current execution chain according to the test case coverage result, and determine the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases;

[0010] A test coverage determination result is generated according to the use case coverage ratio and chain weight corresponding to each of the execution chains.

[0011] In a second aspect, an embodiment of the present invention further provides a device for determining test coverage, the device comprising:

[0012] A chain weight acquisition module is used to obtain the chain weights corresponding to each execution chain in the code to be tested; wherein the chain weight represents the ratio of the number of execution nodes contained in the corresponding execution chain to the total number of execution nodes contained in the code to be tested;

[0013] An execution chain determination module is used to determine the execution chains in the code to be tested that are respectively covered by each test case of the code to be tested during the execution process, and obtain the test case coverage result;

[0014] A coverage ratio determination module, used to determine, for each execution chain in the code to be tested, the number of test cases covering the current execution chain according to the test case coverage result, and determine the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases;

[0015] The result determination module is used to generate a determination result of the test coverage rate according to the use case coverage ratio and the chain weight corresponding to each of the execution chains.

[0016] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for determining test coverage as described in any one of the embodiments of the present invention is implemented.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for determining test coverage as described in any one of the embodiments of the present invention.

[0018] The embodiment of the present invention obtains the chain proportion corresponding to each execution chain in the code to be tested, determines the execution chains in the code to be tested that are respectively covered by each test case for the code to be tested during the execution process, and obtains the test case coverage result. For each execution chain in the code to be tested, the number of test cases covering the current execution chain is determined according to the test case coverage result, and the use case coverage ratio corresponding to the current execution chain is determined according to the number of test cases covering the current execution chain and the total number of test cases. According to the use case coverage ratio and the chain proportion corresponding to each of the execution chains, a determination result of the test coverage is generated, which solves the problem that there is currently no standard, scientific, and quantifiable method for determining test coverage and the test coverage evaluation is imperfect, realizes standard, scientific and quantitative evaluation of test case coverage, improves the accuracy of coverage evaluation, and thus realizes the effect of selecting more comprehensive test cases according to the evaluation results to ensure the comprehensiveness of the test scope. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 is a flow chart of a method for determining test coverage in Embodiment 1 of the present invention;

[0021] Figure 2 is a flow chart of a method for determining test coverage in Embodiment 2 of the present invention;

[0022] Figure 3 is a schematic diagram of a syntax tree in an embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of an execution chain and execution nodes in an embodiment of the present invention;

[0024] Figure 5 is a schematic diagram of a determination result of a test coverage in an embodiment of the present invention;

[0025] Figure 6 is a flow chart of a method for determining test coverage in Embodiment 3 of the present invention;

[0026] Figure 7 is a schematic diagram of the architecture of a test coverage management system in an embodiment of the present invention;

[0027] Figure 8is a schematic diagram of a method for determining test coverage in an embodiment of the present invention;

[0028] Fig. 9 is a schematic diagram of a device for determining test coverage in a fourth embodiment of the present invention;

[0029] Fig.10 It is a structural diagram of a computer device in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0031] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0032] Embodiment 1

[0033] Figure 1 A flowchart of a method for determining test coverage provided in the first embodiment of the present invention is provided. This embodiment can be applied to the case of determining the test coverage of a test case during the execution of a code to be tested. The method can be executed by a device for determining test coverage in the embodiment of the present invention. The device can be implemented in software and / or hardware. Figure 1 As shown, the method specifically comprises the following steps:

[0034] S110, obtaining chain proportions corresponding to each execution chain in the code to be tested.

[0035] The execution chain refers to the possible execution trajectory of the code parsed in the syntax tree, which can be understood as a hierarchical chain formed by multiple execution nodes in the execution process of the code to be tested. The execution node can be understood as a grammatical structure in the program code, such as a package, type, modifier, operator, interface, return value or code comment, which can be used as an execution node. Generally, a line of code can be used as an execution node. A code to be tested usually contains multiple execution chains.

[0036] Specifically, the code to be tested is obtained and parsed to determine the chain weights corresponding to each execution chain. The chain weight represents the ratio of the number of execution nodes contained in the corresponding execution chain to the total number of execution nodes contained in the code to be tested, reflecting the ratio of the number of execution nodes of each execution chain to the total number of execution nodes contained in the code to be tested.

[0037] Exemplarily, the number N of execution chains in the code to be tested is obtained, the number of execution nodes contained in each execution chain is determined as runPointNum, which is recorded as the chain depth runPointDeep, then the chain depth of the i-th execution chain is runPointDeep_i=sum(runPointNum); the sum of the number of execution nodes contained in each execution chain is determined, which is recorded as the method depth methodDeep, then the method depth of the code to be tested is The chain proportion is the ratio of the number of execution nodes contained in the corresponding execution chain to the total number of execution nodes contained in the code to be tested, that is, the chain proportion runPointRatio of the i-th execution chain is the ratio of the chain depth runPointDeep_i of the i-th execution chain to the method depth methodDeep, that is, runPointRatio_i=runPointDeep_i / methordDeep.

[0038] Optionally, the code to be tested includes code of a callable interface in a software system.

[0039] S120, determining the execution chains in the code to be tested that are respectively covered by each test case for the code to be tested during the execution process, and obtaining the test case coverage result.

[0040] Among them, the test case can be understood as an example written or selected by a tester for testing the code to be tested. Usually, in order to be able to test various functions of the code to be tested, multiple different test cases are used to execute the code to be tested in sequence. The embodiment of the present invention does not limit the selection of test cases.

[0041] Among them, the test case coverage results are used to represent the execution chain in the test code covered by each test case during the execution process. The test case coverage results can be represented by a data table, a mapping table or other forms. The embodiment of the present invention does not impose any restrictions on the representation form of the test case coverage results.

[0042] Specifically, in the process of executing the code to be tested using each test case, the execution chain in the code to be tested covered by each test case is determined respectively, that is, the execution chain formed by the execution nodes of each test case included in the code to be tested during the execution process is determined.

[0043] S130, for each execution chain in the code to be tested, determine the number of test cases covering the current execution chain according to the test case coverage result, and determine the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases.

[0044] Among them, the use case coverage ratio corresponding to the current execution chain can be understood as the number of times the current execution chain is executed by each test case.

[0045] Exemplarily, in the process of executing the code to be tested using different test cases, the total number of test cases is recorded, recorded as TestCaseNum, and the execution chains in the test code covered by each test case during the execution process are obtained according to the test case coverage results, so as to obtain the number of test cases covering the current execution chain for each execution chain in the code to be tested, recorded as TestCaseNum_i; the ratio of the number of test cases covering the current execution chain to the number of test cases is determined as the use case coverage ratio corresponding to the current execution chain, recorded as coverRatio_i, that is, coverRatio_i=TestCaseNum_i / TestCaseNum.

[0046] S140, generating a determination result of the test coverage rate according to the use case coverage ratio and the chain weight corresponding to each of the execution chains.

[0047] Exemplarily, according to the use case coverage ratio coverRatio_i and chain weight runPointRatio_i respectively corresponding to each of the execution chains, the size relationship, difference and ratio between the use case coverage ratio and the chain weight respectively corresponding to each of the execution chains can be obtained. Therefore, the determination result of the test coverage rate can be generated according to the size relationship, difference or ratio between the use case coverage ratio and the chain weight respectively corresponding to each of the execution chains, or the use case coverage ratio and the chain weight respectively corresponding to each of the execution chains can be directly used as the determination result of the test coverage rate to evaluate the test coverage rate of the test case.

[0048] The method of evaluating the test coverage of test cases based on the determination results of test coverage can be: if the coverage ratio of use cases corresponding to the execution chain is less than the chain proportion, it means that the test cases are not in place for the execution chain, and it is necessary to add test cases that can cover the execution chain; if the coverage ratio of use cases corresponding to the execution chain is greater than or equal to the chain proportion, it means that the test cases are in place for the execution chain, and there is no need to add test cases covering the execution chain.

[0049] The technical solution of this embodiment obtains the chain proportion corresponding to each execution chain in the code to be tested, determines the execution chains in the code to be tested that are respectively covered by each test case for the code to be tested during the execution process, and obtains the test case coverage result. For each execution chain in the code to be tested, the number of test cases covering the current execution chain is determined according to the test case coverage result, and the use case coverage ratio corresponding to the current execution chain is determined according to the number of test cases covering the current execution chain and the total number of test cases. According to the use case coverage ratio and chain proportion corresponding to each of the execution chains, a determination result of the test coverage is generated, which can evaluate the coverage of test cases in a standard, scientific and quantitative manner, improve the accuracy of coverage evaluation, and thus select more comprehensive test cases according to the evaluation results to ensure the comprehensiveness of the test scope.

[0050] Embodiment 2

[0051] Figure 2 This is a flow chart of a method for determining test coverage in Example 2 of the present invention. This embodiment is optimized based on the above embodiment. In this embodiment, the chain weight corresponding to each execution chain in the code to be tested is obtained, including: parsing the code to be tested to obtain a syntax tree corresponding to the code to be tested; according to the syntax tree, parsing the execution chains contained in the code to be tested, and determining the execution nodes respectively contained in each of the execution chains; for each of the execution chains, determining the chain weight corresponding to the current execution chain according to the number of execution nodes contained in the current execution chain and the total number of execution nodes contained in the code to be tested.

[0052] like Figure 2 As shown, the method of this embodiment specifically includes the following steps:

[0053] S210, acquiring execution chains contained in the code to be tested collected by a collection plug-in set in the business system, and information of execution nodes respectively contained in each execution chain.

[0054] The business system may be a system for executing the code to be tested to implement a specific function. The acquisition plug-in is a computer application that provides a predetermined acquisition function by interacting with the main application. Optionally, the acquisition plug-in may include an agent plug-in or a plugin plug-in.

[0055] In this embodiment, the collection plug-in is introduced into the business system that executes the code to be tested through the project management tool Maven. When the business system is started, the collection plug-in is loaded through the Spring application launcher, and the code to be tested is parsed through the collection plug-in to collect the execution chain contained in the code to be tested and the information of the execution nodes respectively contained in each execution chain. The information of the execution nodes contained in the execution chain can be the number of execution nodes.

[0056] Optionally, the acquisition plug-in acquires information about the execution chains contained in the code to be tested and the execution nodes respectively contained in each of the execution chains, including:

[0057] Parsing the code to be tested to obtain a syntax tree corresponding to the code to be tested;

[0058] According to the syntax tree, the execution chains contained in the code to be tested and the information of the execution nodes respectively contained in each of the execution chains are parsed.

[0059] The syntax tree is a tree representation of the abstract syntax structure of the code. In the syntax tree, the execution chain can be a path from the root node to the leaf node of the syntax tree, and the execution node in the execution chain can be a node of the syntax tree.

[0060] Specifically, the JavaScript Parser can be used to parse the code to be tested to obtain a syntax tree corresponding to the code to be tested, and the embodiment of the present invention does not limit the parsing tool. The syntax tree is further parsed to obtain the execution chains contained in the code to be tested and the information of the execution nodes respectively contained in each execution chain.

[0061] For example, an example of the code to be tested is given as follows:

[0062]

[0063]

[0064] The program set in the acquisition plug-in is used to parse the sample of the code to be tested, and the syntax tree corresponding to the code to be tested is obtained, such as Figure 3 As shown; the syntax tree is further parsed to obtain the execution chains contained in the syntax tree and the information of the execution nodes respectively contained in each execution chain, such as Figure 4 As shown in the figure, it can be seen intuitively that the syntax tree contains three execution chains in total. Execution chain 1 is composed of execution nodes "type is null value" and execution node "return null"; execution chain 2 is composed of execution nodes "type is not null value", "type is not vip" and "return default value"; execution chain 3 is composed of execution nodes "type is not null value", "type is vip" and "return super user".

[0065] S220: For each of the execution chains, determine the chain weight corresponding to the current execution chain according to the number of execution nodes included in the current execution chain and the total number of execution nodes included in the code to be tested.

[0066] For example, Figure 4 In the syntax tree corresponding to the code to be tested shown, the number of execution nodes contained in execution chain 1 is 2; the number of execution nodes contained in execution chain 2 and execution chain 3 are both 3; the total number of execution nodes contained in the code to be tested is the total number of execution nodes corresponding to the three execution chains, a total of 8 execution nodes. Therefore, according to the chain weight corresponding to the current execution chain being the ratio of the number of execution nodes contained in the current execution chain to the total number of execution nodes contained in the code to be tested, the chain weight corresponding to execution chain 1 is 2 / 8=0.25; the chain weights corresponding to execution chain 2 and execution chain 3 are 3 / 8=0.375.

[0067] S230, determining the execution chains in the code to be tested that are respectively covered by each test case for the code to be tested during the execution process, and obtaining the test case coverage result.

[0068] Specifically, the code to be tested is executed based on multiple different test cases, and the execution chain in the code to be tested covered by each test case during the execution process is determined, that is, the execution path of each side case in the syntax tree of the code to be tested is determined. For example, 10 different test cases may be used to execute the code to be tested, and the execution chain in the code to be tested covered by each test case during the execution process is determined respectively.

[0069] For example, in the code to be tested example provided in the embodiment of the present invention, if test case 1 is "type = Zhang San", and it is known that Zhang San is a VIP, then the execution chain in the code to be tested covered by test case 1 is execution chain 3; if test case 2 is "type = Li Si", and it is known that Li Si is not a VIP user, then the execution chain in the code to be tested covered by test case 2 is execution chain 2. Thus, the test case coverage result is determined as follows: test case 1 covers execution chain 3 in the code to be tested during execution, and test case 2 covers execution chain 2 in the code to be tested during execution.

[0070] S240, for each execution chain in the code to be tested, determine the number of test cases covering the current execution chain according to the test case coverage result, and determine the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases.

[0071] Exemplarily, in the code to be tested example provided in the embodiment of the present invention, if the number of test cases is 10, according to the execution chains in the code to be tested respectively covered by each test case during the execution process represented by the test case coverage result, it is determined that the number of test cases covering the first execution chain is 0, the number of test cases covering the second execution chain is 2, and the number of test cases covering the third execution chain is 8. Accordingly, according to the number of test cases covering the current execution chain and the total number of test cases, it is determined that the use case coverage ratio corresponding to the first execution chain is 0 / 10=0; the use case coverage ratio corresponding to the second execution chain is 2 / 10=0.2; and the use case coverage ratio corresponding to the third execution chain is 8 / 10=0.8.

[0072] S250, generating a determination result of the test coverage rate according to the use case coverage ratio and the chain weight corresponding to each of the execution chains.

[0073] The technical solution of this embodiment obtains the chain proportion corresponding to each execution chain in the code to be tested, determines the execution chains in the code to be tested that are respectively covered by each test case for the tested code during the execution process, and for each execution chain in the code to be tested, determines the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases, and generates a determination result of the test coverage rate according to the use case coverage ratio and chain proportion corresponding to each execution chain, which can evaluate the coverage rate of the test cases in a standard, scientific and quantitative manner, improve the accuracy of the coverage rate evaluation, and thus select more comprehensive test cases according to the evaluation results to ensure the comprehensiveness of the test scope.

[0074] Optionally, obtaining information about execution chains contained in the code to be tested and execution nodes respectively contained in each execution chain collected by a collection plug-in set in the business system includes:

[0075] Reading, from a first data cache unit of the message middleware, information about the execution chains included in the code to be tested and the execution nodes respectively included in each of the execution chains;

[0076] The acquisition plug-in stores the collected execution chains contained in the code to be tested and the information of the execution nodes respectively contained in each execution chain in the first data cache unit in real time.

[0077] Among them, the message middleware can use an efficient and reliable message transmission mechanism to exchange data regardless of the platform, and integrate distributed systems based on data communication. By providing message transmission and message queuing models, it can expand the communication between processes in a distributed environment. In an embodiment of the present invention, the message middleware can be used to store the information of the execution nodes of each test case of the code to be tested during the execution process.

[0078] Exemplarily, the collection plug-in stores the collected execution chains contained in the code to be tested and the information of the execution nodes respectively contained in each execution chain in real time in the first data cache unit of the message middleware. When it is necessary to use the information of the execution nodes to generate a determination result of the test coverage, the information of the execution nodes of each test case of the code to be tested during the execution process is read from the first data cache unit of the message middleware.

[0079] Optionally, a test coverage determination result is generated according to the use case coverage ratio and chain weight corresponding to each of the execution chains, including at least one of the following:

[0080] Generate a chart containing the use case coverage ratio and chain weight corresponding to each of the execution chains, and use the chart as the determination result of the test coverage;

[0081] An execution chain whose use case coverage ratio is less than the chain proportion is determined as an execution chain with insufficient use case coverage, and the execution chain with insufficient use case coverage is used as a determination result of the test coverage rate;

[0082] For each of the execution chains, determine the difference between the use case coverage ratio corresponding to the current execution chain and the chain proportion corresponding to the current execution chain, and use the difference as the determination result of the test coverage rate.

[0083] In a specific example, a chart containing the use case coverage ratio and chain weight corresponding to each of the execution chains is generated, wherein the chart may be in the form of a two-dimensional table, a straight line chart or a pie chart, and the embodiment of the present invention may not limit the form of the chart. Exemplarily, taking the code to be tested provided in this embodiment as an example, the determination result of the test coverage rate may be a table as shown in Table 1, or a line chart as shown in the figure.

[0084] Table 1

[0085] illustrate Chain 1 Chain 2 Chain 3 Chain weight 0.25 0.375 0.375 Use case coverage ratio 0 0.2 0.8

[0086] from Figure 5It can be seen intuitively from the test coverage determination results shown in Table 1 that the use case coverage ratios corresponding to execution chain 1 and execution chain 2 are less than the chain proportions, indicating that the test cases currently used by the code to be tested do not fully cover execution chain 1 and execution chain 2, and the use case coverage ratio corresponding to execution chain 3 is greater than the chain proportion, indicating that the test cases currently used by the code to be tested fully cover execution chain 3. Therefore, according to the test coverage determination results, test cases that can cover execution chain 3 can be added to ensure that the test cases can fully cover each execution chain in the code to be tested, thereby ensuring the comprehensiveness of the test scope.

[0087] In another specific example, if the use case coverage rate corresponding to the current execution chain is greater than or equal to the chain weight, it means that the test case covers the current execution chain of the code to be tested in place, and there is no need to add test cases that can cover the current execution chain. Therefore, the execution chain with a use case coverage ratio less than the chain weight is determined as an execution chain with insufficient use case coverage, and the execution chain with insufficient use case coverage is used as the determination result of the test coverage rate.

[0088] In another specific example, for each of the execution chains, the difference between the use case coverage ratio corresponding to the current execution chain and the chain weight corresponding to the current execution chain is determined, and the difference is used as the result of determining the test coverage rate. It can be seen how large the gap is between the use case coverage ratio corresponding to the current execution chain and the chain weight, which makes it convenient to determine the number of test cases that can cover the current execution chain based on the difference.

[0089] Embodiment 3

[0090] Figure 6 This is a flow chart of a method for determining test coverage in Example 3 of the present invention. This embodiment is optimized based on the above embodiment. In this embodiment, the execution chains in the code to be tested that are respectively covered by each test case for the code to be tested during the execution process are determined to obtain the test case coverage result, including: obtaining information on the execution nodes of each test case for the code to be tested during the execution process collected by a collection plug-in set in the business system; for each test case, the execution chain of the execution node in the code to be tested that contains the current test case during the execution process is determined as the execution chain in the code to be tested covered by the current test case during the execution process to obtain the test case coverage result.

[0091] like Figure 6 As shown, the method of this embodiment specifically includes the following steps:

[0092] S310, obtaining chain proportions corresponding to each execution chain in the code to be tested.

[0093] S320, obtaining information of execution nodes during the execution of each test case of the code to be tested collected by a collection plug-in set in the business system.

[0094] Specifically, the collection plug-in is introduced into the business system that executes the code to be tested through the project management tool Maven. When the business system is started, the collection plug-in is loaded through the Spring application launcher so that the collection plug-in can collect information about the execution nodes of each test case of the code to be tested during the execution process.

[0095] S330, for each of the test cases, determine the execution chain in the code to be tested that contains the execution node of the current test case during the execution process as the execution chain in the code to be tested covered by the current test case during the execution process, and obtain the test case coverage result.

[0096] Specifically, for each test case, the execution nodes of the current test case contained in the code to be tested during the execution process and the execution chain formed by each execution node are determined, and the execution chain is determined as the execution chain in the code to be tested covered by the current test case during the execution process.

[0097] S340, for each execution chain in the code to be tested, determine the number of test cases covering the current execution chain according to the test case coverage result, and determine the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases.

[0098] S350: Generate a test coverage determination result according to the use case coverage ratio and chain weight corresponding to each of the execution chains.

[0099] The technical solution of this embodiment obtains the chain proportion corresponding to each execution chain in the code to be tested, determines the execution chains in the code to be tested that are respectively covered by each test case in the execution process of the code to be tested, and for each execution chain in the code to be tested, determines the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases, and generates a determination result of the test coverage rate according to the use case coverage ratio and chain proportion corresponding to each execution chain, which can evaluate the coverage rate of the test cases in a standard, scientific and quantitative manner, improve the accuracy of the coverage rate evaluation, and thus select more comprehensive test cases according to the evaluation results to ensure the comprehensiveness of the test scope.

[0100] Optionally, the collection plug-in collects information about execution nodes of each test case of the code to be tested during execution, including:

[0101] Reading the embedded code pre-set in the code to be tested, and the code execution information recorded during the execution of each test case for the code to be tested;

[0102] The execution node information of each test case during the execution process is determined according to the read code execution information.

[0103] Among them, data tracking is a commonly used method of data collection, namely event tracking, which is the related technology and its implementation process for capturing, processing and sending specific user behaviors or events.

[0104] Exemplarily, the acquisition plug-in can perform the operation of setting the embedded code at the pre-set execution node in the test code after parsing the execution chain contained in the code to be tested according to the syntax tree, and the execution nodes respectively contained in each execution chain. In the process of executing the code to be tested based on each test case, the execution information of the code is recorded when the embedded code is executed, and the acquisition plug-in determines the information of the execution node of each test case during the execution process by reading the recorded execution information of the code to be tested.

[0105] Optionally, obtaining information about execution nodes during the execution of each test case for the code to be tested collected by a collection plug-in set in the business system includes:

[0106] Read the information of the execution nodes of each test case of the code to be tested during the execution process from the second data cache unit of the message middleware; wherein the collection plug-in stores the collected information of the execution nodes of each test case of the code to be tested during the execution process in the second data cache unit in real time.

[0107] Exemplarily, the information of the execution nodes of each test case of the code to be tested during the execution process is stored in real time through the second data cache unit of the message middleware. When the information of the execution nodes is needed to generate a determination result of the test coverage, the information of the execution nodes of each test case of the code to be tested during the execution process is read from the second data cache unit of the message middleware, which can improve the system response speed and stability of the test coverage determination device.

[0108] For example, Figure 7As shown, the test coverage determination method provided in the embodiment of the present invention can be integrated in a test coverage management system, and the test coverage management system includes a management end, a test case collection end, and a test code parsing and result determination end. Among them, the management end includes a management platform, and the management platform has functions such as business system entry, code maintenance, execution chain management, execution node management, and result display. The collection end includes: a data preprocessing module, a collection plug-in, and a transmission module. The data preprocessing module can perform preprocessing such as embedding and code coloring on the program code or interface code corresponding to the input test case, and determine the code input parameters and output parameters, wherein coloring refers to the process of collecting the information of the execution nodes of each test case of the code to be tested during the execution process. The collection plug-in may include: a Plugin plug-in or an Agent plug-in, which is used to collect the execution chain contained in the code to be tested, and the information of the execution nodes respectively contained in each of the execution chains, and the information of the execution nodes of each test case of the code to be tested during the execution process. The transmission module includes a distributed message transmission platform or a message middleware, which is used to store the execution chains contained in the code to be tested collected by the collection plug-in, as well as the information of the execution nodes respectively contained in each of the execution chains, and the information of the execution nodes of each test case of the code to be tested during the execution process. The test code parsing and result determination end includes: a parsing and calculation module and a storage module, the parsing and calculation module is used to parse the code to be tested to obtain the chain proportions corresponding to each execution chain in the code to be tested; obtain the execution chains in the code to be tested covered by each test case of the code to be tested during the execution process, and each execution chain determines the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases; according to the use case coverage ratio and chain proportion corresponding to each execution chain, the determination result of the test coverage is generated. Among them, the parsing and calculation process can be completed online in real time or offline calculation can be completed through the scheduling module. The storage module can include: any one of SQL database, Redis database or hbase database, which is used to store the determination result of the test coverage. And the process of generating the determination result of the test coverage is monitored in real time through the monitoring segment.

[0109] For example, Figure 8As shown, based on the above-mentioned test coverage management system, the specific steps of the test coverage determination method provided by the embodiment of the present invention include: the management end is used to register or settle in the test coverage management system for the business system, set up a collection plug-in in the business system, collect the execution chain contained in the code to be tested, and the information of the execution nodes respectively contained in each execution chain for the code to be tested through the syntax parser in the collection plug-in set in the business system, and color the test cases through the collection plug-in, that is, collect the information of the execution nodes of each test case of the code to be tested during the execution process. For each of the execution chains, according to the number of execution nodes contained in the current execution chain and the total number of execution nodes contained in the code to be tested, determine the chain proportion corresponding to the current execution chain; determine the execution chains in the code to be tested that are respectively covered by each test case of the code to be tested during the execution process; for each execution chain in the code to be tested, according to the number of test cases covering the current execution chain and the total number of test cases, determine the use case coverage ratio corresponding to the current execution chain; according to the use case coverage ratio and chain proportion corresponding to each execution chain, generate the determination result of the test coverage, and send the determination result to the management end for display.

[0110] Embodiment 4

[0111] Fig. 9 This is a schematic diagram of the structure of a device for determining test coverage provided by Embodiment 4 of the present invention. This embodiment can be applied to the case of determining the test coverage of a test case during the execution of a code to be tested. The device can be implemented in software and / or hardware. The device can be integrated in any device that provides a function of determining test coverage, such as Fig. 9 As shown, the test coverage determination device specifically includes: a chain weight acquisition module 410, an execution chain determination module 420, a coverage ratio determination module 430 and a result determination module 440.

[0112] The chain weight acquisition module 410 is used to acquire the chain weights corresponding to the execution chains in the code to be tested; wherein the chain weight represents the ratio of the number of execution nodes included in the corresponding execution chain to the total number of execution nodes included in the code to be tested;

[0113] An execution chain determination module 420 is used to determine the execution chains in the code to be tested that are respectively covered by each test case of the code to be tested during the execution process, and obtain the test case coverage result;

[0114] A coverage ratio determination module 430 is used to determine, for each execution chain in the code to be tested, the number of test cases covering the current execution chain according to the test case coverage result, and determine the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases;

[0115] The result determination module 440 is used to generate a test coverage determination result according to the use case coverage ratio and chain weight corresponding to each of the execution chains.

[0116] Optionally, the chain specific gravity acquisition module 410 includes:

[0117] A first information acquisition unit is used to acquire the execution chains contained in the code to be tested collected by the collection plug-in set in the business system, and the information of the execution nodes respectively contained in each of the execution chains;

[0118] The weight determination unit is used to determine, for each of the execution chains, the chain weight corresponding to the current execution chain according to the number of execution nodes included in the current execution chain and the total number of execution nodes included in the code to be tested.

[0119] Optionally, the collection plug-in collects the execution chains contained in the code to be tested, and the information of the execution nodes respectively contained in each of the execution chains, including:

[0120] Parsing the code to be tested to obtain a syntax tree corresponding to the code to be tested;

[0121] According to the syntax tree, the execution chains contained in the code to be tested and the information of the execution nodes respectively contained in each of the execution chains are parsed.

[0122] Optionally, the first information acquisition unit is specifically used to:

[0123] Reading, from a first data cache unit of the message middleware, information about the execution chains included in the code to be tested and the execution nodes respectively included in each of the execution chains;

[0124] The acquisition plug-in stores the collected execution chains contained in the code to be tested and the information of the execution nodes respectively contained in each execution chain in the first data cache unit in real time.

[0125] Optionally, the execution chain determination module 420 includes:

[0126] A second information acquisition unit is used to acquire information of execution nodes during the execution of each test case of the code to be tested collected by a collection plug-in set in the business system;

[0127] The execution chain determination unit is used to determine, for each of the test cases, the execution chain of the code to be tested that contains the execution node of the current test case during the execution process as the execution chain in the code to be tested covered by the current test case during the execution process.

[0128] Optionally, the collection plug-in collects information about execution nodes of each test case of the code to be tested during execution, including:

[0129] Reading the embedded code pre-set in the code to be tested, and the code execution information recorded during the execution of each test case for the code to be tested;

[0130] The execution node information of each test case during the execution process is determined according to the read code execution information.

[0131] Optionally, the second information acquisition unit is specifically used to:

[0132] Reading information about execution nodes of each test case during the execution of the code to be tested from a data cache unit of the message middleware;

[0133] The collection plug-in stores the collected information of the execution nodes of each test case of the code to be tested during the execution process in the second data cache unit in real time.

[0134] Optionally, the acquisition plug-in includes an agent plug-in or a plugin plug-in.

[0135] Optionally, the result determination module 440 includes at least one of the following units:

[0136] A first determination unit is used to generate a chart containing the use case coverage ratio and chain weight corresponding to each of the execution chains, and use the chart as a determination result of the test coverage;

[0137] The second determination unit is used to determine the execution chain whose use case coverage ratio is less than the chain weight as an execution chain with insufficient use case coverage, and use the execution chain with insufficient use case coverage as the determination result of the test coverage;

[0138] The third determination unit is used to determine, for each of the execution chains, a difference between a use case coverage ratio corresponding to the current execution chain and a chain weight corresponding to the current execution chain, and use the difference as a determination result of the test coverage.

[0139] The above-mentioned product can execute the test coverage determination method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0140] Embodiment 5

[0141] Fig.10 This is a schematic diagram of the structure of a computer device in Embodiment 4 of the present invention. Fig.10 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Fig.10 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0142] like Fig.10 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processor 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processor 16).

[0143] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0144] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0145] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Fig.10 not shown, usually called a "hard drive"). Although Fig.10 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0146] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0147] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with the computer device 12, and / or communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. In addition, the computer device 12 in this embodiment, the display 24 does not exist as an independent individual, but is embedded in the mirror surface, and when the display surface of the display 24 is not displayed, the display surface of the display 24 and the mirror surface are visually integrated. In addition, the computer device 12 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the computer device 12 through a bus 18. It should be understood that although not shown in the figures, other hardware and / or software modules may be used in conjunction with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0148] The processor 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, to implement the method for determining the test coverage provided in the embodiment of the present invention: obtaining the chain proportion corresponding to each execution chain in the code to be tested; determining the execution chains in the code to be tested that are respectively covered by each test case for the code to be tested during the execution process, and obtaining the test case coverage result; for each execution chain in the code to be tested, determining the number of test cases covering the current execution chain according to the test case coverage result, and determining the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases; generating the test coverage determination result according to the use case coverage ratio and chain proportion corresponding to each of the execution chains.

[0149] Embodiment 6

[0150] Embodiment 6 of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for determining test coverage as provided in all embodiments of the invention of the present application: obtaining the chain proportion corresponding to each execution chain in the code to be tested; determining the execution chains in the code to be tested that are respectively covered by each test case for the code to be tested during the execution process, and obtaining a test case coverage result; for each execution chain in the code to be tested, determining the number of test cases covering the current execution chain according to the test case coverage result, and determining the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases; generating a test coverage determination result according to the use case coverage ratio and the chain proportion corresponding to each of the execution chains.

[0151] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0152] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0153] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0154] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0155] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for determining test coverage, characterized in that: The method comprises: Obtain the chain weight corresponding to each execution chain in the code to be tested; wherein the chain weight represents the ratio of the number of execution nodes contained in the corresponding execution chain to the total number of execution nodes contained in the code to be tested; wherein the execution chain refers to the path from the root node to the leaf node of the syntax tree corresponding to the code to be tested; Determine the execution chains in the code to be tested that are respectively covered by each test case for the code to be tested during the execution process, and obtain the test case coverage result; For each execution chain in the code to be tested, determine the number of test cases covering the current execution chain according to the test case coverage result, and determine the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases; According to the use case coverage ratio and chain weight respectively corresponding to each of the execution chains, a determination result for evaluating the test coverage of the test case is generated.

2. The method according to claim 1, characterized in that Get the chain weights corresponding to each execution chain in the code to be tested, including: Acquire the execution chains contained in the code to be tested collected by the collection plug-in set in the business system, and the information of the execution nodes respectively contained in each of the execution chains; For each of the execution chains, the chain weight corresponding to the current execution chain is determined according to the number of execution nodes included in the current execution chain and the total number of execution nodes included in the code to be tested.

3. The method according to claim 2, characterized in that The acquisition plug-in acquires information about the execution chains contained in the code to be tested and the execution nodes respectively contained in each of the execution chains, including: Parsing the code to be tested to obtain a syntax tree corresponding to the code to be tested; According to the syntax tree, the execution chains contained in the code to be tested and the information of the execution nodes respectively contained in each of the execution chains are parsed.

4. The method according to claim 2, characterized in that: Acquiring information about the execution chains contained in the code to be tested and the execution nodes respectively contained in each of the execution chains collected by the collection plug-in set in the business system, including: Reading, from a first data cache unit of the message middleware, information about the execution chains included in the code to be tested and the execution nodes respectively included in each of the execution chains; The acquisition plug-in stores the collected execution chains contained in the code to be tested and the information of the execution nodes respectively contained in each execution chain in the first data cache unit in real time.

5. The method according to claim 1, characterized in that Determining the execution chains in the code to be tested that are respectively covered by each test case for the code to be tested during execution, and obtaining test case coverage results, including: Acquire information of execution nodes of each test case during the execution of the code to be tested collected by a collection plug-in set in the business system; For each of the test cases, the execution chain of the code to be tested that contains the execution node of the current test case during the execution process is determined as the execution chain in the code to be tested covered by the current test case during the execution process, and the test case coverage result is obtained.

6. The method according to claim 5, characterized in that The collection plug-in collects information about execution nodes of each test case of the code to be tested during execution, including: Reading the embedded code pre-set in the code to be tested, and the code execution information recorded during the execution of each test case for the code to be tested; The execution node information of each test case during the execution process is determined according to the read code execution information.

7. The method according to claim 5, characterized in that Obtaining information about execution nodes of each test case during the execution of the code to be tested collected by a collection plug-in set in the business system, including: Reading information about execution nodes of each test case during the execution of the code to be tested from a second data cache unit of the message middleware; The collection plug-in stores the collected information of the execution nodes of each test case of the code to be tested during the execution process in the second data cache unit in real time.

8. The method according to any one of claims 2 to 7, characterized in that: The acquisition plug-in includes an agent plug-in or a plugin plug-in.

9. The method according to any one of claims 1 to 7, characterized in that: According to the use case coverage ratio and chain weight corresponding to each of the execution chains, a test coverage determination result is generated, including at least one of the following: Generate a chart containing the use case coverage ratio and chain weight corresponding to each of the execution chains, and use the chart as the determination result of the test coverage; An execution chain whose use case coverage ratio is less than the chain proportion is determined as an execution chain with insufficient use case coverage, and the execution chain with insufficient use case coverage is used as a determination result of the test coverage rate; For each of the execution chains, determine the difference between the use case coverage ratio corresponding to the current execution chain and the chain proportion corresponding to the current execution chain, and use the difference as the determination result of the test coverage rate.

10. A device for determining test coverage, characterized in that: The device comprises: A chain weight acquisition module, used to obtain the chain weights corresponding to each execution chain in the code to be tested; wherein the chain weight represents the ratio of the number of execution nodes contained in the corresponding execution chain to the total number of execution nodes contained in the code to be tested; wherein the execution chain refers to the path from the root node to the leaf node of the syntax tree corresponding to the code to be tested; An execution chain determination module is used to determine the execution chains in the code to be tested that are respectively covered by each test case of the code to be tested during the execution process, and obtain the test case coverage result; A coverage ratio determination module, used to determine, for each execution chain in the code to be tested, the number of test cases covering the current execution chain according to the test case coverage result, and determine the use case coverage ratio corresponding to the current execution chain according to the number of test cases covering the current execution chain and the total number of test cases; The result determination module is used to generate a determination result for evaluating the test coverage of the test case according to the use case coverage ratio and the chain weight corresponding to each of the execution chains.

11. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for determining the test coverage as described in any one of claims 1-9 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for determining the test coverage as described in any one of claims 1 to 9 is implemented.

Images