Test method, terminal device and computer readable storage medium

Abstract

The present application is applicable to the field of computer testing technology, and provides a testing method, terminal device and computer-readable storage medium, which obtains first coverage information of a first test code of a first version, obtains second coverage information of a second test code of a second version, and determines third coverage information according to the first coverage information and the second coverage information, wherein the third coverage information indicates the ratio of the number of tested code lines in the second code line to the total number of the second code lines, and the second code line is the code line where the first test code changes relative to the fourth test code of the initial version. The above method can accurately reflect the specific coverage information of the changed code during the testing process.

Description

Technical Field

[0001] The present application belongs to the field of computer testing technology, and in particular, relates to an incremental code coverage analysis method and terminal device. Background Art

[0002] Incremental code coverage is an important indicator in the software testing process. It is used to measure the proportion of code covered when a test case is executed. Calculating incremental code coverage is an effective means to improve software quality.

[0003] In the related technology, all preset test cases are usually executed based on a fixed code version to ensure that all code paths, branches and statements in the code are covered. However, in the actual development and testing process, there are situations where developers change part of the code. The above method cannot accurately reflect the specific coverage information of the code for the required changes. Summary of the invention

[0004] The embodiments of the present application provide a testing method, a terminal device, and a computer-readable storage medium, which can accurately reflect the specific coverage information of the changed code during the testing process.

[0005] In a first aspect, an embodiment of the present application provides a testing method, comprising:

[0006] Obtaining first coverage information of a first test code of a first version, where the first coverage information indicates a ratio of the number of tested code lines to the total number of code lines in the first test code;

[0007] Obtaining second coverage information of the second test code of the second version; wherein the second coverage information represents the ratio of the number of tested code lines in the first code line to the total number of the first code lines, the first code line being the code line in which the second test code changes relative to the third test code of the third version; the second version is the previous version of the first version, and the third version is the previous version of the second version;

[0008] The third coverage information is determined based on the first coverage information and the second coverage information, wherein the third coverage information represents the ratio of the number of tested code lines in the second code lines to the total number of second code lines, and the second code lines are code lines where the first test code changes relative to the fourth test code of the initial version.

[0009] In an embodiment of the present application, the test code coverage information (first coverage information) of the current test version is obtained, and then the coverage information (second coverage information) of the incremental code corresponding to the historical test code (second test code) is obtained, and the coverage information (third coverage information) corresponding to the changed code is calculated based on the above two types of coverage information. In this way, it is possible to analyze which lines of code have been tested in various versions in history, and which lines of code are newly added or modified. In this way, the specific coverage information of the changed code can be accurately obtained, which helps to improve the testing quality of the software.

[0010] In a possible implementation manner of the first aspect, obtaining first coverage information of a first test code of a first version includes:

[0011] Obtaining a first code execution trace corresponding to the first test code, wherein the first code execution trace represents an execution order of code lines in the first test code;

[0012] First coverage information of a first test code of a first version is obtained according to the first code execution trace.

[0013] In an embodiment of the present application, by obtaining coverage information from the execution trace, it is possible to more accurately calculate which codes are covered by the test and their execution order, thereby obtaining a more accurate test coverage rate.

[0014] In a possible implementation manner of the first aspect, obtaining a first code execution trace corresponding to the first test code includes:

[0015] Obtaining a second code execution trace, wherein the second code execution trace is an execution trace of a machine code obtained by performing a stub test in a machine code corresponding to the first test code;

[0016] The first code execution trace corresponding to the first test code is obtained according to the second code execution trace.

[0017] In an embodiment of the present application, the machine code execution trace obtained through the stub test can more accurately reflect the execution status of the test code, thereby improving the accuracy and completeness of the coverage.

[0018] In a possible implementation manner of the first aspect, obtaining second coverage information of a second test code of a second version includes:

[0019] Calculate a first hash value corresponding to each code line in the second test code;

[0020] Calculate a second hash value corresponding to each code line in the third test code;

[0021] Second coverage information of the second test code of the second version is obtained according to each first hash value and each second hash value.

[0022] In the embodiment of the present application, by calculating the hash value, different code lines can be quickly compared, thereby improving the efficiency of obtaining coverage information.

[0023] In a possible implementation manner of the first aspect, determining the third coverage information according to the first coverage information and the second coverage information includes:

[0024] Determine whether there is a code running record corresponding to the first test code in the first database;

[0025] If there is a code running record corresponding to the first test code in the first database, obtain fourth coverage information corresponding to the running record, the fourth coverage information indicating the ratio of the number of tested code lines in the third code line to the total number of code lines, wherein the third code is a code line of the fifth test code that has changed relative to the second test code of the second version, wherein the fifth test code is a code line that has been tested when an interruption occurs during the testing of the first test code;

[0026] Merging the first coverage information corresponding to the first test code with the fourth coverage information corresponding to the code running record to obtain fifth coverage information;

[0027] The third coverage information is determined according to the second coverage information and the fifth coverage information.

[0028] In an embodiment of the present application, by merging different coverage information, a more accurate test coverage result can be obtained, which helps to more comprehensively understand the execution status of the test code.

[0029] In a possible implementation manner of the first aspect, the method further includes:

[0030] During the testing of the first test code, if the first test code is interrupted, obtaining the running record of the third code;

[0031] The running record of the third code line is saved in the first database.

[0032] In an embodiment of the present application, when a test is interrupted, saving the running record can generate a more detailed test report, which includes the code lines that have been executed when the test is interrupted, which helps the tester analyze the test coverage and test completeness.

[0033] In a possible implementation manner of the first aspect, determining the third coverage information according to the first coverage information and the second coverage information includes:

[0034] Determine a first value according to the first coverage information and the second coverage information, the first value being the sum of the number of tested code lines in the first test code and the number of tested code lines in the first code line;

[0035] Determine a second value according to the first value and the number of the first code lines, the second value indicating the number of the first code lines included in the fourth code;

[0036] The third coverage information is determined according to the second value and the number of the first code lines.

[0037] In a possible implementation manner of the first aspect, the method further includes:

[0038] generating a test report according to the third coverage information;

[0039] Generate a download address based on the test report;

[0040] Return the download address to the client.

[0041] In the embodiment of the present application, by automatically generating test reports and providing them for download, test results are easier to obtain and share, which helps to improve the efficiency of software development and testing.

[0042] In a second aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a test method as described in any one of the first aspects above is implemented.

[0043] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the test method as described in any one of the first aspects above.

[0044] In a fourth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any one of the test methods of the first aspect.

[0045] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0047] Figure 1 is a schematic diagram of a test system provided in an embodiment of the present application;

[0048] Figure 2 It is a flow chart of the testing method provided in the embodiment of the present application;

[0049] Figure 3 This is a schematic diagram of the process of obtaining coverage information provided in the embodiment of the present application. Figure 1 ;

[0050] Figure 4 This is a schematic diagram of the process of obtaining coverage information provided in the embodiment of the present application. Figure 2 ;

[0051] Figure 5 It is a structural schematic diagram of the testing method provided in the embodiment of the present application;

[0052] Figure 6 It is a schematic diagram of the test process of the test method provided by the present application;

[0053] Figure 7 It is a structural diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0055] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0056] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0057] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0058] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0059] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0060] Incremental code coverage is an important indicator in the software testing process. It is used to measure the proportion of code covered when a test case is executed. Calculating incremental code coverage is an effective means to improve software quality.

[0061] In the related art, all preset test cases are usually executed based on a fixed code version to ensure that all code paths, branches and statements in the code are covered. However, in the actual development and testing process, there are cases where developers change part of the code, and the above method cannot accurately reflect the specific coverage information of the code required to change. Moreover, after the developer changes the code, the tester re-executes the code coverage executed before the change, which is inefficient.

[0062] In order to solve the problems in the above-mentioned related technologies, the present application provides a testing system, such as Figure 1 , is a schematic diagram of the test system provided in an embodiment of the present application, including a business service end and a line coverage platform, wherein the business service end is a terminal device, i.e., a device that executes the test code, and the line coverage platform is used to calculate the line coverage based on the data of the business service end.

[0063] The present application proposes a testing method using a testing system, which uses bytecode technology to logically insert stubs into business service codes in a business service end, i.e., a terminal device, such as code jumps, returns, exception throwing, etc., and then saves the inserted execution traces into memory when the code is executed. At the same time, the business service end will automatically allocate a transmission service, through which the port and address information will be reported to the code coverage backend, and a callback will be made before the registered business service stops, and the code execution trace insertion information will be checked in memory, and this information will be reported to the backend service of the line coverage platform, which can realize code trace collection through the code coverage backend service, and realize multi-version incremental code coverage analysis and calculation and subsequent code line coverage report generation through the service source code.

[0064] Among them, the transmission service is a framework for implementing TCP communication in Java. Since communication with the transmission service requires port and address information, the address and port information will be reported to the coverage backend service at startup. The coverage backend service will maintain the connection information in the database. For example, when exiting, the status will be updated to active and when going online, the status will be updated to invalid. However, there are also abnormal situations. When the coverage frontend service terminates abnormally, it exits without calling the callback before stopping. At this time, the status stored in the coverage backend service is active, but the status of the coverage frontend service is actually invalid, resulting in a timeout and resource consumption when establishing a connection between the coverage backend service and the coverage frontend service. In order to solve this problem, before the coverage backend service connects to the coverage frontend service, the coverage backend service will detect whether the coverage frontend address and port are available. If not, the process will be terminated without executing the connection establishment and subsequent code extraction and execution trajectory operations. At the same time, since it is necessary to set a port to establish a transmission service communication server in Java, if a fixed port is used, there is a problem of not being able to determine whether it will conflict with other existing ports. Therefore, when establishing a transmission service communication server on the coverage front end, the strategy of not setting a fixed port and checking the current free port is adopted to avoid the port conflict problem caused by a fixed port.

[0065] See also Figure 2 , is a flow chart of the test method provided in the embodiment of the present application. As an example but not a limitation, the method may include the following steps:

[0066] S101, obtaining first coverage information of a first test code of a first version, where the first coverage information indicates a ratio of the number of tested code lines to the total number of code lines in the first test code.

[0067] In an embodiment of the present application, the first version is the current version of the software, that is, the latest or tested version. In the current version (that is, the first version) of a software project, a test code (first test code) is executed once, and then the number of code lines covered by this test is calculated. The number of executed code lines is divided by the total number of code lines to obtain a ratio value, which is the first coverage information. Based on this ratio value, the adequacy of the test can be evaluated. If this value is close to 1, it means that the test is relatively sufficient. If this value is very small, it means that more tests may need to be supplemented.

[0068] The above-mentioned “first test code” refers to the first test case or the first batch of test cases written for the current version.

[0069] In one embodiment, see Figure 3 , is a schematic diagram of the process of obtaining the first coverage information provided in the embodiment of the present application Figure 1 , step S101 includes:

[0070] S201, obtaining a first code execution trace corresponding to a first test code, wherein the first code execution trace represents an execution order of code lines in the first test code.

[0071] In the embodiment of the present application, obtaining the first code execution trace corresponding to the first test code means recording and analyzing the execution order of the first test code. This usually involves tracking the execution flow of the code, including which lines of code are executed and the order in which they are executed. The purpose of the first code execution trace is to help developers understand how the test code is run and whether it is executed in the expected order. This is crucial for debugging and ensuring the accuracy of the test.

[0072] In one embodiment, step S201 may be implemented as follows:

[0073] A second code execution trace is obtained, wherein the second code execution trace is an execution trace of a machine code obtained by performing a stub test in a machine code corresponding to the first test code; and a first code execution trace corresponding to the first test code is obtained according to the second code execution trace.

[0074] In the embodiment of the present application, during the software testing and debugging process, it is sometimes necessary to obtain more detailed code execution information, which can be achieved through instrumentation technology. Instrumentation is a technology that can add new code to record execution information, measure performance or perform other monitoring tasks without modifying the original code. The second code execution trace is the execution trace at the machine code level generated in the original code (first test code). This usually involves inserting special monitoring code (i.e., instrumentation code) into the original code, which will record the execution of each machine instruction at runtime.

[0075] Specifically, the business service code can be logically inserted through bytecode technology, such as code jumps, returns, exceptions thrown, etc., and then the insertion execution information (second code execution trace) is saved to the memory when the code is executed. Inserting business service code through bytecode technology is a method of adding additional functions without modifying the source code. This technology is used in performance testing, monitoring, debugging, and security checking. Bytecode insertion can insert specific bytecodes at key points in the execution path of the second code (the machine code corresponding to the first test code) to monitor and control code execution.

[0076] After the execution trace (second code trace) of the machine code after the instrumentation is saved in the memory of the terminal device, the second code trace can be obtained by establishing a communication connection with the terminal device, and the execution trace corresponding to the first code can be pulled according to the second code trace.

[0077] In the above method, the machine code execution trace obtained through the stub test can more accurately reflect the execution status of the test code, thereby improving the accuracy and completeness of the coverage.

[0078] S202: Obtain first coverage information of a first test code of a first version according to a first code execution trace.

[0079] In an embodiment of the present application, the first code trace can be used to see the execution flow of the code, that is, which code blocks are passed from the beginning to the end of the program, that is, the proportion of tested code lines in the first test code can be analyzed (first coverage information).

[0080] Specifically, after the execution trace of the source code (first test code) is pulled according to the second code execution trace, the source code is built into a class file recognizable by the Java virtual machine through the build command. This is a file format that can be executed by the Java virtual machine. The class file contains the bytecode compiled from the source code (first test code). According to the first execution trace, the first source code and the class file, it is possible to compare and determine which source code lines are covered by the test code. Line coverage information refers to which source code lines are executed by the test code and which are not executed. The number of executed code lines is divided by the total number of code lines to obtain a ratio value, which is the first coverage information.

[0081] It should be noted that after the first build, the Class files and source code files will be saved on the disk, so the next time the same process is triggered, the code copying and building process can be skipped.

[0082] In the above method, by obtaining coverage information from the execution trace, it is possible to more accurately calculate which codes are covered by the test and their execution order, thereby obtaining a more accurate test coverage rate.

[0083] S102, obtaining second coverage information of the second test code of the second version; wherein the second coverage information represents the ratio of the number of tested code lines in the first code lines to the total number of the first code lines, the first code lines being the code lines where the second test code has changed relative to the third test code of the third version; the second version is the previous version of the first version, and the third version is the previous version of the second version.

[0084] In one embodiment, the second version refers to the previous version of the current version, and the third version is the previous version of the second version. Each version usually contains a series of changes and fixes, so the code may need to be modified multiple times. The second coverage information refers to the coverage information about the second version code obtained by executing the second test code. Here, the second coverage information pays special attention to the first code lines that have changed in the second version relative to the third version.

[0085] Specifically, it is necessary to compare the code differences between the second and third versions and analyze the coverage of these differences in the test code of version 3. This can help developers understand whether the changes made in the second version have been fully tested and whether there are new lines of code that need to be added to the test code.

[0086] In one embodiment, step S102 includes:

[0087] Calculate a first hash value corresponding to each code line in the second test code; calculate a second hash value corresponding to each code line in the third test code; and obtain second coverage information of the second test code of the second version according to each first hash value and each second hash value.

[0088] In an embodiment of the present application, a cryptographic hash function is used to calculate a hash value of a specific Java source file (test code) covered by the source code execution, and the hash value is compared with the incremental code change line hash value to obtain the incremental code coverage line information (second coverage information). In this process, a cryptographic hash function (such as SHA-256) is used to calculate the hash value of each line of code. A cryptographic hash function is a function that can convert data of any length into fixed-length data, and this process is one-way, that is, a hash value can be obtained from the original data, but the original data cannot be deduced from the hash value. When the code is changed (i.e., incremental code changes), the hash value of the changed line is recalculated. Then, the new hash value is compared with the previous hash value to determine which lines have been modified. By comparing the coverage obtained by executing the coverage test twice, it can be determined which lines are newly executed due to code changes and which lines are no longer executed due to code changes, thereby obtaining the coverage line information (second coverage information) of the incremental code.

[0089] In one implementation, when there are only two versions in the historical version of the current version, namely the second version and the third version, for each line in the second test code of the second version, a hash function (such as SHA-256) is used to calculate its hash value (first hash value), and this hash value will be used as the unique identifier of the code line. For each line in the third test code of the third version, the same hash function is used to calculate its hash value (second hash value), so that each code line also has a corresponding hash value in the third version, and the first hash value of each line of the second test code of the second version is compared with the second hash value of each line of the third test code of the third version. If the two hash values ​​are the same, it means that the corresponding code line is consistent in the two test codes and has not been modified; if they are different, it means that the code line has been modified. According to the comparison result, the code line information of the second version of the test code relative to the third version of the test code can be obtained, which is usually expressed as a ratio or a detailed report, showing which code lines are executed in the incremental test and which are not executed.

[0090] In another implementation, when there are multiple historical versions in the historical version of the current version, it is necessary to calculate the coverage information of each version with the previous version (except the current version). The method is the same as the above-mentioned method for obtaining the second coverage information, and will not be repeated here.

[0091] In the above method, by calculating the hash value, different code lines can be quickly compared, thereby improving the efficiency of obtaining coverage information.

[0092] S103, determining third coverage information based on the first coverage information and the second coverage information, wherein the third coverage information represents the ratio of the number of tested code lines in the second code lines to the total number of second code lines, and the second code lines are code lines where the first test code changes relative to the fourth test code of the initial version.

[0093] In the embodiment of our application, the third coverage information reflects the test coverage of the code lines in the first test code of the current version that have changed relative to the fourth test code of the initial version. The "second code lines" here refer to the code lines that have been modified or added in the first test code of the current version compared to the fourth test code of the initial version. In this way, through the third coverage information, it is possible to understand which parts of the test are sufficient after the code changes, and which parts may need further testing or attention.

[0094] Among them, the total number of the second code lines, that is, the code lines that have changed in the first test code relative to the fourth test code of the initial version, is obtained by querying the code library to compare the current branch (first test code) with the fourth test code of the initial version to obtain the code change information. The code change information includes the class permission name, the added line number, and the specific content of the line. Only the added lines are counted because the source code no longer exists after being deleted, and it is impossible to count from the code track, so only the added lines are counted. And based on the code added line information and all the logical lines of the current version of the source code, that is, all the code logic lines that can be covered by the code track are counted through the stub algorithm, and the intersection between the two is taken to get the total number of changed lines (second code lines).

[0095] In one embodiment, step S103 includes:

[0096] S301, determining whether there is a code running record corresponding to the first test code in the first database.

[0097] In an embodiment of the present application, the system queries the first database to find the code execution record that is consistent with the current terminal device ID and the code library branch (the first test code). This is because during the execution of the first test code, the test process may be interrupted due to automatic exit of the tested software, etc. In the event of an interruption, the trace of the first test code that has been executed will be stored in the first database.

[0098] S302. If there is a code execution record corresponding to the first test code in the first database, obtain the fourth coverage information corresponding to the execution record, wherein the fourth coverage information indicates the ratio of the number of tested code lines in the third code lines to the total number of third code lines, wherein the third code is the code line of the fifth test code that has changed relative to the second test code of the second version, wherein the fifth test code is the code line that has been tested when an interruption occurs during the testing of the first test code.

[0099] In the embodiment of the present application, if the query finds that there is a running record corresponding to the first test code, it means that the test code has been executed before and may have generated coverage information (fourth coverage information), and the system obtains the fourth coverage information from the query running record. This coverage information is for the coverage of the code (fifth test code) that has been tested when an interruption occurs during the test of the first test code.

[0100] For example, if the number of first test code lines is 100, and a test interruption occurs for some reason when testing the first test code in the record, the system detects that 40 lines of code have been run at the time of the interruption (the fourth test code), and 10 lines of code have been changed between the first test code and the second test code (the third code lines), and it is detected that 3 lines of code have been changed among the 40 lines of code that have been run, then 3 / 10 is the fourth coverage information.

[0101] If the code running record information corresponding to the current version is not found, the currently obtained third coverage information is saved in the first database.

[0102] S303: Merge the first coverage information corresponding to the first test code and the fourth coverage information corresponding to the code execution record to obtain fifth coverage information.

[0103] In the embodiment of the present application, when the code version recorded in the first database is the same as the current code version being processed, in this case, the code coverage line information recorded in the database (the fourth coverage information) is merged with the current code coverage line information (the first coverage information), and merging this information means combining the coverage results of the two codes to obtain the fifth coverage information, so as to more accurately reflect the actual coverage of the code.

[0104] S304: Determine the third coverage information based on the second coverage information and the fifth coverage information.

[0105] In an embodiment of the present application, the total incremental code coverage information (third coverage information) is calculated using the merged fifth coverage information and the incremental coverage information (second coverage information) of different historical versions.

[0106] In the above method, by merging different coverage information, more accurate test coverage results can be obtained, which helps to understand the execution of the test code more comprehensively.

[0107] In one embodiment, the testing method further comprises:

[0108] During the process of testing the first test code, if the test of the first test code is interrupted, the running record of the third code is obtained; and the running record of the third code line is saved in the first database.

[0109] In an embodiment of the present application, when testing the first test code corresponding to the current version, when the test is interrupted, it is necessary to obtain the code running record at the moment of the test interruption. The "third code" here refers to the code line or code segment that has been run, and its running record needs to be specially recorded when the test is interrupted. The obtained running record of the third code line needs to be saved for subsequent analysis, debugging or coverage calculation. These records will be stored in the first database, which is used to store information about the execution of the test code.

[0110] In the above method, when the test is interrupted, saving the running record can generate a more detailed test report, which includes the code lines that have been executed when the test is interrupted, which helps the tester analyze the test coverage and test completeness.

[0111] In one embodiment, step S103 further includes:

[0112] A first value is determined based on the first coverage information and the second coverage information, the first value being the sum of the number of tested code lines in the first test code and the number of tested code lines in the first code line; a second value is determined based on the first value and the number of the second code lines, the second value indicating the number of the second code lines included in the first value; and third coverage information is determined based on the second value and the number of the second code lines.

[0113] In an embodiment of the present application, the coverage information of the current version (first coverage information) needs to be merged with the incremental coverage information of the historical version found in the first database (second coverage information). This means that the coverage information of different versions will be unified into a total coverage information. After merging the coverage information of different versions, a total number of covered lines will be obtained. This number represents the total number of lines of code that have been tested and executed in all versions (first value). Next, the intersection of the total number of covered lines and the number of lines of code that have been changed (added) this time needs to be calculated to obtain the second value. This is to determine which newly added lines of code have been covered by the test. Finally, the ratio between the second value and the second line of code is calculated to obtain the third coverage information.

[0114] See also Figure 4, is a schematic diagram of the process of obtaining the third coverage information provided in the embodiment of the present application Figure 2 ,like Figure 4 As shown, the steps of obtaining the third coverage information are:

[0115] 1. Query the code line coverage information (second coverage information) calculated from the current historical code versions A, B, and C in the memory through the business service address and code library branch;

[0116] 2. Query the code library to obtain the code change information by comparing the current version with the initial version (the default is the master branch), and based on the code addition line information and all the logical lines of the current version source code, that is, all the code logical lines that can be covered by the code track are counted through the instrumentation algorithm, and the intersection between the two is taken to get the total number of changed lines as the denominator (the second code).

[0117] 3. According to the code trajectory pulled by the terminal device, combined with the source code copied from the code library and the class file after the source code is built, the current version line coverage information (first coverage information) is analyzed, and then combined with the code line coverage information calculated by querying the current historical code versions A, B, and C in the memory to obtain the total number of coverage lines, and then the total coverage lines and the added lines are intersected to obtain the multi-version incremental code coverage line information (third coverage information).

[0118] For example, Figure 4 As shown in , if the number of code lines corresponding to the initial version is 100 lines, and the number of code lines of the current version D saved for the last time is 120 lines, by comparison, it can be seen that the number of code lines added is 20 lines. If the number of logical code lines corresponding to the current version D is 100 lines, the intersection of the added 20 lines of code and the logical line code is taken to obtain the total number of changed lines. If the number of changed logical line code is 15 lines, the total number of changed lines is 15 lines, which is the second code line in the above step S103.

[0119] Get the full coverage line number corresponding to the current version D. If the full coverage line number is 9, it is the first coverage information. Then get the incremental coverage information corresponding to multiple historical versions from the first database, that is, the second coverage information. If the coverage line numbers corresponding to the historical versions A, B, and C are 8, 9, and 7 respectively, then merge the first coverage information of the current version D and the second coverage information corresponding to the multiple historical versions A, B, and C to obtain the total coverage information. For example, if the total coverage line number obtained by merging is 10 lines, finally take the intersection of the total coverage line number and the total number of changed lines obtained above to obtain the incremental coverage information. If the incremental coverage information is 9 lines through calculation, then the line number coverage information in the incremental code can be calculated based on the total number of changed lines of 15 lines (the third coverage information), that is, 9 / 15=60% (the third coverage information).

[0120] In one embodiment, the method further comprises:

[0121] Generate a test report according to the third coverage information; generate a download address according to the test report; and return the download address to the client.

[0122] In an embodiment of the present application, the third coverage information refers to the previously calculated coverage line information, which may be based on multiple versions of code and test results. Based on this coverage information, the system will generate a detailed test report. This report will usually contain information such as coverage data, uncovered lines of code, test pass rate, etc. The generated test report needs to be placed in an accessible location, usually a server or file storage system, such as distributed object storage (XCFS storage). This process will generate a download address for the test report, and finally, the download address will be sent back to the client.

[0123] In the above method, by automatically generating test reports and providing them for download, test results are made easier to obtain and share, which helps to improve the efficiency of software development and testing.

[0124] See also Figure 5 , is a schematic diagram of the structure of the test method provided in the embodiment of the present application, such as Figure 5 The following figure shows the operation content and data interaction mode of the code coverage backend service and code coverage service. The code coverage backend service and code coverage frontend service can be deployed on multiple network nodes. The coverage backend service and code coverage frontend service between nodes communicate through the HTTP protocol network. The code coverage frontend is mainly used for business code logic instrumentation, business code track statistics, business code track reporting, etc. The code coverage backend service is mainly used for business code track collection, business code track preservation, business code pulling, business code construction, business code coverage calculation, code coverage report generation, code coverage report coloring, etc.

[0125] Most of the data of the code coverage backend service during operation is stored in the database, and the business service source code and build products are saved on the disk, and the code line coverage details and the final code line coverage report are stored in XCFS. The code coverage frontend service stores the code execution track in memory. Among them, XCFS is a distributed storage service that stores massive files. Users can store and view data at any time through the network. Since the code line coverage details and the final code line coverage report are large in size and not suitable for storage in memory, the data is saved on XCFS, and the download address is saved in memory, giving full play to the storage capacity of XCFS for massive data.

[0126] The present invention downloads the business service source code to the local computer before calculating and analyzing the code line coverage, and constructs the source code into a class file recognizable by the virtual machine through a build command. However, after the first build, the class file and the source code file will be saved on the disk, so the next time the same process is triggered, the process of copying the code and building can be skipped, which greatly improves the efficiency of calculating and analyzing the code line coverage and reduces the use of resources. However, since the code coverage backend service adopts a distributed high-availability deployment, and different instances have different disk spaces and cannot be shared with each other, the network file system is used to realize space sharing between different instances.

[0127] See also Figure 6 , is a schematic diagram of the test flow of the test method provided by the present application, such as Figure 6 As shown, in the coverage front-end service, the business code logic is plugged, the transmission service is started and the address and port information are reported, the callback before registration is stopped, and the business code execution trace is collected and stored in the memory. The user triggers the code line coverage back-end service to generate a report. The code line coverage back-end service will pull the code execution trace from the coverage front-end service through the connection information reported above, and then determine whether there is a source code build product on the local disk. If not, it will be pulled and built from the remote code repository. Then the code line coverage calculation is performed, the code line coverage report is generated, and the download address is returned to the user by uploading it to XCFS.

[0128] In the above method, the tester only needs to trigger the entire process to obtain a multi-version incremental code line coverage report during the entire demand process of the business service. The report clearly displays the tested code logic and untested code logic of the tester in the test process by dyeing. The present invention analyzes the incremental code coverage, which makes the tester's test of the business code logic more intuitive, and has low technical requirements for operators; at the same time, by integrating the multi-version code coverage, the tester only needs to continue testing the changed code after the code is changed, which improves the efficiency of the overall test. The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other and are not used to limit the protection scope of this application. The specific working process of the units and modules in the above system can refer to the corresponding process in the above method embodiment, which will not be repeated here.

[0129] Figure 7 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Figure 7 As shown, the terminal device 7 of this embodiment includes: at least one processor 70 ( Figure 7 Only one is shown in the figure) a processor, a memory 71, and a computer program 72 stored in the memory 71 and executable on at least one processor 70, and when the processor 70 executes the computer program 72, the steps in any of the above-mentioned test method embodiments are implemented.

[0130] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 7 It is only an example of the terminal device 7 and does not constitute a limitation on the terminal device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0131] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0132] In some embodiments, the memory 71 may be an internal storage unit of the terminal device 7, such as a hard disk or memory of the terminal device 7. In other embodiments, the memory 71 may also be an external storage device of the terminal device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the terminal device 7. The memory 71 is used to store an operating system, an application program, a boot loader, data, and other programs, such as program codes of a computer program. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0133] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0134] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0135] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0136] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0137] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0138] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A testing method, characterized in that , the method comprises: Obtaining first coverage information of a first test code of a first version, where the first coverage information indicates a ratio of the number of tested code lines to the total number of code lines in the first test code; Obtaining second coverage information of a second test code of a second version; wherein the second coverage information represents a ratio of the number of tested code lines in a first code line to the total number of the first code lines, the first code lines being code lines in which the second test code changes relative to the third test code of a third version; the second version is a previous version of the first version, and the third version is a previous version of the second version; Determine third coverage information according to the first coverage information and the second coverage information, wherein the third coverage information represents a ratio of the number of tested code lines in the second code lines to the total number of the second code lines, and the second code lines are code lines in which the first test code changes relative to the fourth test code of the initial version; The determining the third coverage information according to the first coverage information and the second coverage information includes: Determine a first value according to the first coverage information and the second coverage information, where the first value is the sum of the number of tested code lines in the first test code and the number of tested code lines in the first code lines; determining a second value according to the first value and the number of the first code lines, wherein the second value indicates the number of the first code lines included in the fourth test code; determining the third coverage information according to the second value and the number of the first code lines; The determining the third coverage information according to the first coverage information and the second coverage information includes: Determine whether there is a code running record corresponding to the first test code in the first database; If there is a code running record corresponding to the first test code in the first database, obtain fourth coverage information corresponding to the running record, wherein the fourth coverage information indicates a ratio of the number of tested code lines in the third code lines to the total number of code lines, wherein the third code is a code line of the fifth test code that has changed relative to the second test code of the second version, and wherein the fifth test code is a code line that has been tested when an interruption occurs during the testing of the first test code; Merging the first coverage information corresponding to the first test code with the fourth coverage information corresponding to the code running record to obtain fifth coverage information; The third coverage information is determined according to the second coverage information and the fifth coverage information.

2. The testing method according to claim 1, characterized in that: The obtaining first coverage information of the first test code of the first version includes: Obtaining a first code execution trace corresponding to the first test code, wherein the first code execution trace represents an execution order of code lines in the first test code; First coverage information of the first test code of the first version is obtained according to the first code execution trace.

3. The testing method according to claim 2, characterized in that: The obtaining a first code execution trace corresponding to the first test code includes: Obtaining a second code execution trace, wherein the second code execution trace is an execution trace of a machine code obtained by performing a stub test in a machine code corresponding to the first test code; Obtain a first code execution trace corresponding to the first test code according to the second code execution trace.

4. The testing method according to claim 1, characterized in that: The obtaining second coverage information of the second test code of the second version includes: Calculate a first hash value corresponding to each code line in the second test code; Calculate a second hash value corresponding to each code line in the third test code; Second coverage information of the second test code of the second version is obtained according to each of the first hash values ​​and each of the second hash values.

5. The testing method according to claim 1, characterized in that: The method further comprises: During the process of testing the first test code, if the first test code is interrupted, obtaining the running record of the third code; The running record of the third code line is saved in the first database.

6. The testing method according to claim 1, characterized in that: The method further comprises: Generate a test report according to the third coverage information; Generate a download address according to the test report; The download address is returned to the client.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images