Coverage rate test processing method and device and coverage rate test server and system

[0037] Hereinafter, the application will be described in detail with reference to the drawings and in conjunction with embodiments. It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other if there is no conflict.

[0038] figure 1 It is a schematic diagram of the architecture of the coverage test according to the embodiment of the present application. For the purpose of description, figure 1 The drawn architecture is only an example of the applicable environment of the embodiments of the present application, and does not limit the scope of use or functions of the present application. The principles of the present application can be operated using other general or dedicated computing or communication environments or configurations. Examples of well-known computing systems, environments and configurations applicable to this application include, but are not limited to, personal computers, servers, multi-processor systems, micro-processing-based systems, minicomputers, mainframe computers, and including any of the foregoing systems or devices Distributed computing environment. Below figure 1 Take the architecture in as an example to illustrate, such as figure 1 As shown, the architecture includes one or more coverage test servers 10 and a coverage test processing device 12, and the coverage test processing device 12 is connected to one or more coverage test servers 10. It should be noted that the coverage rate The test processing device 12 can communicate with the coverage test server 10. Therefore, the connection here includes, but is not limited to: a direct connection between the coverage test processing device 12 and the coverage test server 10, or a connection through other intermediate network elements, Wired connection, or wireless connection, etc.

[0039] The coverage test server 10 can choose to use existing coverage test methods or tools according to different programming languages. Therefore, in the following embodiments, the specific implementation manners of performing coverage testing on these single coverage test servers 10 are not Repeat it again. For the coverage test processing device 12, it can be located in one or more separate physical servers, or as a functional module located in an existing server that provides other service functions. The specific application can be based on different network environments and/or Different requirements of the coverage test processing device 12 are selected.

[0040] figure 2 It is a flowchart of a coverage test processing method according to an embodiment of the present application, which is combined with figure 1 The architecture pair shown in figure 2 Explain, such as figure 2 As shown, the process includes the following steps:

[0041] Step S202: Receive the result of the coverage test performed by the coverage test server 10 reported by one or more coverage test servers 10, where each coverage test server is used to test the coverage of part of the code to be tested ;

[0042] Step S204, merge the received coverage test results, and generate a test report.

[0043] Through the above steps, the test results on each coverage test server 10 can be merged to generate a test report. So you can get a relatively complete coverage test report. It helps to understand the overall coverage test result of a project or a functional module in a project, provides a guarantee for improving the coverage of the test, reduces the occurrence of hidden problems to a certain extent, and improves the project platform The stability of its operations.

[0044] It can be seen from the above steps that a distributed coverage test is proposed in this embodiment. In the existing technology, the coverage test only provides a version running on a single machine. In this way, for software developers, they are limited by the coverage testing technology in the prior art and can only be tested on a single computer. Although the concept of distributed may also exist in other fields, for coverage testing, there is no existing technology that implies or expresses that distributed coverage testing can be performed for coverage testing.

[0045] In the prior art, if the overall coverage test of a project needs to be performed, then all the codes of the project are only concentrated on one coverage test server, that is, the coverage test of the code of each part and the overall project The coverage test is conducted separately. by figure 2 The steps shown in realize the merging process, that is, after a single coverage test server executes the coverage test of part of the code, the merging action can control the overall coverage test results of the entire project or multiple projects. Compared with the practice of concentrating all the code of a project on one coverage testing server, the efficiency of testing is improved.

[0046] During implementation, the coverage test server that needs to merge test results can be set according to actual needs.

[0047] For example, for a project A, all the coverage test servers include ten coverage test servers A1-A10, where A1 to A3 perform the coverage test of function module B1, and A4 to A6 perform the coverage test of function module B2 Rate test, other executive function module A3 coverage test. If you need the total coverage test results of B1 and B2, you can merge the test results of A1 to A6, and if you need the coverage information of the entire project, you can merge the test results of A1 to A10.

[0048] For another example, a project can be split into two parts, and coverage tests are run on two coverage test servers respectively. When collecting coverage statistics, the data on both machines can be collected. For the convenience of description, the coverage test processing device is called the server, and the coverage test server is called the client. In this preferred example, the server can send a command to notify the client that it needs to collect the current data. After receiving the command, the client can lock the collection container of the client (the purpose of locking is to avoid the generation of dirty data), and The data of the collection container is directly converted into a bytecode object with the detailed structure of the project package and class and corresponding coverage information, and transmitted to the server. When the server obtains the bytecode objects of the two test machines, Merge the information of the bytecode object (for example, you can read the information of the second bytecode object based on one of the bytecode objects, and then determine whether there is a second bytecode object The information of the bytecode object, if it does not exist, add it. If it does exist, you can judge whether it needs to be superimposed according to the type of information. Of course, the way of merging is not limited to this one. You can choose different according to the format of the coverage test result Merging method). In some programming languages ​​(for example, java), because the hierarchy of bytecode objects is relatively clear, coverage information can be collected according to the organization structure of the language package (for example, if the organization structure of the programming language package is a tree Structure, then you only need to traverse the classes in the package to collect coverage one by one), that is, the combination of coverage test results can be done according to the structure of the project in different programming languages.

[0049] The above two examples are based on the same project as an example. However, they are not limited to this. The embodiments and preferred implementations involved in this application are also applicable to merging the coverage of different projects. The implementation is consistent with the merging implementation of the same project, and different identifications can be set for different projects to be able to distinguish the projects. Of course, there is no need to identify the project, because for some programming languages, the code naming is sufficient to distinguish the project to which each code belongs. The merging of the coverage rates between different projects has far-reaching significance. Because the coverage rates of different projects can be uniformly counted, the results of coverage rates can be compared from a larger scope. For example, according to product line statistics, according to department Statistics, so that coverage information can be reflected from multiple dimensions, and more optimally, coverage reports with consistent styles can be generated. No matter which of the above is possible, the existing technology cannot accomplish it.

[0050] Regarding when the coverage test server 10 reports, two preferred methods are provided in this embodiment:

[0051] Manner 1: A message may be sent to one or more coverage test servers 10, and the message is used to instruct the coverage test server 10 that received the message to report the result of the coverage test performed by the coverage test server 10. In this way, the coverage test processing device 12 can be better utilized, so that the device can fully grasp when the coverage test server is required to report. In this preferred manner, it is not necessary for the coverage test server 10 to report the test results at any time. Instead, the coverage test processing device 12 informs the coverage test server 10 to report when needed, compared to the coverage test server timing The report processing method saves bandwidth resources between the coverage test server 10 and the coverage test processing device 12 on the one hand, and on the other hand, reduces the load of the coverage test processing device 12 to process the test results. However, this method may not collect enough information. For example, the coverage test server conducted coverage tests at 10:00 and 12:00, and the coverage test processing device 12 required to report the test results. However, due to coverage The rate test server has already conducted another test at 12:00, and it may delete the test results obtained at 10:00 according to its own strategy. At this time, if the coverage test processing device 12 needs the test result at 10:00, it will not be able to obtain it. Of course, there is a relatively simple way to solve this problem, that is, all test results can be saved on the coverage test server 10. Because the coverage test server 10 also needs to execute the save code, run the code, run the coverage test tool, etc. Therefore, if it is necessary to save all test results on the server, the requirements for the coverage test server 10 are relatively high.

[0052] In this way, as a more preferable implementation, the coverage test server 10 can provide the coverage test processing device 12 with address information that can obtain the coverage test result, so that the coverage test processing device 12 can obtain the coverage The coverage test result is actively acquired on the rate test server 10, and this mode may be called a pull mode.

[0053] Manner 2: The coverage test server 10 may periodically report the results of the coverage test performed by the coverage test server. For example, it can be reported after the completion of each coverage test. Compared with the first method, the coverage test server 10 can have greater flexibility in this processing method. Developers can test the server locations according to different coverage rates. Different test codes are used to set different periodic reporting strategies. For example, every 3 seconds, the code under test will automatically use a processing method similar to heartbeat messages to push the coverage information to the platform that collects coverage information (for example, the coverage test processing device 12). This mode can be called Push mode.

[0054] For the above two methods, in actual implementation, they can be used alone or in combination. In order to obtain test results more conveniently on the coverage test server, an open interface can be provided on the coverage test server. Of course, an open interface can also be provided on the coverage test processing device, both of which can be called by other platforms, which will provide greater convenience for integration with other platforms or applications.

[0055] Regardless of the above reporting method, it can be reported according to time information and/or code information. For example, the coverage test server 10 can report according to the time information or coverage carried in the message received from the coverage test processing device 12 The time information configured locally by the rate test server 10 reports the result of performing the coverage test in the time period corresponding to the time information; for another example, the coverage rate test server 10 may also report the code information or the coverage rate test server carried in the message 10 The locally configured code information reports the results of the coverage test performed on the code corresponding to the code information. By means of message sending time information and code information, the coverage test processing device 12 can more easily control the coverage test server 10 in a centralized manner. By means of self-reporting by the coverage testing server, the reporting of the coverage testing server can be more in line with the reporting requirements of the test performed by the coverage testing server. These preferred embodiments can be combined in different ways according to actual needs.

[0056] For the test tools executed on the coverage test server, in the prior art, the test results are generally returned after all the codes to be tested are executed. For example, in the existing Cobertura implementation, the instrumentation is completed The latter code can only generate the corresponding coverage information after the test is completed, which means that the coverage information cannot be obtained in real time during the test, and effective coverage report generation; if only the coverage result of the final test state can be obtained It will make code coverage deceptive. In actual implementation, for situations where it may be necessary to obtain test results in real time during the test process, this embodiment provides a better implementation. In this preferred implementation, if the coverage test server 10 is executing During the coverage test, it is determined that the coverage test result needs to be reported (for example, according to the notification of the coverage test processing device 12, or the coverage test result needs to be reported according to the local configuration); the coverage test that has been executed can be reported the result of. For example, if the coverage test server 10 is performing a test and receives a message from the coverage test processing device 12 at this time or needs to report the test result according to its local configuration, the coverage test server 10 can suspend the current test. Report the completed test results. Of course, the current test can be continued after the report is completed. In this way, the test results can be obtained in more real time.

[0057] As a preferred embodiment, the coverage test server 10 periodically sends a heartbeat message during the execution of the coverage test, where the heartbeat message serves the purpose of keep-alive, and the coverage test processing device 12 can be informed by the keep-alive Which coverage test server is working properly. The coverage test processing device 12 realizes real-time monitoring of the coverage test server through the heartbeat message. If the coverage test processing device 12 does not receive a heartbeat message from a certain coverage test server within a predetermined period of time, it can be considered as the The server fails, so you can deal with alarms.

[0058] More preferably, the heartbeat message can also report at least one of the following information: the code under test, the IP address and/or port, the project information under test, etc. Reporting the IP address and/or port through the heartbeat message allows the coverage test processing device to know which coverage test servers are connected, so as to avoid retaining the connection addresses of some coverage servers that have been disconnected.

[0059] In order to enable the coverage test processing device 12 to better control the coverage test server 10, another two preferred implementation manners are provided in this embodiment, and these two implementation manners can be used alone or in combination.

[0060] Manner 1: The coverage test processing device 12 may send a clear message to one or more coverage test servers 10, where the clear message is used to indicate that the recipient of the clear message will cover the local storage after receiving the clear message. The results of the rate test are cleared. Through this preferred method, on the one hand, the information capacity stored on the coverage test server 10 can be reduced, and on the other hand, it can also avoid the superimposition of test information when retesting the part of the code.

[0061] Manner 2: The coverage test processing device 12 may also send a test command to one or more coverage test servers 10, where the test command carries time information and/or code information required to perform the coverage test. More preferably, the test command can be used in combination with a message for instructing the coverage test server to report the coverage test result, so that the coverage test processing device 12 can control the coverage test server 10 more flexibly.

[0062] In this embodiment, the coverage test server 10 can be implemented in a manner of instrumenting, which is an existing processing method, and will not be described in this embodiment. However, in the existing instrumentation processing, nesting processing cannot be realized. In this embodiment, the coverage test server can perform the coverage test by instrumenting the bytecode corresponding to the code referenced by the bytecode. This realizes the nested execution of coverage testing. By combining the method of instrumenting the bytecode corresponding to the applied code and directly instrumenting the bytecode, the coverage test coverage can be wider.

[0063] Through the above-mentioned embodiments and preferred implementations, real-time coverage monitoring can be performed on the codes being tested scattered on various coverage test servers. In addition, functions such as automatic emptying and collection can also be realized. It should be noted that these advantages are only available in certain preferred embodiments, and it is not necessary to achieve these advantages at the same time to implement any of the above-mentioned preferred embodiments.

[0064] The modules involved in the aforementioned coverage test processing device 12 will be described below. As used below, the term "module" can implement a combination of software and/or hardware with predetermined functions. Although the devices described in the following embodiments are preferably implemented by software, hardware or a combination of software and hardware is also possible and conceived.

[0065] image 3 Is a structural block diagram of a coverage test processing device according to an embodiment of the application, such as image 3 As shown, the structure includes: a receiving module 32 and a generating module 34. The structure will be described in detail below.

[0066] The receiving module 32 is configured to receive the result of the coverage test performed by the coverage test server reported by one or more coverage test servers, where the coverage test server is used to test the coverage of part of the code to be tested; The generating module 34, which is connected to the receiving module 32, is used to merge the received coverage test results and generate a test report.

[0067] Figure 4 It is a preferred structural block diagram of the coverage test processing device according to the embodiment of the application, such as Figure 4 As shown, the device further includes: a first sending module 42 for sending a message to one or more coverage test servers, where the message is used to instruct the coverage test server that received the message to report the coverage rate The result of the coverage test performed by the test server. As a preferred embodiment, the message may carry time information and/or code information, and the time information is used to instruct the coverage test server to report the result of performing the coverage test within the time period corresponding to the time information. The code information is used to instruct the coverage test server to report the result of the coverage test performed on the code corresponding to the code information.

[0068] Figure 5 Is another preferred structural block diagram of the coverage test processing device according to the embodiment of the application, such as Figure 5 As shown, the device further includes: a second sending module 52, configured to send a clearing message to one or more coverage test servers, where the clearing message is used to instruct the recipient of the clearing message after receiving the clearing message , To clear the locally saved coverage test results.

[0069] Image 6 Is another preferred structural block diagram of the coverage test processing device according to the embodiment of the present application, such as Image 6 As shown, the device further includes: a third sending module 62, configured to send a test command to one or more coverage test servers, wherein the test command carries time information and/or code information that needs to perform the coverage test .

[0070] It should be noted that the above Figure 4 , Figure 5 with Image 6 The preferred structures shown in can be implemented individually or together.

[0071] The modules involved in the coverage test server 10 described above are described below. Likewise, as used below, the term "module" can implement a combination of software and/or hardware with predetermined functions. Although the devices described in the following embodiments are preferably implemented by software, hardware or a combination of software and hardware is also possible and conceived.

[0072] Figure 7 Is a structural block diagram of a coverage test server according to an embodiment of the application, such as Figure 7 As shown, the structure includes: an execution module 72 and a reporting module 74. The structure is described below.

[0073] The execution module 72 is used to execute the coverage test of part of the code under test. The module can be executed by using existing coverage test methods or tools, and will not be repeated in this embodiment; the reporting module 74 is the module Connected to the execution module 72, which is configured to report the result of the coverage test according to the received message and/or local configuration information for indicating the report of the coverage test result.

[0074] The coverage test server 10 in this embodiment is no longer a server that performs a coverage test alone, but has a reporting function. This reporting is used to understand the test results performed by the coverage test server. Possibly, no matter which network element (or called a device or equipment) the test result is reported to, the network element can obtain the test status performed by the server, thereby providing the possibility for the network element to provide the next operation. The coverage test server and the coverage test processing device can be implemented as an integral system, and of course, it can also be implemented as an integral system with other network elements.

[0075] Preferably, the execution module 72 is configured to perform the coverage test by instrumenting the specified bytecode and/or the bytecode corresponding to the code referenced by the bytecode.

[0076] Figure 8 Is a preferred structural block diagram of the coverage test server according to the embodiment of the application, such as Figure 8 As shown, the reporting module 74 may include: a first reporting module 742, configured to report, according to the time information carried in the message or the time information locally configured by the coverage test server, the results obtained by performing the coverage test in the time period corresponding to the time information Result; and/or, a second reporting module 744, which is configured to report the result of performing a coverage test on the code corresponding to the code information according to the code information carried in the message or the code information locally configured by the coverage test server.

[0077] Preferably, the reporting module 74 may be used to report the completed coverage test result when the coverage test result needs to be reported during the execution of the coverage test by the execution module. Preferably, the reporting module 74 may also be used to periodically send a heartbeat message during the execution of the coverage test, where the heartbeat message is used to keep alive the coverage test server.

[0078] For the coverage test system in this embodiment, it may be any coverage test processing device 12 described above, and one or more any coverage test server 10 described above.

[0079] The following uses Cobertura as an example to describe a preferred embodiment. Although Cobertura is used as an example, it is not limited to this. Other Java coverage testing tools or coverage testing tools in other languages ​​are also applicable. .

[0080] Picture 9 Is a schematic diagram of a coverage test system according to a preferred embodiment of the present application, such as Picture 9 As shown, the Cobertura instance is located on the side of the coverage test server 10, and the Cobertura server instance is located on the side of the coverage test processing device 12. The processes in these two instances will be described below.

[0081] The Cobertura instance includes: Client Process (CP for short) and Register Proxy Process (RPP for short). Among them, RPP is used to send heartbeat messages (or heartbeat registration messages) to the Cobertura server instance. ).

[0082] The Cobertura server instance includes two parts: the shared pool and the background process. The shared pool is used to store information such as the server address of the tested code registered in real time. The implementation of the shared pool can be implemented in accordance with the existing implementation. This will not be repeated here. Background processes may include: Register Process (RP), Client Proxy Process (CPP), Register Monitor (RMON), History Monitor (History Monitor, CPP) It is HMON) and the service process (Server Process, referred to as SP). Among them, CPP communicates with CP, and RPP and RP cooperate to send and receive heartbeat messages. RMON is used to monitor whether the data in the shared pool is out of date. HMON uses For monitoring historical data, CP, as the main process of the Cobertura server instance, is used to complete the merging of test reports and manage the Cobertura instance.

[0083] Combine below Picture 9 The thread in the description. When the instrumented code is being tested, RPP will register information with the Cobertura server instance at regular intervals. The RP of the Cobertura server instance will update the information of the code under test to the shared pool, and RMON will respond to the heartbeat message sent by PRR. The situation checks the information in the shared pool (the shared pool is a special area in the database, and the use of the shared pool is the same as the existing usage, so I won't repeat it here), and clear the expired information. The completed test report can be saved in the form of a file on the file server. HMON is used to maintain the information of the database and file system, and corresponding rules can be configured to delete obsolete data. It can be seen from the above that RMON is mainly used to maintain the information in the shared pool, while HMON is mainly used to maintain the information in the file system.

[0084] The external system can obtain coverage information through the API interface or page. CPP will notify the CP in the eligible Cobertura instance, collect the current coverage information, and merge the coverage information from each Cobertura instance, and generate it after the merge is completed The coverage report can be stored in the server and returned to the access address.

[0085] In this preferred embodiment, it can also support real-time clearing of coverage information scattered on each Cobertura instance. For example, you can query the registered information in the shared pool through SP, and then notify each Cobertura instance through CPP to perform real-time Empty operation. This operation can avoid superimposition of test information in the process of retesting the code.

[0086] In addition, because Cobertura's support for recursion is relatively weak, it is extended in this preferred embodiment, that is, in the instrumentation stage, the coverage tool can support recursive instrumentation, that is, it can instrument the specified bytecode and can The jar packages referenced by these bytecodes are instrumented, and the results of the jar packages are counted. Since the jar package referenced by the bytecode can be instrumented, recursive processing is realized. For other tools, the package referenced by the bytecode can also be instrumented as needed to achieve recursive processing.

[0087] The above-mentioned preferred embodiment improves the efficiency of testers to check coverage and compare coverage. It is possible to obtain the specific conditions of the code being tested distributed in each Cobertura instance and the overall condition after merging in real time. This advantage is particularly prominent in large-scale applications , Can help test engineers to monitor test quality globally, and provide the possibility to obtain information about automated test process.

[0088] In another embodiment, a software is also provided, which is used to execute the technical solutions described in the above-mentioned embodiments and preferred implementations.

[0089] In another embodiment, there is also provided a storage medium in which the above-mentioned software is stored. The storage medium includes, but is not limited to, an optical disk, a floppy disk, a hard disk, and a rewritable memory.

[0090] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of this application can be implemented by a general computing device, and they can be concentrated on a single computing device or distributed in a network composed of multiple computing devices. Above, alternatively, they can be implemented with program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or Multiple modules or steps are made into a single integrated circuit module to achieve. In this way, this application is not limited to any specific hardware and software combination.

[0091] The foregoing descriptions are only preferred embodiments of the application, and are not intended to limit the application. For those skilled in the art, the application may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.