A method, apparatus, and electronic device for generating coverage data.

Abstract

This specification provides a method for generating coverage data. The method involves constructing monitoring rules with test phase markers, acquiring a program to be tested, randomly assigning ports to the program in a virtual machine, marking the assigned ports as first ports, compiling and running the program using the first ports, storing the compilation results, and simultaneously executing the monitoring rules and marking the running process according to test phases. This generates and stores marked running process information. Coverage data is then generated by combining the compilation results with the marked running process information. By marking and storing the running process according to test phases, coverage data can still be generated after compilation and execution, unaffected by virtual machine restarts or other failures, thus ensuring high reliability.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to a method, apparatus, and electronic device for generating coverage data. Background Technology

[0002] With the development of the software industry, code coverage, as one of the software quality indicators, has gradually gained importance in the industry. Essentially, code coverage is the extent to which the code developed for testing covers the target, such as whether the target statements are present and whether a certain function can be executed. Currently, for programs under test, the majority of the method involves compiling them in a virtual machine and testing the statements simultaneously during compilation, then calculating code coverage based on the statements that are hit.

[0003] However, this method relies on a synchronous compilation process. If a restart or other failure occurs during the process, it will cause the final test results to deviate and reduce the accuracy. This testing method has poor reliability, so it is necessary to provide a more reliable method. Summary of the Invention

[0004] This specification provides a method, apparatus, and electronic device for generating coverage data to improve test reliability.

[0005] This specification also provides a method for generating coverage data, including:

[0006] Build monitoring rules with testing phase markers;

[0007] Obtain the program to be tested, randomly assign a port to the program to be tested in the virtual machine, and mark the assigned port as the first port;

[0008] The program to be tested is compiled and run using the first port, the compilation results are stored, the monitoring rules are executed and the running process is marked according to the test stage, and the running process information with the marks is generated and stored.

[0009] Coverage data is generated by combining the compilation results with the tagged runtime information.

[0010] Optionally, it also includes:

[0011] Obtain the program change sets for each historical version, and calculate the coverage data of the program under test to the program change sets by combining the compilation results and the marked runtime information.

[0012] Optionally, generating coverage data by combining the compilation results with the tagged runtime information includes:

[0013] The compilation results and runtime information are categorized according to the testing phase, and the coverage data for each testing phase is statistically analyzed.

[0014] Optionally, generating coverage data by combining the compilation results with the tagged runtime information includes:

[0015] Various coverage data are generated according to preset integration test rules and / or system test rules and / or regression test rules.

[0016] Optionally, the storage of compilation results includes:

[0017] Obtain the compilation results generated during compilation and store them persistently.

[0018] Optionally, the storage of compilation results further includes:

[0019] Obtain the persistent storage address of the compilation result and record it in the program to be tested;

[0020] When the virtual machine restarts, the program under test persists the compilation results generated after the virtual machine restarts according to the persistent storage address.

[0021] Optionally, the step of persistently storing the compilation results generated by the compilation includes:

[0022] Establish a data warehouse on the server to persistently store the compilation results.

[0023] Optionally, the monitoring rules include a test release process, and the test release process includes a test phase identifier.

[0024] This specification also provides an apparatus for generating coverage data, comprising:

[0025] The monitoring rules module constructs monitoring rules with test phase markers;

[0026] The port allocation module obtains the program to be tested, randomly allocates a port to the program to be tested in the virtual machine, and marks the allocated port as the first port;

[0027] The recording module compiles and runs the program to be tested using the first port, stores the compilation results, executes the monitoring rules, marks the running process according to the test stage, and generates and stores the marked running process information.

[0028] The statistics module combines the compilation results with the tagged runtime information to generate coverage data.

[0029] This specification also provides an electronic device, wherein the electronic device includes:

[0030] Processor; and,

[0031] A memory that stores a computer executable program, which, when executed, causes the processor to perform any of the methods described above.

[0032] This specification also provides a computer-readable storage medium that stores one or more programs that, when executed by a processor, implement any of the methods described above.

[0033] The various technical solutions provided in the embodiments of this specification acquire the program to be tested by constructing monitoring rules with test phase markers, randomly assigning ports to the program to be tested in a virtual machine, marking the assigned ports as first ports, compiling and running the program to be tested using the first ports, storing the compilation results, and simultaneously executing the monitoring rules and marking the running process according to the test phases, generating and storing marked running process information, and combining the compilation results with the marked running process information to generate coverage data. By marking and storing the running process according to the test phases, coverage data can still be generated after compilation and running, unaffected by failures such as virtual machine restarts, thus ensuring high reliability. Attached Figure Description

[0034] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0035] Figure 1 A schematic diagram illustrating the principle of a method for generating coverage data provided in an embodiment of this specification;

[0036] Figure 2 A schematic diagram of a device for generating coverage data provided in an embodiment of this specification;

[0037] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification;

[0038] Figure 4 This is a schematic diagram of a computer-readable medium provided for embodiments of this specification. Detailed Implementation

[0039] Exemplary embodiments of the invention will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limiting the invention to the embodiments set forth herein. Rather, these exemplary embodiments are provided to make the invention more comprehensive and complete, and to facilitate a full communication of the inventive concept to those skilled in the art. The same reference numerals in the drawings denote the same or similar elements, components, or parts, and therefore repeated descriptions of them will be omitted.

[0040] Subject to the technical concept of this invention, the features, structures, characteristics or other details described in a particular embodiment may be combined in one or more other embodiments in a suitable manner.

[0041] In the description of specific embodiments, the features, structures, characteristics, or other details described in this invention are intended to enable those skilled in the art to fully understand the embodiments. However, it is not excluded that those skilled in the art can practice the technical solutions of this invention without one or more of the specific features, structures, characteristics, or other details.

[0042] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0043] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0044] The terms “and / or” or “and / or” include all combinations of any one or more of the listed items.

[0045] Figure 1 This is a schematic diagram illustrating the principle of a method for generating coverage data provided in an embodiment of this specification. The method may include:

[0046] S101: Construct monitoring rules with test phase markers.

[0047] In the embodiments of this specification, stage markers can be set according to the statements to be tested, the functions to be tested, the logic flow to be tested, etc., thereby constructing monitoring rules. In this way, stages can be divided according to the required testing dimensions. The monitoring rules can mark when the program under test has the corresponding statements, logic flow, or can achieve the corresponding functions, thereby enabling marking according to the required testing dimensions.

[0048] Different testing phases can be parallel or inclusive. For example, a process can have multiple steps, each of which can be a phase. Or, a process can be a phase with corresponding markers, and the steps within that process can also be a phase with corresponding markers.

[0049] Therefore, in the embodiments of this specification, the monitoring rules include a test release process, and the test release process includes a test phase identifier.

[0050] Specifically, test phase identifiers can be set according to the attributes of the release process, thereby accurately identifying each time node and operation process of the test phase. This allows for testing in multiple dimensions, such as regression testing and integration testing, breaking free from the limitation of only being able to perform unit tests at the code statement level.

[0051] Therefore, monitoring rules constructed according to different dimensions of phase division can be integration test rules and / or system test rules and / or regression test rules.

[0052] This means that monitoring rules can include multiple rules, enabling monitoring and labeling of multiple dimensions at once.

[0053] Of course, to improve the targeting and efficiency of monitoring, only one rule may be included; no restrictions are placed here.

[0054] The test release process can have multiple test phase identifiers, and different test phase identifiers can be listed side by side or included within each other.

[0055] In practice, building monitoring rules with test phase markers can be done by developing some interception code and manually configuring the test phase markers, or by developing a general monitoring template and a configuration page with drag-and-drop components. The drag-and-drop components carry the test phase marker information, so the monitoring rules can be built by dragging and dropping the components.

[0056] S102: Obtain the program to be tested, randomly assign a port to the program to be tested in the virtual machine, and mark the assigned port as the first port.

[0057] When using a virtual machine for testing, if multiple test processes are running simultaneously, port confusion may occur. Therefore, to reduce port confusion, we can mark the port after assigning it (such as a JVM port). Subsequently, we can directly associate the input and output information from the first port with the current program to be tested.

[0058] In the embodiments of this specification, different microservices on the same virtual machine can be managed in a distributed manner. Different random ports are allocated through a random algorithm, and microservice option parameters are added to the random ports. When a microservice starts, it will start on the port corresponding to the parameter.

[0059] S103: Compile and run the program to be tested using the first port, store the compilation results, execute the monitoring rules and mark the running process according to the test stage, and generate and store the marked running process information.

[0060] By compiling the program to be tested, the compilation result (such as a classes file) can reflect the statements in the program to be tested, and thus can be tested. The compilation result can be persisted for later use.

[0061] The compiled result can also be run, and the running process can reflect the specific execution flow and has phased characteristics. Therefore, these phased characteristics can be used to match the corresponding test phase and then mark it.

[0062] Among them, the stage characteristics may refer to whether information is input or output, what type of information is input or output, the execution logic of multiple statements, the order of calling, etc., which will not be elaborated in detail here.

[0063] In practice, you can use the jacoco tool to mark stages, or you can customize your own dedicated tool for marking; there are no restrictions here.

[0064] In the embodiments described in this specification, storing the compilation results may include:

[0065] Obtain the compilation results generated during compilation and store them persistently.

[0066] The persistent storage address can be a remote database address.

[0067] Therefore, the process of obtaining and persistently storing the compilation results can include:

[0068] Establish a data warehouse on the server to persistently store the compilation results.

[0069] You can create a dedicated data warehouse for each microservice.

[0070] To address the storage interruption caused by virtual machine restarts, and considering that the program under test will be in the execution state during the testing phase, we can use this execution to overcome the impact of virtual machine interruption.

[0071] Therefore, in the embodiments of this specification, storing the compilation results may further include:

[0072] Obtain the persistent storage address of the compilation result and record it in the program to be tested;

[0073] When the virtual machine restarts, the program under test persists the compilation results generated after the virtual machine restarts according to the persistent storage address.

[0074] This allows the compilation results before and after the restart to be pieced together into a complete part, thus preventing the compilation results from being missing.

[0075] In the embodiments of this specification, the execution progress of the compilation results can also be marked. In this way, after restarting, execution can continue from the progress mark without having to start from the beginning, thereby improving efficiency.

[0076] S104: Combine the compilation results with the marked runtime information to generate coverage data.

[0077] This method involves constructing monitoring rules with test phase markers, acquiring the program under test, randomly assigning ports to the program under test in a virtual machine, marking the assigned ports as first ports, compiling and running the program under test using the first ports, storing the compilation results, and simultaneously executing the monitoring rules and marking the running process according to the test phases. This generates and stores marked running process information, and combines the compilation results with the marked running process information to generate coverage data. By marking and storing the running process according to the test phases, coverage data can still be generated after compilation and execution, unaffected by virtual machine restarts or other failures, thus ensuring high reliability.

[0078] When upgrading code, some program change sets are generated. It is these program change sets that enable the code upgrade. Therefore, we can use program change sets to verify whether the program under test has achieved the upgrade goal.

[0079] Therefore, it also includes:

[0080] Obtain the program change sets for each historical version, and calculate the coverage data of the program under test to the program change sets by combining the compilation results and the marked runtime information.

[0081] It also allows for cross-comparison of code coverage during the testing phase with code change sets.

[0082] Specifically, a program change set may include a test release process that marks the test phase, as well as test cases to test the functionality of the program under test for the input instance.

[0083] In this embodiment of the specification, generating coverage data by combining the compilation results with the tagged runtime information may include:

[0084] The compilation results and runtime information are categorized according to the testing phase, and the coverage data for each testing phase is statistically analyzed.

[0085] The classification can be based on stage markers or different testing dimensions, such as different levels of abstraction of program functions. This will not be elaborated on here.

[0086] By using categorized statistics, the accuracy of the statistics is improved, and dimensions with poor coverage can be easily identified, allowing for targeted improvements.

[0087] Of course, they can also be categorized according to different test types.

[0088] Therefore, in the embodiments of this specification, the step of generating coverage data by combining the compilation results with the tagged runtime information may include:

[0089] Various coverage data are generated according to preset integration test rules and / or system test rules and / or regression test rules.

[0090] The specific form of coverage data can be a coverage report.

[0091] The above embodiments standardize the test release process by using stage markers, clearly identifying each test stage, thereby generating accurate and effective coverage reports.

[0092] In the embodiments of this specification, in order to simplify the operation during the testing process, we can build a tester through development. The tester has a test flow control program to control the execution flow of the entire testing process. By executing the tester, the test environment can be automatically deployed, restarted, and the test can begin: compiling the program to be tested and persistently storing the compilation results, running the compilation results, marking the running process, adding a timestamp of the running time, saving the running process information, and then generating a coverage test report based on the saved information.

[0093] In practice, developers can initiate a test release process. Developers can pre-record data such as code commit identifiers and time nodes included in the test process control program, and then control the deployment, operation and restart of microservices through the test process control program.

[0094] Figure 2 This is a schematic diagram of a device for generating coverage data provided in an embodiment of this specification. The device may include:

[0095] Monitoring rules module 201, constructs monitoring rules with test phase markers;

[0096] The port allocation module 202 obtains the program to be tested, randomly allocates a port to the program to be tested in the virtual machine, and marks the allocated port as the first port;

[0097] The recording module 203 compiles and runs the program to be tested using the first port, stores the compilation results, executes the monitoring rules, marks the running process according to the test stage, and generates and stores the marked running process information.

[0098] The statistics module 204 combines the compilation results with the tagged runtime information to generate coverage data.

[0099] This device acquires the program under test by constructing monitoring rules with test phase markers. It randomly assigns ports to the program under test in a virtual machine, designating each port as a first port. The program under test is then compiled and run using this first port, and the compilation results are stored. Simultaneously, the monitoring rules are executed, and the running process is marked according to the test phase. Marked running process information is generated and stored. Coverage data is then generated by combining the compilation results with the marked running process information. Because the running process is marked and stored according to the test phase, coverage data can still be generated after compilation and execution, unaffected by virtual machine restarts or other failures, thus ensuring high reliability.

[0100] Based on the same inventive concept, embodiments of this specification also provide an electronic device.

[0101] The following describes embodiments of the electronic device of the present invention, which can be considered as specific implementations of the methods and apparatus embodiments of the present invention described above. Details described in the embodiments of the electronic device of the present invention should be considered as supplements to the methods or apparatus embodiments described above; details not disclosed in the embodiments of the electronic device of the present invention can be implemented with reference to the methods or apparatus embodiments described above.

[0102] Figure 3 This is a schematic diagram of an electronic device provided as an embodiment of this specification. Refer to the following... Figure 3 The electronic device 300 according to this embodiment of the present invention will be described. Figure 3 The electronic device 300 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0103] like Figure 3 As shown, the electronic device 300 is presented in the form of a general-purpose computing device. The components of the electronic device 300 may include, but are not limited to: at least one processing unit 310, at least one storage unit 320, a bus 330 connecting different system components (including storage unit 320 and processing unit 310), a display unit 340, etc.

[0104] The storage unit stores program code that can be executed by the processing unit 310, causing the processing unit 310 to perform the steps described in the processing method section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 310 can perform, for example... Figure 1 The steps are shown.

[0105] The storage unit 320 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 3201 and / or a cache storage unit 3202, and may further include a read-only memory unit (ROM) 3203.

[0106] The storage unit 320 may also include a program / utility 3204 having a set (at least one) program module 3205, such program module 3205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0107] Bus 330 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0108] Electronic device 300 can also communicate with one or more external devices 400 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with the electronic device 300, and / or with any device that enables the electronic device 300 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 350. Furthermore, electronic device 300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 360. Network adapter 360 can communicate with other modules of electronic device 300 via bus 330. It should be understood that, although... Figure 3 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0109] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described in this invention can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the above-described method according to this invention. When the computer program is executed by a data processing device, it enables the computer-readable medium to implement the above-described method of this invention, i.e.: as... Figure 1 The method shown.

[0110] Figure 4 This is a schematic diagram of a computer-readable medium provided for embodiments of this specification.

[0111] accomplish Figure 1The computer program of the method shown can be stored on one or more computer-readable media. A computer-readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0112] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0113] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0114] In summary, this invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that in practice, general-purpose data processing devices such as microprocessors or digital signal processors (DSPs) can be used to implement some or all of the functions of some or all of the components according to the embodiments of the invention. The invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the invention can be stored on a computer-readable medium or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0115] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device, or electronic device, and various general-purpose devices can also implement the present invention. The above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0116] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0117] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for generating coverage data, characterized in that, include: Based on the statements, functions, and logic flows to be tested, set phase markers and construct monitoring rules with test phase markers. The monitoring rules include a test release process, and test phase identifiers are set according to the attributes of the test release process to identify the time phase and operation process of the test. The program to be tested is obtained, and multiple test processes of the program to be tested are randomly assigned ports in a virtual machine. The assigned ports are marked as the first port. The method also includes adding microservice option parameters to the randomly assigned ports. When different microservices are executed in the same virtual machine, the ports corresponding to the microservice option parameters are started. The test process corresponding to the program under test is compiled and run using the first port, the compilation result is stored, the monitoring rules are executed and the running process is marked according to the test stage, and running process information with test stage marking is generated and stored. The compilation results and runtime information are categorized according to the test phase markers, and the coverage data for each test phase is statistically analyzed.

2. The method according to claim 1, characterized in that, Also includes: Obtain the program change sets for each historical version, and calculate the coverage data of the program under test to the program change sets by combining the compilation results and the marked runtime information.

3. The method according to claim 1 or 2, characterized in that, The coverage data for each testing phase includes: Various coverage data are generated according to preset integration test rules and / or system test rules and / or regression test rules.

4. The method according to claim 1, characterized in that, The stored compilation results include: Obtain the compilation results generated during compilation and store them persistently.

5. The method according to claim 4, characterized in that, The stored compilation results also include: Obtain the persistent storage address of the compilation result and record it in the program to be tested; When the virtual machine restarts, the program under test persists the compilation results generated after the virtual machine restarts according to the persistent storage address.

6. The method according to claim 5, characterized in that, The step of persistently storing the compilation results generated by the compilation includes: Establish a data warehouse on the server to persistently store the compilation results.

7. An apparatus for generating coverage data, characterized in that, include: The monitoring rules module sets phase markers based on the statement to be tested, the function to be tested, and the logic flow to be tested, and constructs monitoring rules with test phase markers. The monitoring rules include a test release process, and test phase identifiers are set according to the attributes of the test release process, thereby identifying the time phase and operation process of the test phase. The port allocation module acquires the program to be tested, randomly allocates ports to multiple test processes of the program to be tested in the virtual machine, and marks the allocated port as the first port; it also includes adding microservice option parameters to the randomly allocated port, and starting the port corresponding to the microservice option parameters when different microservices are executed in the same virtual machine. The recording module uses the first port to compile and run the test process corresponding to the program under test, stores the compilation results, executes the monitoring rules and marks the running process according to the test stage, and generates and stores the running process information with the test stage mark. The compilation results and runtime information are categorized according to the test phase markers, and the coverage data for each test phase is statistically analyzed.

8. An electronic device, wherein, The electronic device includes: Processor; and, A memory storing a computer executable program, which, when executed, causes the processor to perform the method according to any one of claims 1-6.

9. A computer-readable storage medium, wherein, The computer-readable storage medium stores one or more programs that, when executed by a processor, implement the method of any one of claims 1-6.

Images