Software test coverage rate determination method and system, storage medium and electronic equipment
By counting the ratio of the number of times a function is called to the total number in software testing, the problem that the existing technology cannot judge the software business logic design and code implementation from a macro perspective is solved, function-level code coverage test is realized, and the code is discovered Flaws in the design.
Patent Information
- Application Number
- CN202310531088.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-11
- Publication Date
- 2024-11-12
AI Technical Summary
The existing code coverage standards cannot judge whether the business logic design of software projects is reasonable from a macro perspective, and cannot fully discover the defects in code implementation.
By starting from the project home directory of the software to be tested, the ratio of the defined function is counted to the total number of functions to determine the software test coverage, thereby reflecting the processing of code design logic and business logic.
The basic unit of coverage test is promoted from every line of code or every branch to every function level, realizing the ability to reflect the code design logic and business logic from the function level and discover potential defects.
Smart Images

Figure CN118939524A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method, system, storage medium and electronic device for determining software test coverage. Background Art
[0002] Code coverage is a measurement method in software testing. It is a white box testing method that describes the proportion and degree of source code in a program being tested. The resulting proportion is called code coverage. When doing software testing, code coverage is often used as an indicator to measure the quality of the test. The significance of using code coverage is mainly to trace back whether the design of the project is reasonable and whether there are confusion points in the code based on the uncovered parts of the code tested. This can remind developers and designers to sort out the logical relationship of the code and improve the code quality.
[0003] There are four common coverage standards at present, namely statement coverage, decision coverage, condition coverage and path coverage. Statement coverage, also known as line coverage, is the most common coverage method. It measures whether each executable statement in the tested code has been executed. Decision coverage, also known as branch coverage, measures whether each branch of the judgment in the program code has been executed. Condition coverage measures whether the true and false results of each sub-expression in the judgment have been tested. Path coverage, also known as assertion coverage, measures whether each branch of the function has been executed.
[0004] The above-mentioned coverage schemes all have their own defects. They cannot judge whether the business logic design of the software project is reasonable from a macro perspective, nor can they fully discover the defects in the code implementation from a macro perspective. Summary of the invention
[0005] The purpose of the embodiment of the present invention is to provide a method that can better reflect the processing of business logic in the code and discover defects in business logic design and code implementation from a macro perspective.
[0006] In order to achieve the above object, an embodiment of the present invention provides a method for determining software test coverage, the method comprising:
[0007] Start from the main directory of the project of the software to be tested and scan the directories level by level to determine all the project files;
[0008] Scan each project file in turn in the order in which the project files are scanned to determine the total number of functions defined in the software to be tested;
[0009] Run the code of the software to be tested, monitor the functions called during the running process, and count the number of times the defined functions are called;
[0010] After the test of the software to be tested is completed, the code of the software to be tested is stopped, and the coverage of the software to be tested is determined by the ratio of the number of times the defined function is called to the total number of functions.
[0011] Optionally, the step of scanning directories level by level starting from the main directory of the project of the software to be tested to determine all project files includes:
[0012] Get the main project directory of the software to be tested;
[0013] Scan the directories level by level starting from the main directory of the project;
[0014] Store the scanned go files in memory in the order in which they were scanned.
[0015] Optionally, if the code structure of the software to be tested is defined according to levels, the number of functions defined at each level is counted respectively according to the defined levels.
[0016] Furthermore, the levels of the code structure include one or more of the following: a control layer, a business layer, and a warehouse layer.
[0017] Preferably, in the process of counting the number of times the defined function is called, for the code executed in the conditional loop, the number of times the function defined in the loop is actually called and executed is not considered, and the number of times the code executed in the conditional loop is looped is used as the number of executions of the function.
[0018] Preferably, in the process of counting the number of times the defined function is called, for the code executed in a conditional loop, the number of times the code executed in the conditional loop is looped is not considered, and the number of times the defined function is actually called and executed is used as the number of executions of the function.
[0019] Preferably, while running the code of the software to be tested, monitoring the functions called during the running process, and counting the number of times the defined functions are called, the monitoring and statistical results are output to a test log file for viewing the test process of the software to be tested.
[0020] In another aspect, the present invention provides a system for determining software test coverage, the system comprising:
[0021] A scanning module is used to scan the directories level by level starting from the main directory of the project of the software to be tested to determine all the project files, and to scan each project file in turn in the order in which the project files are scanned to determine the total number of functions defined in the software to be tested;
[0022] A monitoring and counting module, which monitors the functions called in the process of running the code of the software to be tested, and counts the number of times the defined functions are called; and
[0023] The coverage determination module stops running the code of the software to be tested after the test of the software to be tested is completed, and determines the coverage of the software to be tested by the ratio of the number of times the defined function is called to the total number of functions.
[0024] On the other hand, the present invention provides a machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute: any of the methods for determining software test coverage described above in the present application, or any of the methods for determining software useless code described above in the present application.
[0025] On the other hand, an embodiment of the present invention provides an electronic device, characterized in that it includes: at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute: any one of the software test coverage determination methods mentioned above in the present application, or the software useless code determination method mentioned above in the present application.
[0026] Through the above technical scheme, the project file of the software to be tested is first scanned to find all the functions defined in the project and count the total number of all functions. Then, the code of the software to be tested is run, the functions called during the running process are monitored, and the number of times the defined functions are called is counted. After the test of the software to be tested is completed, the test coverage of the software to be tested is determined by the ratio of the total number of times the defined functions are called to the total number of all functions. Therefore, the problems existing in the code design logic and business logic can be reflected from a macro perspective at the function level.
[0027] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:
[0029] Figure 1 is a flow chart of an embodiment of a method for determining software test coverage of the present invention;
[0030] Figure 2 yes Figure 1 A schematic diagram of the project directory structure of an embodiment;
[0031] Figure 3 is a structural diagram of an embodiment of a system for determining software test coverage of the present invention; and
[0032] Figure 4 It is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0033] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.
[0034] The method for determining software test coverage of the present invention aims to reflect the problems existing in code design logic and business logic at the function level. Figure 1 and Figure 2 The implementation process of one of the embodiments is described.
[0035] As shown in the figure, the steps of the method for determining software test coverage in this embodiment include:
[0036] Step 1: Scan the directories one by one from the main directory of the project of the software to be tested to determine all the project files;
[0037] In this embodiment, the corresponding process of performing software testing on the Go language testing tool also requires necessary preparations before step 1, including: building an .exe executable file through go build, executing go mod tidy in the main directory of the code to obtain all dependencies required for the code to run, and then building an exe executable file through go build. The executable file can be named arbitrarily, such as a.exe, func.exe, etc., and all of the following are referred to as exe executable files. At this time, the file has all the dependencies required by the original code. At this time, the executable file is placed in the main directory of the code that needs to test code coverage.
[0038] The project directory structure of the embodiment is as follows Figure 2 As shown, if the executable file is placed in the bookstore project directory and executed, the function-level coverage of the code in the bookstore project will be tested. Figure 2 The functest.exe file is the executable file generated in this embodiment. After the executable file is in the corresponding project folder, you can cd to the corresponding project directory through the terminal or command prompt in the Windows environment, and then execute the executable file with . / .exe.
[0039] After the file is executed, it starts scanning from the project directory, which is the level above the executable file, and starts a hierarchical scan from this level of directory. The program stores the scanned go files (files ending with go) in the order in which they appear in the memory.
[0040] Step 2: Scan each project file in turn in the order in which the project files are scanned to determine the total number of functions defined in the software to be tested;
[0041] In this embodiment, the number of functions and methods defined in the go file is scanned in turn, with func as the keyword. At this time, the number of defined functions is counted, and an upgrade can also be done here.
[0042] In some implementations, if the project structure of the test is clear, such as being divided into a control layer, a business layer, a warehouse layer, etc., the number of defined functions and methods can be counted separately according to the level of the project.
[0043] Step 3: Run the code of the software to be tested, monitor the functions called during the running process, and count the number of times the defined functions are called;
[0044] In this embodiment, after the executable file obtains the number of defined functions and methods, it will output a prompt, and at this time, the code of the project to be tested needs to be run, and the business of the project code needs to be tested. At the same time, the program run by the executable file will monitor the methods and functions called during the test process, and count the number of times the defined functions and methods are called. At the same time, combined with conditional coverage, mainly the loop in the code, at this time, record how many times the loop has been looped and how many times the loop has been executed. Considering the number of times advantage attached to the loop structure, two function calculation schemes in the loop are designed: one is to replace the number of calls of the function in the loop with the number of times the loop is executed without considering how many times the function in the loop is executed; the other is to count completely according to the number of times the function is executed, regardless of whether it is in the loop.
[0045] The first method is mainly to avoid the excessive weight of the function due to too many cycles of certain businesses, which will affect the statistics of function coverage. However, this calculation method loses the count in the cycle, which will cause a slight distortion in the final result. Therefore, the second full-volume counting and statistics scheme is retained. The data source of this counting scheme is not distorted, but the counting method is affected by the project design structure, and the frequency and coverage of the function obtained will also deviate.
[0046] However, combining the two counting methods can better analyze the project's code design and framework design as described above.
[0047] Step 4: After the test of the software to be tested is completed, stop running the code of the software to be tested, and determine the test coverage of the software to be tested by the ratio of the number of times the defined function is called to the total number of functions.
[0048] In this embodiment, after the test project and business are completed, enter "stop program" or press ctrl+c in the terminal interface to force the running code to stop. At this time, the coverage test result will be synchronously output on the terminal to show the coverage result of the software to be tested.
[0049] In some implementations, the result of the coverage test will synchronously generate a test log file in the current directory, and the log file includes the monitoring and statistical results, as well as the test process of the software to be tested.
[0050] Compared with the prior art, the technical advantage of this embodiment is that the basic unit of coverage testing is upgraded from the code level of each line of code or each branch of code to the unit of each function. It does not focus on the coverage of the specific code in the function, but reflects the code design logic and business logic from a macro perspective at the function level.
[0051] The embodiment of the present invention provides a system for determining software test coverage, such as Figure 3 As shown, the system includes:
[0052] A scanning module is used to scan the directories level by level starting from the main directory of the project of the software to be tested to determine all the project files, and to scan each project file in turn in the order in which the project files are scanned to determine the total number of functions defined in the software to be tested;
[0053] A monitoring and counting module, which monitors the functions called in the process of running the code of the software to be tested, and counts the number of times the defined functions are called; and
[0054] The coverage determination module stops running the code of the software to be tested after the test of the software to be tested is completed, and determines the coverage of the software to be tested by the ratio of the number of times the defined function is called to the total number of functions.
[0055] An embodiment of the present invention provides a device for determining software test coverage, which includes a processor and a memory. The above-mentioned scanning module, monitoring and statistical module, coverage determination module, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to implement corresponding functions.
[0056] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the software test coverage is determined by adjusting the kernel parameters.
[0057] An embodiment of the present invention provides a storage medium on which a program is stored. When the program is executed by a processor, the method for determining software test coverage is implemented.
[0058] An embodiment of the present invention provides a processor, which is used to run a program, wherein the method for determining software test coverage is executed when the program is running.
[0059] An embodiment of the present invention provides an electronic device, such as Figure 4 As shown, the electronic device includes at least one processor, at least one memory connected to the processor, and a bus; wherein the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the above-mentioned method for determining software test coverage. The device in this article can be a server, PC, PAD, mobile phone, etc.
[0060] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: Step 1: Scanning directories level by level starting from the main directory of the project of the software to be tested to determine all project files;
[0061] Step 2: Scan each project file in turn in the order in which the project files are scanned to determine the total number of functions defined in the software to be tested;
[0062] Step 3: Run the code of the software to be tested, monitor the functions called during the running process, and count the number of times the defined functions are called;
[0063] Step 4: After the test of the software to be tested is completed, stop running the code of the software to be tested, and determine the test coverage of the software to be tested by the ratio of the number of times the defined function is called to the total number of functions.
[0064] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0065] In a typical configuration, the device includes one or more processors (CPU), memory and bus. The device may also include input / output interface, network interface and the like.
[0066] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.
[0067] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0068] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0069] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0070] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.
Claims
1. A method for determining software test coverage, comprising: Start from the main directory of the project of the software to be tested and scan the directories level by level to determine all the project files; Scan each project file in turn in the order in which the project files are scanned to determine the total number of functions defined in the software to be tested; Run the code of the software to be tested, monitor the functions called during the running process, and count the number of times the defined functions are called; After the test of the software to be tested is completed, the code of the software to be tested is stopped, and the coverage of the software to be tested is determined by the ratio of the number of times the defined function is called to the total number of functions.
2. The method for determining software test coverage according to claim 1, characterized in that: The method starts from the main directory of the project of the software to be tested and scans the directories level by level to determine all the project files, including: Get the main project directory of the software to be tested; Scan the directories level by level starting from the main directory of the project; Store the scanned go files in memory in the order in which they were scanned.
3. The method for determining software test coverage according to claim 1, characterized in that: If the code structure of the software to be tested is defined according to levels, the number of functions defined in each level is counted respectively according to the defined levels.
4. The method for determining software test coverage according to claim 3, characterized in that: The levels of the code structure include one or more of the following: a control layer, a business layer, and a warehouse layer.
5. The method for determining software test coverage according to claim 1, characterized in that: In the process of counting the number of times the defined function is called, for the code executed in the conditional loop, the number of times the function defined in the loop is actually called and executed is not considered, and the number of times the code executed in the conditional loop is looped is used as the execution number of the function.
6. The method for determining software test coverage according to claim 1, characterized in that: In the process of counting the number of times the defined function is called, for the code executed in a conditional loop, the number of times the code executed in the conditional loop is looped is not considered, and the number of times the defined function is actually called and executed is used as the execution number of the function.
7. The method for determining software test coverage according to claim 1, characterized in that: While running the code of the software to be tested, monitoring the functions called during the running process, and counting the number of times the defined functions are called, the monitoring and statistical results are output to a test log file for viewing the test process of the software to be tested.
8. A system for determining software test coverage, comprising: A scanning module is used to scan the directories level by level starting from the main directory of the project of the software to be tested to determine all the project files, and to scan each project file in turn in the order in which the project files are scanned to determine the total number of functions defined in the software to be tested; A monitoring and statistics module monitors the functions called in the process of running the code of the software to be tested, and counts the number of times the defined functions are called; as well as The coverage determination module stops running the code of the software to be tested after the test of the software to be tested is completed, and determines the coverage of the software to be tested by the ratio of the number of times the defined function is called to the total number of functions.
9. A machine-readable storage medium having instructions stored thereon, the instructions being used to cause a machine to execute: a method for determining software test coverage as described in any one of claims 1-7.
10. An electronic device, characterized in that: include: At least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; The processor is used to call the program instructions in the memory to execute: the method for determining software test coverage according to any one of claims 1 to claim 7.
