Coverage rate generation and visualization method and device based on static analysis

Abstract

The invention discloses a coverage rate generation and visualization method and device based on static analysis, and the method comprises the steps: 1, receiving a system configuration file and starting parameters inputted by a user, analyzing the system configuration file into internal data of a coverage rate tool, and preparing a corresponding environment for coverage rate generation and static analysis; 2, creating a tool class according to the system configuration file; 3, calling a tool class to carry out coding instrumentation on the source code to obtain coverage rate original data; 4, performing static analysis on the source code to obtain static analysis data; 5, performing comprehensive analysis on the coverage rate original data and the static analysis data to obtain coverage rate sample data based on static analysis; and 6, generating an HTML file by the coverage rate tool according to the coverage rate sample data, and realizing visual display of the coverage rate. According to the method, unreasonable codes, including too complex functions, deep nested conditional statements and the like, in the codes can be analyzed and identified, and the running performance of a system can be improved by reconstructing the codes.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a static analysis-based coverage generation and visualization method and device. Background Art

[0002] In the field of software testing, accurately evaluating test coverage has always been a crucial goal. Traditional test coverage evaluation methods mainly rely on dynamically executing test cases to collect information, but this approach has some limitations.

[0003] As software systems become increasingly complex and their scale continues to expand, dynamic testing often faces problems such as high time cost and difficulty in covering all execution paths. Especially in some large and complex systems, it takes a lot of time and resources to achieve a high coverage rate through dynamic testing.

[0004] In addition, dynamic testing can usually only be performed in specific test scenarios, and may not be able to effectively cover some special situations that are difficult to trigger or complex logic branches. Moreover, once the system changes, re-running dynamic testing requires a lot of manpower and time. Summary of the invention

[0005] The present invention provides a coverage generation and visualization method and device based on static analysis to solve the technical problems mentioned in the background technology.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] The present invention provides a coverage generation and visualization method based on static analysis, comprising the following steps:

[0008] S1. Receive the system configuration file and startup parameters input by the user, parse the system configuration file into internal data of the coverage tool, and then prepare the corresponding environment for coverage generation and static analysis based on the internal data of the coverage tool or through a pre-prepared image;

[0009] S2. Create a tool class based on the system configuration file and startup parameters;

[0010] S3. Call the tool class to encode and instrument the source code and obtain the original coverage data;

[0011] S4. Perform static analysis on the source code to obtain static analysis data;

[0012] S5. Comprehensively analyze the original coverage data and static analysis data to obtain coverage sample data based on static analysis;

[0013] S6. Generate an HTML file using the coverage tool and based on the coverage sample data to achieve visual display of coverage.

[0014] Furthermore, the system configuration file in S1 covers the specified compilation tool chain, programming language type, strategy type, HTTP information for pushing coverage files, and notification interface information; different strategy types will specify different output formats in the processing logic, including the format conversion required for the corresponding output format, and the toolkit for generating the coverage format.

[0015] The startup parameters include input type, source code path, target name, corpus path, Git address and Token information; the input type is the type of source code source, including local source code, Git code library or other types; if the source code source type is local source code, the coverage tool runs as SideCar in S6; if the source code source type is Git code library or other types, before compiling, the source code needs to be dynamically pulled to the specified directory according to the system configuration file.

[0016] Furthermore, the corresponding environment in S1 refers to the installation of the system software package.

[0017] Furthermore, the S2 specifically includes the following steps:

[0018] S21. Create a tool class based on the system configuration file and startup parameters; the tool class includes an LLVM tool chain or a GNU tool chain;

[0019] S22. Create corresponding policy classes based on the coverage tool and the policy type specified in the system configuration file. The policy classes are used to support more coverage formats in the future. The coverage formats include at least gcovr, llvm-cov, and lcov.

[0020] Furthermore, the S3 specifically includes the following steps:

[0021] S31, the tool class specifies Clang or Clang++ and enables a specific encoder option to perform encoding and stubbing on the source code; during the encoding and stubbing process, the compiler inserts additional code at the entry and exit of each basic block of the source code to record the number of executions of the basic block;

[0022] S32. After the source code is run, a .gcda file is generated in the working directory of the source code. The .gcda file is used to record the original coverage data during the source code run;

[0023] S33. Generate a coverage report in text format using the original coverage data. The coverage report includes the execution status of the source code, as well as line coverage, function coverage, and branch coverage.

[0024] Furthermore, the specific encoder option in S31 is a compiler option -fprofile-instr-generate-fcoverage-mapping in Clang.

[0025] Furthermore, the S4 specifically includes the following steps:

[0026] S41. Add Checker module and LLVM PASS module to the encoder. Checker module is used to provide static analysis tools for the encoder and enable the compiler to export complexity and call tree functions. LLVM PASS module is used to collect indicator data such as cyclomatic complexity, reachability, test depth, and call tree.

[0027] S42, enabling the complexity data generation function in the encoder, and then using the Checker module and the LLVM PASS module in the encoder to perform static analysis and indicator data collection on the source code, so as to obtain cyclomatic complexity and call tree information;

[0028] S43. Combine the function symbol table, parameter type data, basic blocks, and reachability data to obtain quantified analysis data for subsequent processing; the quantified analysis data is static analysis data.

[0029] Furthermore, the S5 specifically includes the following steps:

[0030] S51, based on the function symbol table and the corresponding code behavior benchmark, mapping and matching the original coverage data obtained in S3 with the actual source code file;

[0031] S52. Comprehensively analyze the original coverage data and static analysis data, and indicate various code coverage indicators, including line coverage, block coverage, branch coverage, and function coverage; various code coverage indicators are coverage sample data based on static analysis.

[0032] Furthermore, the S6 specifically includes the following steps:

[0033] S61, rendering the coverage sample data based on static analysis into an HTML page; the HTML page includes line coverage, block coverage, branch coverage, and function coverage of the source code displayed in a table form;

[0034] S62. Correspond the corresponding functions to various code coverage indicators one by one, render the coverage sample data on the call tree, and display the status of the coverage sample data on the call tree, then mark the unreached functions in red. Through the function name, you can jump to the corresponding function code and display the function coverage execution information.

[0035] The second aspect of the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above coverage generation and visualization method.

[0036] Beneficial effects of the present invention:

[0037] 1. The present invention discloses a coverage generation and visualization method based on static analysis. The core of the method is based on static analysis technology, which can analyze and identify unreasonable codes in the code, including overly complex functions, deeply nested conditional statements, etc. By reconstructing these codes, the system's operating performance can be improved, and the readability and maintainability of the code can also be improved.

[0038] 2. The coverage generation and visualization method disclosed in the present invention inherits all the advantages of traditional coverage generation methods and can output relevant coverage data such as code line coverage, block coverage, branch coverage and function coverage.

[0039] 3. The present invention can be used in the early stage of the project development cycle and can be integrated into the continuous integration and continuous deployment (CI / CD) process to automatically analyze the code and output highly readable reports to ensure that only high-quality code can enter the production environment, greatly reducing the time cost of the integration test phase and the system test phase. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flowchart of the coverage generation and visualization method in the present invention. DETAILED DESCRIPTION

[0041] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Preferred embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many other different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0042] Reference Figure 1 , the embodiment of the present application provides a coverage generation and visualization method based on static analysis, comprising the following steps:

[0043] S1. Receive the system configuration file and startup parameters input by the user, parse the system configuration file into internal data of the coverage tool, and then prepare the corresponding environment for coverage generation and static analysis based on the internal data of the coverage tool or through a pre-prepared image;

[0044] S2. Create a tool class based on the system configuration file and startup parameters;

[0045] S3. Call the tool class to encode and instrument the source code and obtain the original coverage data;

[0046] S4. Perform static analysis on the source code to obtain static analysis data;

[0047] S5. Comprehensively analyze the original coverage data and static analysis data to obtain coverage sample data based on static analysis;

[0048] S6. Generate an HTML file using the coverage tool and based on the coverage sample data to achieve visual display of coverage.

[0049] The core of the present invention is based on static analysis technology, which can evaluate the test coverage by in-depth analysis of code structure and logic without actually executing the code. This technology makes up for the shortcomings of traditional dynamic testing methods and provides a more efficient and comprehensive coverage evaluation method for software testing. In addition, by analyzing the cyclomatic complexity of the code structure and the call tree, a quantifiable evaluation method for code complexity and accessibility can be provided for software engineering.

[0050] In addition, the present invention can analyze and identify unreasonable codes in the code, including overly complex functions, deeply nested conditional statements, etc. By reconstructing these codes, the operating performance of the system can be improved, and the readability and maintainability of the code can also be improved.

[0051] In some embodiments, the system configuration file in S1 covers the specified compilation tool chain, programming language type, strategy type, HTTP information for pushing coverage files, and notification interface information; the strategy type is used to generate a tool chain for subsequent coverage processing, and different strategy types will specify different output formats in the processing logic, including the format conversion required for the corresponding output format, and the toolkit for generating the coverage format.

[0052] The startup parameters include input type, source code path, target name, corpus path, Git address and Token information; the input type is the type of source code source, including local source code, Git code library or other types; if the source code source type is local source code, the coverage tool runs as SideCar in S6; if the source code source type is Git code library or other types, before compiling, the source code needs to be dynamically pulled to the specified directory according to the system configuration file.

[0053] In some embodiments, the corresponding environment in S1 refers to the installation of a system software package.

[0054] In some embodiments, S2 specifically includes the following steps:

[0055] S21. Create a tool class based on the system configuration file and startup parameters; the tool class includes an LLVM tool chain or a GNU tool chain;

[0056] S22. Create corresponding policy classes based on the coverage tool and the policy type specified in the system configuration file. The purpose of designing the policy class is to support more coverage formats for possible subsequent needs. Coverage formats include gcovr, llvm-cov, lcov, etc. The current scenario mainly uses llvm-cov.

[0057] In some embodiments, S3 specifically includes the following steps:

[0058] S31. The tool class specifies Clang or Clang++ and enables specific encoder options to perform encoding and instrumentation on the source code. During the encoding and instrumentation process, an intermediate file .gcno is generated. These intermediate files .gcno contain metadata for each basic block in the program and additional code for tracking coverage. The compiler inserts additional code into the entry and exit of each basic block of the source code to record the number of executions of the basic block.

[0059] S32. After the source code is run, when the compiled program is executed, the additional code inserted by the compiler will record the execution times of each basic block. After the program is run, a .gcda file will be generated in the working directory of the source code. The .gcda file is used to record the original coverage data during the source code run;

[0060] S33. Generate a coverage report in text format using the original coverage data. The coverage report includes the execution status of the source code, as well as line coverage, function coverage, and branch coverage.

[0061] In some embodiments, the specific encoder option in S31 is a compiler option -fprofile-instr-generate-fcoverage-mapping in Clang.

[0062] In some embodiments, the S4 specifically includes the following steps:

[0063] S41. Add Checker module and LLVM PASS module to the encoder. Checker module is used to provide static analysis tools for the encoder and enable the compiler to export complexity and call tree functions. LLVM PASS module is used to collect indicator data such as cyclomatic complexity, reachability, test depth, and call tree.

[0064] S42, enabling the complexity data generation function in the encoder, and then using the Checker module and the LLVM PASS module in the encoder to perform static analysis and indicator data collection on the source code, so as to obtain cyclomatic complexity and call tree information;

[0065] S43. Combine the function symbol table, parameter type data, basic blocks, and reachability data to obtain quantified analysis data for subsequent processing; the quantified analysis data is static analysis data.

[0066] In some embodiments, S5 specifically includes the following steps:

[0067] S51, based on the function symbol table and the corresponding code behavior benchmark, mapping and matching the original coverage data obtained in S3 with the actual source code file;

[0068] S52. Comprehensively analyze the original coverage data and static analysis data, and indicate various code coverage indicators, including line coverage, block coverage, branch coverage, and function coverage; various code coverage indicators are coverage sample data based on static analysis.

[0069] In some embodiments, the S6 specifically includes the following steps:

[0070] S61, rendering the coverage sample data based on static analysis into an HTML page; the HTML page includes line coverage, block coverage, branch coverage, and function coverage of the source code displayed in a table form;

[0071] S62. Correspond the corresponding functions to various code coverage indicators one by one, render the coverage sample data on the call tree, and display the status of the coverage sample data on the call tree, then mark the unreached functions in red. Through the function name, you can jump to the corresponding function code and display the function coverage execution information.

[0072] The present invention supports packaging the generated coverage files and pushing them to the business platform through a standard HTTP interface for visual display.

[0073] The present invention supports instant calling of multiple notification channels such as email and DingTalk after the coverage generation task is successfully completed, and feeds back specific results of the coverage generation to relevant personnel.

[0074] The coverage generation and visualization method disclosed in the present invention inherits all the advantages of traditional coverage generation methods and can output relevant coverage data such as line coverage, block coverage, branch coverage and function coverage of the code.

[0075] The present invention can be used in the early stages of a project development cycle and can be integrated into the continuous integration and continuous deployment (CI / CD) process to automatically analyze the code and output highly readable reports, ensuring that only high-quality code can enter the production environment, thereby significantly reducing the time cost of the integration testing phase and the system testing phase.

[0076] The second aspect of the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above coverage generation and visualization method.

[0077] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention. In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A coverage generation and visualization method based on static analysis, characterized in that: The steps include: S1. Receive the system configuration file and startup parameters input by the user, parse the system configuration file into internal data of the coverage tool, and then prepare the corresponding environment for coverage generation and static analysis based on the internal data of the coverage tool or through a pre-prepared image; S2. Create a tool class based on the system configuration file and startup parameters; S3. Call the tool class to encode and instrument the source code and obtain the original coverage data; S4. Perform static analysis on the source code to obtain static analysis data; S5. Comprehensively analyze the original coverage data and static analysis data to obtain coverage sample data based on static analysis; S6. Generate an HTML file using the coverage tool and based on the coverage sample data to achieve visual display of coverage.

2. The coverage generation and visualization method according to claim 1, characterized in that: The system configuration file in S1 includes the specified compilation tool chain, programming language type, strategy type, HTTP information for pushing coverage files, and notification interface information; different strategy types will specify different output formats in the processing logic, including the format conversion required for the corresponding output format, and the toolkit for generating the coverage format; The startup parameters include input type, source code path, target name, corpus path, Git address and Token information; the input type is the type of source code source, including local source code, Git code library or other types; if the source code source type is local source code, the coverage tool runs as SideCar in S6; if the source code source type is Git code library or other types, before compiling, the source code needs to be dynamically pulled to the specified directory according to the system configuration file.

3. The coverage generation and visualization method according to claim 2, characterized in that: The corresponding environment in S1 refers to the installation of the system software package.

4. The coverage generation and visualization method according to claim 2, characterized in that: The S2 specifically includes the following steps: S21. Create a tool class based on the system configuration file and startup parameters; the tool class includes an LLVM tool chain or a GNU tool chain; S22. Create corresponding policy classes based on the coverage tool and the policy type specified in the system configuration file. The policy classes are used to support more coverage formats in the future. The coverage formats include at least gcovr, llvm-cov, and lcov.

5. The coverage generation and visualization method according to claim 4, characterized in that: The S3 specifically includes the following steps: S31, the tool class specifies Clang or Clang++ and enables a specific encoder option to perform encoding and stubbing on the source code; during the encoding and stubbing process, the compiler inserts additional code at the entry and exit of each basic block of the source code to record the number of executions of the basic block; S32. After the source code is run, a .gcda file is generated in the working directory of the source code. The .gcda file is used to record the original coverage data during the source code run; S33. Generate a coverage report in text format using the original coverage data. The coverage report includes the execution status of the source code, as well as line coverage, function coverage, and branch coverage.

6. The coverage generation and visualization method according to claim 5, characterized in that: The specific encoder option in the S31 is the compiler option -fprofile-instr-generate-fcoverage-mapping in Clang.

7. The coverage generation and visualization method according to claim 5, characterized in that: The S4 specifically includes the following steps: S41. Add Checker module and LLVM PASS module to the encoder. Checker module is used to provide static analysis tools for the encoder and enable the compiler to export complexity and call tree functions. LLVM PASS module is used to collect indicator data such as cyclomatic complexity, reachability, test depth, and call tree. S42, enabling the complexity data generation function in the encoder, and then using the Checker module and the LLVMPASS module in the encoder to perform static analysis and indicator data collection on the source code, so as to obtain cyclomatic complexity and call tree information; S43. Combine the function symbol table, parameter type data, basic blocks, and reachability data to obtain quantified analysis data for subsequent processing; the quantified analysis data is static analysis data.

8. The coverage generation and visualization method according to claim 7, characterized in that: The S5 specifically includes the following steps: S51, based on the function symbol table and the corresponding code behavior benchmark, mapping and matching the original coverage data obtained in S3 with the actual source code file; S52. Comprehensively analyze the original coverage data and static analysis data, and indicate various code coverage indicators, including line coverage, block coverage, branch coverage, and function coverage; various code coverage indicators are coverage sample data based on static analysis.

9. The coverage generation and visualization method according to claim 8, characterized in that: The S6 specifically includes the following steps: S61, rendering the coverage sample data based on static analysis into an HTML page; the HTML page includes line coverage, block coverage, branch coverage, and function coverage of the source code displayed in a table form; S62. Correspond the corresponding functions to various code coverage indicators one by one, render the coverage sample data on the call tree, and display the status of the coverage sample data on the call tree, then mark the unreached functions in red. Through the function name, you can jump to the corresponding function code and display the function coverage execution information.

10. An electronic device, characterized in that: include: at least one processor; And a memory communicatively connected to at least one processor; wherein the memory stores instructions executable by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute the coverage generation and visualization method described in any one of claims 1 to 9.

Images