A defect detection system based on unit test case transformation

Abstract

The present invention provides a defect detection system based on unit test case conversion, including: a case analysis module, which is used to automatically explore the target function and its unit test case, perform reachability analysis on the target function, and confirm the dependency and parameters of the unit test case; a driver generation module, which is used to perform fuzzy target selection on the unit test case and automatically generate a fuzzy test driver; a defect detection module, which is used to implement fuzzy testing at the function level, and mutate the parameters of the fuzzy test driver in the fuzzy test sub-engine through the operation management of the fuzzy test sub-engine; a data analysis module, which is used to perform statistics and defect information analysis on the test results. The defect detection system based on unit test case conversion provided by the present invention realizes automated fuzzy testing by converting the unit test case into a fuzzy test driver, thereby discovering its code defects in the unit test stage and effectively reducing the development and testing costs.

Description

Technical Field

[0001] The embodiment of the present invention relates to the field of software development security technology, and in particular to a defect detection system based on unit test case conversion. Background Art

[0002] Industries with high requirements for code quality and safety, such as smart cars and commercial aircraft, have industry standard requirements for unit testing, such as ISO 26262, DO-178C, etc. These industry standard requirements are mainly related to indicators of code coverage, and lack requirements for defect detection. This has led to companies unilaterally pursuing high coverage and ignoring the detection of code defects, which is contrary to the original intention of industry standard formulation to improve code quality and safety.

[0003] Traditional integration testing can detect code defects, but the relevant tools have a low degree of automation and require manual writing of test cases based on code design requirements. Some semi-automated tools can generate low-quality test cases, but they still require manual screening and modification. There are problems such as low efficiency in case generation, long cycle, and excessively high labor costs.

[0004] The patent with publication number CN117574388A discloses a method for generating fuzzy tests directly through parameter variables of the program to be tested. This method can improve the coverage and efficiency of fuzzy tests in the integrated testing phase. However, this method is heavily dependent on the usage specifications of the program to be tested, and has high requirements for the writing specifications of the program to be tested, which increases the manpower and time costs of development. It has low practical application feasibility and cannot truly solve the above-mentioned defect detection and other problems.

[0005] Therefore, it is necessary to provide a defect detection system based on unit test case conversion to solve the above problems. Summary of the invention

[0006] The present invention provides a defect detection system based on unit test case conversion, which realizes automated fuzzy testing by converting unit test cases into fuzzy test drivers, thereby discovering code defects in the unit test stage and effectively reducing development and testing costs.

[0007] An embodiment of the present invention provides a defect detection system based on unit test case conversion, including:

[0008] A use case analysis module, which is used to automatically explore the target function and its unit test cases, perform reachability analysis on the target function, perform full or incremental analysis on the unit test cases, and confirm the dependencies and parameters of the unit test cases; a driver generation module, which is used to perform fuzzy target selection on the unit test cases, automatically generate fuzzy test drivers, and generate fuzzy test drivers through fuzzy strategies by replacing the identified fuzzy targets with fuzzy input assignment statements; it is also used to complete the drivers for the program under test for which no driver has been generated, determine the drivers that need to be completed based on the test results of the defect detection module and the information of the program under test of the use case analysis module, and complete the drivers for functions with no parameter input, parameter processing failure, and constant parameters; it is also used to handle various special scenarios, including mutation processing strategies for configuration files, no mutation of specific parameters configured by users or mutation within a specified range, and special processing of dependent files;

[0009] A defect detection module is used to implement fuzzy testing at the function level. By running and managing the fuzzy test sub-engine, the module receives the data generated by the use case analysis module and the driver generation module, mutates the parameters of the fuzzy test driver in the fuzzy test sub-engine, and implements a sufficient traversal of the target function to obtain a test result. The test result includes: detecting memory overflow problems, memory error problems, type conversion problems, and undefined behavior problems;

[0010] The data analysis module is used to perform statistics and defect information analysis on the test results, wherein the defect information analysis includes: defect quantity statistics, defect type information analysis, defect location information analysis, and defect log information analysis.

[0011] Preferably, the use case analysis module includes:

[0012] The code detection unit is used to connect to CI / CD tools, detect the user's updated code through Jenkins, identify the code update content, and automatically import the target program code and corresponding unit test cases;

[0013] A page operation unit, which is used to provide a project management interface and a task generation interface, receive user input, and implement the target program file import and test parameter configuration;

[0014] A reachability analysis unit is used to perform reachability analysis on the target program code, including reachability analysis of full code and incremental code, by processing the target function, generating an XML format structure and parsing it, obtaining all entities and their relationships of the target program code, calculating all reachable paths from the target program code entry to any statement or branch, and obtaining information related to the reachable code;

[0015] The use case analysis unit is used to analyze the structure of the unit test case, obtain the combined API interface Setup, TearDown and the specific test content TestBody through interface name matching and analyze them. The analysis content includes dependency analysis between use cases, configuration file analysis, parameter variation analysis, and adaptation of compilation options and output files.

[0016] Preferably, the drive generation module includes:

[0017] A parameter attribute analysis unit, which is used for the structural information of the unit test case to analyze whether the target function has relevant parameter attributes;

[0018] The variation control strategy unit is used to determine the variation range and state of different parameters to handle various special scenarios in the drive generation process;

[0019] A fuzzy target selection unit, which is used to find code statements in the unit test case that can be directly used to drive generation;

[0020] A driver generation unit, which is used to convert the unit test case into a fuzz test driver;

[0021] A driver completion unit, which is used to complete the driver of the program to be tested for which the driver is not generated;

[0022] The driver application unit is used to provide the generated driver to the defect detection module and the data analysis module for use.

[0023] Preferably, the parameter attributes include write and read pointer parameters, array parameters, parameters involved in loops, loop exit judgment parameters and specific parameter tags.

[0024] Preferably, the determination of the variation range and status of different parameters includes variation processing strategies for configuration files, no variation or variation within a specified range of specific parameters configured by the user, special processing of dependent files, compilation and output adaptation, constant variation strategies, and control of memory allocation and input range of loop parameters.

[0025] Preferably, finding the code statements in the unit test case that can be directly used to drive generation includes finding the root parameters, starting from the call point, backtracking to find the root parameters according to parameter attribute information and dependencies, mutating the root parameters, and traversing various behaviors of the target function.

[0026] Preferably, converting the unit test case into a fuzzy test driver comprises:

[0027] Provide assignment for the fuzzy target, and exclude inapplicable root parameters according to the mutation strategy and parameter type. The inapplicable root parameters include: root parameters in header files, constants in compilation, and function pointer parameters;

[0028] Build the corresponding test entry for the fuzz test loop, which is used to call the test statements in sequence in the subsequent fuzz test;

[0029] Get the constant value of the fuzz target to explore the deep program state in the early stage of fuzz testing.

[0030] Preferably, the step of completing the driver for the program to be tested for which the driver is not generated comprises:

[0031] The driver that needs to be completed is determined based on the generation results of the defect detection module and all the program information to be tested of the use case analysis module, and the driver is generated for the function with no parameter input, parameter processing failure, or constant parameter.

[0032] Preferably, the defect detection module comprises:

[0033] An information interaction unit, which is used to interact with the use case analysis module and the driver generation module, obtain the test target program, unit test cases, test configuration information, fuzzy test driver and fuzzy parameters, and provide them to the fuzzy test execution unit and the unit test execution unit to carry out testing;

[0034] The fuzz test execution unit is used to execute the fuzz test process and detect defects in the code. It obtains the corresponding test environment information by parsing the imported test and configuration files, and executes the test task after completing the configuration. It supports the detection of memory overflow problems, memory error problems, type conversion problems, and undefined behavior problems.

[0035] The unit test execution unit is used to execute unit tests, compare the defect detection and coverage improvement results of fuzz testing with the specific use case execution process, provide a reference benchmark for fuzz test optimization and defect detection, parse the unit test cases, and execute tests.

[0036] Preferably, the data analysis module includes:

[0037] A data interface unit, which is used to receive data content from the defect detection module and the driver generation module, classify and manage the data, collect compilation and operation log information of the test process, and provide defect information for the defect analysis unit;

[0038] Defect analysis unit, which is used to collect test result logs, test function codes, test case files, analyze defect types, evaluate defect levels, determine defect causes, locate and annotate defects, and provide stack information, test drivers, and test cases for the defect detection process;

[0039] The defect repair unit is used to provide repair suggestions for detected defects, match the defect type, defect level, and stack information with the accumulated defect type library, and obtain the corresponding defect cause description, repair reference code, and coding suggestions;

[0040] The coverage analysis unit is used to receive test logs, test codes, and test samples, and to count function coverage, line coverage, and branch coverage, thereby realizing coverage statistics at the project level, suite level, and code level.

[0041] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:

[0042] A defect detection system based on unit test case conversion provided by an embodiment of the present invention comprises: a case analysis module, which is used to automatically explore a target function and its unit test cases, perform reachability analysis on the target function, perform full or incremental analysis on the unit test cases, and confirm the dependency and parameters of the unit test cases; a driver generation module, which is used to perform fuzzy target selection on the unit test cases, automatically generate a fuzzy test driver, and generate a fuzzy test driver through a fuzzy strategy by replacing the identified fuzzy target with a fuzzy input assignment statement; the module is also used to complete the driver of a program to be tested for which a driver has not been generated, determine the driver to be completed according to the test results of the defect detection module and the information of the program to be tested of the case analysis module, and complete the driver for functions with no parameter input, parameter processing failure, and constant parameters; It is also used to handle various special scenarios, including mutation processing strategies for configuration files, not mutating specific parameters configured by users or mutating them within a specified range, and special processing of dependent files; a defect detection module, which is used to implement fuzzy testing at the function level, receives data generated by the use case analysis module and the driver generation module through the operation management of the fuzzy test sub-engine, mutates the parameters of the fuzzy test driver in the fuzzy test sub-engine, and implements a sufficient traversal of the target function to obtain test results, and the test results include: detecting memory overflow problems, memory error problems, type conversion problems, and undefined behavior problems; a data analysis module, which is used to perform statistics and defect information analysis on the test results, and the defect information analysis includes: defect quantity statistics, defect type information analysis, defect location information analysis, and defect log information analysis. By converting unit test cases into fuzzy test drivers, automated fuzzy testing is implemented, so that code defects are discovered during the unit test phase, effectively reducing development and testing costs;

[0043] Furthermore, by connecting to CI / CD tools, the full and incremental programs to be tested are automatically tested, the programs to be tested and the test cases are automatically distinguished and identified, and the unreachable codes are analyzed to define the scope of subsequent tests.

[0044] Furthermore, by identifying and analyzing the basic information of unit test cases, including: test case identification, case structure analysis, case dependency, configuration file analysis, compilation options and output file adaptation, we prepare for driver generation;

[0045] Furthermore, by analyzing the attributes of the use case parameters, we can find the fuzz test target parameters, guide the generation of fuzz test drivers through the mutation control strategy, and complete the functions without drivers through the forced generation strategy, so as to achieve higher coverage and defect detection effect;

[0046] Furthermore, defect detection is achieved through the mutation testing process of fuzz testing, including test file parsing, environment preparation, equipment scheduling and test execution, continuous collection of test process and result data, and support for executing unit test case tests as a comparison of the effect of fuzz testing defect detection;

[0047] Furthermore, the analysis of defects and coverage is realized. The defect detection result analysis includes the defect type, defect level, defect cause, defect location analysis, and provides repair suggestions. The coverage supports coverage statistics at the project level, suite level, and function level. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present invention, not all embodiments. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0049] Figure 1 A schematic diagram of the structure of a defect detection system based on unit test case conversion provided by an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of the structure of a use case analysis module of a defect detection system based on unit test case conversion provided by an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of the structure of a driver generation module of a defect detection system based on unit test case conversion provided by an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the structure of a defect detection module of a defect detection system based on unit test case conversion provided by an embodiment of the present invention;

[0053] Figure 5 A schematic diagram of the structure of a data analysis module of a defect detection system based on unit test case conversion provided by an embodiment of the present invention;

[0054] Figure 6 A flowchart of a method for using a defect detection system based on unit test case conversion provided by an embodiment of the present invention;

[0055] Figure 7 A schematic diagram of an implementation of a defect detection system based on unit test case conversion is provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0058] Based on the problems existing in the prior art, an embodiment of the present invention provides a defect detection system based on unit test case conversion. By converting unit test cases into fuzzy test drivers, automated fuzz testing is implemented, thereby discovering code defects in the unit testing stage and effectively reducing development and testing costs.

[0059] Figure 1 A schematic diagram of a defect detection system based on unit test case conversion is provided for one embodiment of the present invention. Figure 1 , providing a defect detection system based on unit test case conversion, including:

[0060] A use case analysis module, which is used to automatically explore the target function and its unit test cases, perform reachability analysis on the target function, perform full or incremental analysis on the unit test cases, and confirm the dependencies and parameters of the unit test cases;

[0061] A driver generation module is used to select fuzzy targets for the unit test cases, automatically generate fuzzy test drivers, and generate fuzzy test drivers through fuzzy strategies by replacing the identified fuzzy targets with fuzzy input assignment statements; it is also used to complete the drivers for the program to be tested for which no driver has been generated, determine the drivers that need to be completed based on the test results of the defect detection module and the information of the program to be tested of the case analysis module, and complete the drivers for functions with no parameter input, parameter processing failure, and constant parameters; it is also used to handle various special scenarios, including mutation processing strategies for configuration files, no mutation of specific parameters configured by users or mutation within a specified range, and special processing of dependent files;

[0062] A defect detection module is used to implement fuzzy testing at the function level. By running and managing the fuzzy test sub-engine, the module receives the data generated by the use case analysis module and the driver generation module, mutates the parameters of the fuzzy test driver in the fuzzy test sub-engine, and implements a sufficient traversal of the target function to obtain a test result. The test result includes: detecting memory overflow problems, memory error problems, type conversion problems, and undefined behavior problems;

[0063] The data analysis module is used to perform statistics and defect information analysis on the test results, wherein the defect information analysis includes: defect quantity statistics, defect type information analysis, defect location information analysis, and defect log information analysis.

[0064] Specifically, the use cases analyzed by the use case analysis module can be used to generate fuzzy test drivers for the driver generation module, and support direct unit testing in the defect detection module, providing reference and comparison for data analysis and statistics.

[0065] The driver generation module can also analyze drivers that fail to generate due to missing or no parameter input, confirm the target program function that needs to generate the driver through the use case analysis module, and improve the success rate of driver generation through automatic completion and forced generation. At the same time, the driver generation module provides the initial seed for the defect detection module and provides driver statistics and details at different levels in the data analysis module. In addition, the driver generation module supports obtaining parameter variation conditions configured on the system interface, and does not mutate the function driver generation implementation or mutates it within the specified range, thereby effectively avoiding problems such as underlying code crashes in special scenarios.

[0066] The defect detection module also supports the detection of other defect problems. The results generated by the defect detection module will be analyzed and displayed by the data analysis module.

[0067] The data analysis module supports code coverage and line coverage statistics at the project level, suite level, and function level, and supports marking code coverage using red, green, and blue colors.

[0068] This system is installed locally on the user's computer. The user interacts with the server through a browser. The entire analysis and testing process does not rely on manual labor, which can greatly improve the efficiency of code defect detection.

[0069] Figure 2 A schematic diagram of the structure of a use case analysis module of a defect detection system based on unit test case conversion is provided for one embodiment of the present invention. Figure 2 , the use case analysis module includes:

[0070] The code detection unit is used to connect to CI / CD tools, detect the user's updated code through Jenkins, identify the code update content, and automatically import the target program code and corresponding unit test cases;

[0071] A page operation unit, which is used to provide a project management interface and a task generation interface, receive user input, and implement the target program file import and test parameter configuration;

[0072] A reachability analysis unit is used to perform reachability analysis on the target program code, including reachability analysis of full code and incremental code, by processing the target function, generating an XML format structure and parsing it, obtaining all entities and their relationships of the target program code, calculating all reachable paths from the target program code entry to any statement or branch, and obtaining information related to the reachable code;

[0073] The use case analysis unit is used to analyze the structure of the unit test case, obtain the combined API interface Setup, TearDown and the specific test content TestBody through interface name matching and analyze them. The analysis content includes dependency analysis between use cases, configuration file analysis, parameter variation analysis, and adaptation of compilation options and output files.

[0074] Specifically, the page operation unit also supports the import of other operation instructions and passes them to the reachability analysis unit and other modules. The page operation unit page can display information such as code and use cases, and support the management and operation of projects and tasks.

[0075] The use case analysis unit provides the basic conditions for running unit test cases and generating fuzzy test drivers.

[0076] Figure 3 A schematic diagram of the structure of a drive generation module of a defect detection system based on unit test case conversion is provided for one embodiment of the present invention. Figure 3 , the driver generation module includes:

[0077] A parameter attribute analysis unit, which is used for the structural information of the unit test case to analyze whether the target function has relevant parameter attributes;

[0078] The variation control strategy unit is used to determine the variation range and state of different parameters to handle various special scenarios in the drive generation process;

[0079] A fuzzy target selection unit, which is used to find code statements in the unit test case that can be directly used to drive generation;

[0080] A driver generation unit, which is used to convert the unit test case into a fuzz test driver;

[0081] A driver completion unit, which is used to complete the driver of the program to be tested for which the driver is not generated;

[0082] The driver application unit is used to provide the generated driver to the defect detection module and the data analysis module for use.

[0083] Specifically, the parameter attribute analysis unit mainly provides basic information for the tasks of subsequent drive generation units.

[0084] The variation control strategy unit can handle various special scenarios in the drive generation process and improve the automation level of the system.

[0085] The driver completion unit solves the problem that some programs under test do not have unit test cases or the cases cannot be converted into fuzz test drivers.

[0086] The driver application unit classifies and manages the generated drivers to ensure that they can be efficiently acquired and distributed to the defect detection module when needed. It also manages the driver metadata to facilitate the retrieval, display and management of the data analysis module.

[0087] In a specific implementation, the parameter attributes include write and read pointer parameters, array parameters, parameters involved in the loop, loop exit judgment parameters and specific parameter tags.

[0088] In a specific implementation, the determination of the variation range and status of different parameters includes variation processing strategies for configuration files, no variation of specific parameters configured by the user or variation within a specified range, special processing of dependent files, compilation and output adaptation, constant variation strategies, and control of the input range of memory allocation and loop parameters.

[0089] In a specific implementation, finding the code statements in the unit test case that can be directly used to drive generation includes finding the root parameters, starting from the call point, backtracking to find the root parameters based on parameter attribute information and dependencies, mutating the root parameters, and traversing various behaviors of the target function.

[0090] In a specific implementation, converting the unit test case into a fuzzy test driver includes:

[0091] Provide assignment for the fuzzy target, and exclude inapplicable root parameters according to the mutation strategy and parameter type. The inapplicable root parameters include: root parameters in header files, constants in compilation, and function pointer parameters;

[0092] Build the corresponding test entry for the fuzz test loop, which is used to call the test statements in sequence in the subsequent fuzz test;

[0093] Get the constant value of the fuzz target to explore the deep program state in the early stage of fuzz testing.

[0094] In a specific implementation, the step of completing the driver for the program to be tested for which the driver is not generated includes:

[0095] The driver that needs to be completed is determined based on the generation results of the defect detection module and all the program information to be tested of the use case analysis module, and the driver is generated for the function with no parameter input, parameter processing failure, or constant parameter.

[0096] Figure 4 A schematic diagram of the structure of a defect detection module of a defect detection system based on unit test case conversion is provided for one embodiment of the present invention. Figure 4 , the defect detection module comprises:

[0097] An information interaction unit, which is used to interact with the use case analysis module and the driver generation module, obtain the test target program, unit test cases, test configuration information, fuzzy test driver and fuzzy parameters, and provide them to the fuzzy test execution unit and the unit test execution unit to carry out testing;

[0098] The fuzz test execution unit is used to execute the fuzz test process and detect defects in the code. It obtains the corresponding test environment information by parsing the imported test and configuration files, and executes the test task after completing the configuration. It supports the detection of memory overflow problems, memory error problems, type conversion problems, and undefined behavior problems.

[0099] The unit test execution unit is used to execute unit tests, compare the defect detection and coverage improvement results of fuzz testing with the specific use case execution process, provide a reference benchmark for fuzz test optimization and defect detection, parse the unit test cases, and execute tests.

[0100] Specifically, the information interaction unit is also used to schedule the test equipment according to the equipment information in the configuration file to implement parallel execution of the test operations.

[0101] The fuzz test execution unit also supports the detection of other defect problems. At the same time, according to the device information in the configuration file, the test equipment is scheduled and the test operations are performed in parallel. In addition, the fuzz test execution unit continues to collect information during the test and sends it to the data analysis module through the information interaction unit for further analysis, summary and display.

[0102] The unit test execution unit is also used to collect the corresponding test process and result information after the unit test is completed, and send it to the data analysis module for further comparison and analysis.

[0103] Figure 5 A schematic diagram of the structure of a data analysis module of a defect detection system based on unit test case conversion is provided for one embodiment of the present invention. Figure 5 , the data analysis module includes:

[0104] A data interface unit, which is used to receive data content from the defect detection module and the driver generation module, classify and manage the data, collect compilation and operation log information of the test process, and provide defect information for the defect analysis unit;

[0105] Defect analysis unit, which is used to collect test result logs, test function codes, test case files, analyze defect types, evaluate defect levels, determine defect causes, locate and annotate defects, and provide stack information, test drivers, and test cases for the defect detection process;

[0106] The defect repair unit is used to provide repair suggestions for detected defects, match the defect type, defect level, and stack information with the accumulated defect type library, and obtain the corresponding defect cause description, repair reference code, and coding suggestions;

[0107] The coverage analysis unit is used to receive test logs, test codes, and test samples, and to count function coverage, line coverage, and branch coverage, thereby realizing coverage statistics at the project level, suite level, and code level.

[0108] Specifically, the system not only supports defect detection of the program under test, but also supports the detection of defects in unit test cases to ensure the reliability of the test code itself.

[0109] The coverage analysis unit can also display the function coverage of each program under test, using red, green and blue colors to mark uncovered, unit test case covered and fuzzy test covered, making the coverage effect clear and explicit.

[0110] Figure 6 A flowchart of a method for using a defect detection system based on unit test case conversion provided by an embodiment of the present invention; Figure 7A schematic diagram of an implementation of a defect detection system based on unit test case conversion provided by an embodiment of the present invention. Figure 6 and Figure 7 ,The defect detection process of the defect detection system based on unit test case conversion is as follows:

[0111] 1) The system automatically reads the information of the program to be tested, creates a test task and reads related files. The test task can also be created manually in the system. The creation of a test task mainly includes: the compressed package of the program code to be tested, test cases, configuration files, and fuzzy parameter ranges, etc., among which the fuzzy parameter range is not required.

[0112] 2) The system automatically analyzes the program code and test cases to be tested, including:

[0113] a) Reachability analysis: determine which codes are reachable and testable, and clarify the test scope. Unreachable codes are thrown out and no longer tested.

[0114] b) Determine the use case information: including the use case dependencies, parameters, etc. For the program functions to be tested that are missing use cases, they can be directly completed by the driver in 5).

[0115] 3) Parameter attribute analysis to identify the parameters that can be used for variation and related information.

[0116] 4) Fuzzy target selection: the system looks for root parameters that can be used to drive generation as the basis for mutation.

[0117] 5) Driver generation: Generate test drivers based on parameters and other information. Some drivers require mutation strategy restrictions, and these drivers will be generated according to the mutation strategy requirements. Some programs under test lack drivers, and the driver completion strategy will be used to force the generation of drivers to cover the corresponding code.

[0118] 6) Code defect detection: Code defect detection is achieved through fuzz testing. It also supports the detection of test codes and directly runs unit test cases for comparison.

[0119] 7) Defect analysis: receive error information and operation information collected during the test process, analyze the defects and give analysis results, including defect type, defect level, defect location, defect repair suggestions, test samples corresponding to the defects, etc.

[0120] 8) Coverage statistics: Calculate coverage information at different levels, coverage code details, etc.

[0121] 9) Generate a test report using the analysis and statistical results obtained in 7) and 8).

[0122] In summary, a defect detection system based on unit test case conversion according to an embodiment of the present invention includes: a use case analysis module, which is used to automatically explore a target function and its unit test cases, perform reachability analysis on the target function, perform full or incremental analysis on the unit test cases, and confirm the dependencies and parameters of the unit test cases; a driver generation module, which is used to perform fuzzy target selection on the unit test cases, automatically generate a fuzzy test driver, and generate a fuzzy test driver through a fuzzy strategy by replacing the identified fuzzy target with a fuzzy input assignment statement; it is also used to complete the driver of a program to be tested for which a driver has not been generated, determine the driver to be completed according to the test results of the defect detection module and the information of the program to be tested of the use case analysis module, and complete the driver for functions with no parameter input, parameter processing failure, and constant parameters; It is also used to handle various special scenarios, including mutation processing strategies for configuration files, not mutating specific parameters configured by users or mutating them within a specified range, and special processing of dependent files; a defect detection module, which is used to implement fuzzy testing at the function level, receives data generated by the use case analysis module and the driver generation module through the operation management of the fuzzy test sub-engine, mutates the parameters of the fuzzy test driver in the fuzzy test sub-engine, and implements a sufficient traversal of the target function to obtain test results, and the test results include: detecting memory overflow problems, memory error problems, type conversion problems, and undefined behavior problems; a data analysis module, which is used to perform statistics and defect information analysis on the test results, and the defect information analysis includes: defect quantity statistics, defect type information analysis, defect location information analysis, and defect log information analysis. By converting unit test cases into fuzzy test drivers, automated fuzzy testing is implemented, so that code defects are discovered during the unit test phase, effectively reducing development and testing costs;

[0123] Furthermore, by connecting to CI / CD tools, the full and incremental programs to be tested are automatically tested, the programs to be tested and the test cases are automatically distinguished and identified, and the unreachable codes are analyzed to define the scope of subsequent tests.

[0124] Furthermore, by identifying and analyzing the basic information of unit test cases, including: test case identification, case structure analysis, case dependency, configuration file analysis, compilation options and output file adaptation, we prepare for driver generation;

[0125] Furthermore, by analyzing the attributes of the use case parameters, we can find the fuzz test target parameters, guide the generation of fuzz test drivers through the mutation control strategy, and complete the functions without drivers through the forced generation strategy, so as to achieve higher coverage and defect detection effect;

[0126] Furthermore, defect detection is achieved through the mutation testing process of fuzz testing, including test file parsing, environment preparation, equipment scheduling and test execution, continuous collection of test process and result data, and support for executing unit test case tests as a comparison of the effect of fuzz testing defect detection;

[0127] Furthermore, the analysis of defects and coverage is realized. The defect detection result analysis includes the defect type, defect level, defect cause, defect location analysis, and provides repair suggestions. The coverage supports coverage statistics at the project level, suite level, and function level.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A defect detection system based on unit test case conversion, characterized in that: include: A use case analysis module, which is used to automatically explore the target function and its unit test cases, perform reachability analysis on the target function, perform full or incremental analysis on the unit test cases, and confirm the dependencies and parameters of the unit test cases; The use case analysis module includes: a code detection unit, which is used to connect to CI / CD tools, detect the user-updated code through Jenkins, identify the code update content, and automatically import the target program code and the corresponding unit test cases; a page operation unit, which is used to provide a project management interface and a task generation interface, receive user input, and realize the import of the target program file and test parameter configuration; a reachability analysis unit, which is used to perform reachability analysis on the target program code, including reachability analysis of full code and incremental code, by processing the target function, generating an XML format structure and parsing it, obtaining all entities and their relationships of the target program code, calculating all reachable paths from the target program code entry to any statement or branch path, and obtaining reachable code related information; a use case analysis unit, which is used to analyze the structure of the unit test case, obtain the combined API interface Setup, TearDown and the specific test content TestBody through interface name matching and analyze them, and the analysis content includes dependency analysis between use cases, configuration file analysis, parameter variation analysis, and adaptation of compilation options and output files; A driver generation module is used to select fuzzy targets for the unit test cases, automatically generate fuzzy test drivers, and generate fuzzy test drivers through fuzzy strategies by replacing the identified fuzzy targets with fuzzy input assignment statements; it is also used to complete the drivers for the program to be tested for which no driver has been generated, determine the drivers that need to be completed based on the test results of the defect detection module and the program to be tested information of the case analysis module, and complete the drivers for functions with no parameter input, parameter processing failure, and constant parameters; It is also used to process various special scenarios, including mutation processing strategies for configuration files, no mutation or mutation within a specified range for specific parameters configured by users, and special processing for dependent files; the driver generation module includes: a parameter attribute analysis unit, which is used for the structural information of the unit test case to analyze whether the target function has relevant parameter attributes; a mutation control strategy unit, which is used to determine the mutation range and state of different parameters to achieve processing of various special scenarios in the driver generation process; a fuzzy target selection unit, which is used to find code statements in the unit test case that can be directly used for driver generation; a driver generation unit, which is used to convert the unit test case into a fuzzy test driver; a driver completion unit, which is used to complete the driver of the program to be tested for which the driver is not generated; a driver application unit, which is used to provide the generated driver to the defect detection module and the data analysis module for use; The defect detection module is used to implement fuzzy testing at the function level. By operating and managing the fuzzy test sub-engine, the data generated by the use case analysis module and the driver generation module are received, and the parameters of the fuzzy test driver are mutated in the fuzzy test sub-engine to implement sufficient traversal of the target function to obtain the test result. The test result includes: detecting memory overflow problems, memory error problems, type conversion problems, and undefined behavior problems; the defect detection module includes: an information interaction unit, which is used to interact with the use case analysis module and the driver generation module to obtain the test target program, unit test cases, test configuration information, fuzzy test driver and fuzzy parameters. The number is provided to the fuzz test execution unit and the unit test execution unit for testing; the fuzz test execution unit is used to execute the fuzz test process, detect defects in the code, obtain corresponding test environment information by parsing the imported test and configuration files, and execute the test task after completing the configuration, supporting the detection of memory overflow problems, memory error problems, type conversion problems, and undefined behavior problems; the unit test execution unit is used to execute unit testing, compare the defect detection and coverage improvement results of fuzz testing with the specific use case execution process, provide a reference benchmark for fuzz test optimization and defect detection, parse the unit test cases, and execute the test; A data analysis module is used to perform statistics and defect information analysis on the test results, and the defect information analysis includes: defect quantity statistics, defect type information analysis, defect location information analysis, and defect log information analysis; the data analysis module includes: a data interface unit, which is used to receive the data content of the defect detection module and the driver generation module, classify and manage the data, collect the compilation and operation log information of the test process, and provide defect information for the defect analysis unit; the defect analysis unit is used to collect test result logs, test function codes, and test case files, analyze defect types, evaluate defect levels, determine the causes of defects, achieve defect location and annotation, and provide stack information, test drivers, and use cases of the defect detection process; a defect repair unit is used to provide repair suggestions for detected defects, match the defect type, defect level, and stack information with the accumulated defect type library, and obtain the corresponding defect cause description, repair reference code, and coding suggestions; a coverage analysis unit is used to receive test logs, test codes, and test samples, and count function coverage, line coverage, and branch coverage to achieve coverage statistics at the project level, suite level, and code level.

2. The defect detection system based on unit test case conversion according to claim 1, characterized in that: The parameter attributes include write and read pointer parameters, array parameters, parameters involved in loops, loop exit judgment parameters and specific parameter tags.

3. The defect detection system based on unit test case conversion according to claim 1, characterized in that: The determination of the variation range and status of different parameters includes variation processing strategies for configuration files, no variation or variation within a specified range of specific parameters configured by the user, special processing of dependent files, compilation and output adaptation, constant variation strategies, and control of memory allocation and input range of loop parameters.

4. The defect detection system based on unit test case conversion according to claim 1 is characterized in that: The method of finding the code statements in the unit test case that can be directly used to drive generation includes finding the root parameter, starting from the call point, backtracking to find the root parameter based on parameter attribute information and dependencies, mutating the root parameter, and traversing various behaviors of the target function.

5. The defect detection system based on unit test case conversion according to claim 1 is characterized in that: The converting the unit test case into a fuzz test driver comprises: Provide assignment for the fuzzy target, and exclude inapplicable root parameters according to the mutation strategy and parameter type. The inapplicable root parameters include: root parameters in header files, constants in compilation, and function pointer parameters; Build the corresponding test entry for the fuzz test loop, which is used to call the test statements in sequence in the subsequent fuzz test; Get the constant value of the fuzz target to explore the deep program state in the early stage of fuzz testing.

6. The defect detection system based on unit test case conversion according to claim 1 is characterized in that: The driver completion for the program to be tested for which the driver is not generated includes: The driver that needs to be completed is determined based on the generation results of the defect detection module and all the program information to be tested of the use case analysis module, and the driver is generated for the function with no parameter input, parameter processing failure, or constant parameter.

Images