Physical simulation engine defect detection method based on variation

Abstract

The invention belongs to the field of software testing and defect detection, focuses on reliability verification of a physical simulation engine, is suitable for scenes depending on physical simulation, such as robot simulation, automatic driving testing, unmanned aerial vehicle control, education and scientific research, and particularly relates to a variation-based physical simulation engine defect detection method. Potential defects of core modules of collision detection, mechanical calculation, dynamics and the like are efficiently found through a variation test method. According to the method, a feature library is constructed through static library API analysis and AST / IR code analysis, test code expansion is performed by utilizing a large model, a test driver adaptive to a specific data type is generated, a physical semantic constraint mutation seed is combined, a test is executed through libFuzzer, defect and coverage rate data are analyzed, a use case selection strategy is optimized, and the test efficiency is improved. Defects of core modules such as collision detection and dynamic solution are efficiently detected, and the test pertinence and efficiency are improved.

Description

Technical Field

[0001] This invention belongs to the field of software testing and defect detection, focusing on the reliability verification of physics simulation engines. It is applicable to scenarios that rely on physics simulation, such as robot simulation, autonomous driving testing, drone control, and education and scientific research. Specifically, it is a mutation-based defect detection method for physics simulation engines. It can efficiently discover potential defects in core modules such as collision detection, mechanical calculation, and dynamics for mainstream physics engines such as Bullet and DART through mutation testing. Background Technology

[0002] Physics simulation engines, as a core technological support connecting virtual simulation and real-world scenarios, have become an indispensable infrastructure in fields such as robot R&D, autonomous driving testing, drone control, industrial simulation, and education and research. Their core value lies in accurately reproducing phenomena such as motion laws, collision responses, and force transmission in the physical world through mathematical modeling and numerical calculations, significantly reducing the cost and risk of physical testing. For example, in the field of autonomous driving, simulation testing based on physics simulation engines can cover scenarios that are difficult to reproduce in reality, such as extreme weather and complex road conditions, providing an efficient verification environment for algorithm iteration. In robot R&D, engines can simulate key parameters such as joint dynamics and environmental interaction forces, assisting in the optimization of mechanical structures and control algorithms. Currently, the industry has formed an ecosystem dominated by open-source engines and supplemented by commercial engines. Mainstream open-source solutions include Bullet (known for its real-time performance and stability, widely used in game and robot simulation) and DART (focused on high-precision dynamics calculations, suitable for academic research and complex mechanical systems). These engines, with their flexible scalability and open-source characteristics, occupy the main market share in low-to-mid-range simulation scenarios.

[0003] However, the technical complexity and diverse application scenarios of physics simulation engines pose significant reliability challenges. From a technical architecture perspective, the core modules of the engine encompass key aspects such as collision detection, continuous mechanics calculation, dynamics / inverse mechanics solution, and constraint solution. These modules are tightly coupled: the collision detection module handles contact determination and collision force calculation for rigid / soft bodies, and its algorithm accuracy directly impacts subsequent mechanics simulation results; the dynamics solution module relies on complex numerical solutions to differential equations, which are susceptible to factors such as initial conditions and step size settings, potentially leading to numerical divergence or accumulated accuracy errors; and the constraint solution module must balance real-time performance and accuracy, making it prone to solution conflicts in multi-constraint scenarios. Furthermore, different application scenarios have significantly different requirements for the engine: industrial simulation demands micrometer-level accuracy, robot control requires millisecond-level response, and autonomous driving testing requires coverage of hundreds of millions of scenarios. This divergence in requirements necessitates that the engine support diverse parameter configurations and extended interfaces, further increasing the system's complexity.

[0004] Existing testing technologies are insufficient to meet the reliability verification requirements of physics simulation engines, and traditional testing methods have significant shortcomings. Manual testing and unit testing rely on the domain knowledge of testers, making it difficult to cover the massive combinations of engine parameters and complex scenario interactions, and are also inefficient and have poor repeatability; rule-based automated testing is limited by preset logic and cannot cope with unknown defects and edge scenarios. While fuzzing, as a mainstream automated defect detection technology, has achieved significant results in general software testing, it faces three core challenges in physics simulation engine testing: First, the flexibility of test driver generation is insufficient. Existing tools (such as Futag and Utopia) are mostly designed for general software libraries and are difficult to adapt to the specific data types (such as rigid body state vectors and joint constraint matrices) and API call paradigms of physics engines. The generated test programs often fail to effectively trigger the critical logic of core modules. Second, the test cases lack specificity. The seed mutations in traditional fuzzing are mostly random perturbations, lacking semantic understanding of the physical scene. Most of the generated test cases are invalid inputs, making it difficult to focus on high-risk areas and resulting in low testing efficiency. Third, the accuracy of defect judgment is insufficient. The output results of physics engines are continuous and fuzzy, making it difficult to judge defects through simple numerical comparisons. It is necessary to analyze the rationality of the physical scene, and existing tools lack such professional judgment mechanisms.

[0005] While industry research has attempted to address these issues, significant limitations remain. For example, the PhyFu tool, which designs a dedicated fuzzing framework for physics engines, lacks sufficient scalability in its test driver and struggles to adapt to multiple engine types. GraphFuzz optimizes seed mutation through data flow graph modeling but fails to adequately consider the semantic constraints of the physical scene. These studies have failed to systematically address the issues of flexibility, specificity, and accuracy in physics simulation engine testing, resulting in low engine defect detection rates, long testing cycles, and high resource consumption, thus failing to meet the stringent reliability requirements of industrial applications. Therefore, developing a highly adaptable, specific, and efficient physics simulation engine defect detection technology has become an urgent need for industry development and is of great significance for promoting high-quality development in fields such as robotics and autonomous driving that rely on simulation technology. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a mutation-based method for detecting defects in physics simulation engines. It constructs a feature library through static library API parsing and AST / IR code analysis, expands test code using large models, and generates test drivers adapted to specific data types. Combined with physics semantic constraint mutation seeds, tests are executed via libFuzzer, and defect and coverage data are analyzed to optimize test case selection strategies. This efficiently detects defects in core modules such as collision detection and dynamics solving, improving test targeting and efficiency.

[0007] The technical solution of this invention:

[0008] A mutation-based defect detection method for physics simulation engines, the specific steps of which are as follows:

[0009] The first phase is the target library analysis phase. Deep static analysis is performed on the target library's source code, and a physics simulation library is built to generate compilation logs and static libraries. The nm tool is used to extract parameter attribute information from API functions, and the library source code and unit test code are analyzed to generate Clang AST and LLVM IR. Multiple types of analyzers are used to comprehensively analyze each API parameter from multiple dimensions. Specific analysis types are as follows:

[0010] Parameter orientation analysis: Identify whether a parameter is an input, output, or input-output.

[0011] Memory allocation size analysis: Identify whether the parameter is used as the allocation size parameter in a memory allocation function (such as the malloc function).

[0012] Array analysis: Identify array parameters and their corresponding length parameters to determine whether the array is a variable-length array.

[0013] Constant analysis: Extract constant declarations from the AST, including constant values ​​of integer, floating-point, string, and array types.

[0014] File path analysis: Determine whether the parameter is used as a file path (such as when passed to file operation functions such as fopen and open).

[0015] Loop analysis: Analyzes whether function parameters participate in loop control, especially comparison operands used as loop exit conditions.

[0016] Parameter count analysis: Analyze the number of parameters and the starting position of the parameters in the function (consider the return value of the structure and the this pointer of the C++ member function).

[0017] Type analysis: Extract the enumeration type definition from the AST, including the enumeration name and all enumeration values.

[0018] These multi-dimensional analysis results provide rich semantic information for subsequent test-driven generation.

[0019] The second phase is the test code expansion phase. API functions not called by unit tests are extracted. Library code analysis data (Clang AST and LLVM IR) and vectorized library code are stored in a vector library. By analyzing the unit test code of the physics simulation engine, API call patterns, parameter construction logic, and test scenario characteristics (such as rigid body attribute configuration in collision detection scenarios and constraint settings in dynamics solution scenarios) are extracted. Based on the similarity retrieval function of the vector library, APIs not covered by unit tests are matched, and the top-5 code snippets with a cosine similarity of 0.6 or higher are selected as reference samples. Subsequently, the API semantic information, parameter attribute characteristics, and retrieved reference samples are input into large models (such as GPT-4o, DeepSeek-V3, Gemin3, etc.) to generate test code expansion content with data types adapted to the physics simulation engine (such as joint constraint matrices and rigid body state vectors). Simultaneously, corresponding compilation commands are automatically generated, and syntax verification and compilation validation are performed in conjunction with the Clang toolchain to ensure that the extended test code can be executed independently, forming a complete test resource package of "basic test code + extended supplementary logic", providing a structured and reusable code foundation for subsequent root definition analysis and fuzz test-driven generation.

[0020] The third phase is the unit test analysis phase. Test cases are identified from the AST of the test code, supporting multiple mainstream testing frameworks such as Google Tests. Multiple analyzers are used to perform multi-dimensional analysis of the unit test code:

[0021] Type Analyzer: Extracts the enumeration type declaration, including the enumeration name and information about all enumeration members.

[0022] Dependency Analyzer: Analyzes unit test source code and AST, extracts structural information and dependencies from source code files, including the location of the main function, the end offset of the source file, and the list of included header files.

[0023] Root Definition Analyzer: Employs root definition analysis techniques to trace the definition source of API call parameters in each test case.

[0024] Root definition analysis uses forward data flow analysis to trace the source of parameters backward from the API call point, following the Def-Use chain upwards until the root definition is found, such as variable declarations, constant values, function return values, or global variables. This process establishes a precise mapping between API parameters and their definition locations in the test code, ultimately generating an analysis report containing root definition information for every parameter of every API call in each unit test. This provides accurate input construction information for subsequent fuzz-driven generation.

[0025] The fourth stage is the fuzz test driver generation stage. Based on the analysis results of the first three stages, fuzz test driver code is generated. Root definitions are mapped to fuzz test inputs, a unique ID is assigned to each root definition, corresponding Protobuf fields (the basic unit of a single data item) are generated, and the mapping relationship between input variables and API parameters is established. Then, a fuzz test entry function is generated. This function extracts values ​​from the Protobuf fields and assigns them to variables, finally calling the test execution function. The test execution function is injected into the unit test code. This function creates an automated test class that inherits from the original test class and executes the test logic independently. The unit test code is modified, replacing variable definitions with values ​​obtained from the fuzz test inputs. This process involves precise source code modification while maintaining code formatting and comments. The generated fuzz test driver code uses the same compilation configuration as the original unit tests to ensure that the generated driver can correctly link to target libraries and dependencies. The compilation process generates executable fuzz test programs, with each test case corresponding to an independent executable file.

[0026] The fifth stage is the test execution and result analysis stage. In this stage, fuzzing is run automatically using the libFuzzer fuzzer. Each test driver creates an independent corpus directory, copying the initial seed file into the corpus as the starting point for fuzzing. During test execution, the fuzzing engine continuously mutates the input data based on coverage, generating new test cases, executing tests, and monitoring program behavior. A robust error handling mechanism is in place, including timeout control, crash handling, and exception handling, ensuring the stability of the testing process. When crashes, assertion failures, or other abnormal behaviors are detected, the test cases that triggered the problem are automatically saved to a dedicated directory for easy subsequent analysis and reproduction. Code coverage analysis is also integrated, collecting coverage data during test execution and generating detailed coverage reports. These reports not only include overall coverage statistics but also provide detailed coverage information at the function and file levels, helping testers understand the depth and breadth of their tests. Upon completion, a comprehensive test report is generated, including key metrics such as the number of vulnerabilities discovered, code coverage, test execution time, and the number of generated test cases, providing quantitative evidence for evaluating test effectiveness.

[0027] The beneficial effects of this invention are as follows: This invention effectively analyzes physics simulation engines, uses unit test code, and leverages large models to extend the code of core APIs not covered by unit tests to generate fuzz test drivers. It then uses libFuzzer to perform fuzz testing guided by coverage, significantly improving testing efficiency. It significantly improves the quality of test input and code coverage; the generated test drivers typically achieve 60-80% code coverage, far exceeding traditional random fuzz testing. It fully utilizes existing testing resources, transforming effective information from unit tests into fuzz test input, avoiding the repetitive work of writing test code. It supports multiple testing frameworks and has been successfully applied to projects in various fields such as image processing, networking, parsing, and multimedia, demonstrating wide applicability. This invention significantly improves testing efficiency through optimization techniques such as parallel processing and incremental analysis. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the defect detection method for a physics simulation engine based on mutation according to the present invention.

[0029] Figure 2 This is a flowchart of the target library analysis in the defect detection method of the physical simulation engine based on mutation in this invention.

[0030] Figure 3 This is a flowchart of the extended test code in the defect detection method of the mutated physics simulation engine of this invention.

[0031] Figure 4 This is a flowchart generated by the test program in the defect detection method of the physical simulation engine based on mutation in this invention.

[0032] Figure 5 This is a flowchart of the fuzzing test in the defect detection method of the physics simulation engine based on mutation in this invention. Detailed Implementation

[0033] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0034] In this embodiment, the method of the present invention is deployed on a Linux server, and the specific configuration of the server is shown in the table below. The experimental environment is built by installing the relevant compiler version LLVM / Clang-17, the serialization tool Protobuf-v3.14.0, the testing framework GoogleTest-1.8.1, the fuzzing tool libFuzzer-17.0.6, and the software required by the physics simulation engine library for testing, according to the experimental requirements.

[0035] Table 1 Linux Server Configuration Information

[0036] Processor Model Memory operating system Intel Core i7-13700k 32G Ubuntu 22.04

[0037] like Figure 1 As shown, this invention's mutation-based physics simulation engine testing method utilizes physics simulation engine library code and unit test code as test programs to generate defect reports for the physics simulation engine test program. Specifically, this invention generates an API analysis report through a target library API analysis module, and then uses an AST / IR generation module to produce a code abstract syntax tree (AST) and intermediate representation (IR). Next, library code analysis (generating a target library analysis report) and test code expansion (generating expanded test code) are executed in parallel, and the results are merged into a test code analysis module to form a test analysis report. Afterwards, a fuzzy object set is generated by a fuzzy input analysis module, and an executable fuzz test driver is generated by a test driver generation module. Finally, libFuzzer is used to perform fuzz testing and generate defect reports. The entire process realizes a closed loop from static code analysis to dynamic test program generation and fuzz testing, supporting automated fuzz testing of the target library.

[0038] The specific implementation process of this invention is as follows:

[0039] Step 1: Build the physics simulation library to generate compilation logs and static libraries. Manage the download and compilation process of the physics simulation engine library by parsing the YAML configuration file (make.yml).

[0040] The compiler executes compilation commands sequentially according to a predefined build order (org, profile, fuzzer mode). By setting the environment variable (`export COMPILE_LOG="$(realpath .. / output / build.log)"`), the compiler automatically intercepts and records all clang++ and llvm commands to the specified log file. It can dynamically and completely capture all compilation information, including the working directory, compilation options, and source file paths, ensuring the accuracy and completeness of the compilation log.

[0041] Configure CMake to enable testing (`-DBUILD_TESTING=ON`) to ensure that the test program is compiled together with the library code and the generated static library file (such as `libBulletDynamics.a`) facilitates subsequent AST / IR generation and static analysis.

[0042] Step 2: As Figure 2As shown, nm is used to perform API analysis on static libraries. It iterates through the static library files of all target libraries (such as the `libBulletDynamics.a` file) and performs symbol table analysis on each library. The nm tool (such as `nm --no-demangle --defined-only -g path`) is used to extract symbols from the static libraries, where the `--no-demangle` option preserves the original name format of C++ symbols.

[0043] The `--defined-only` option extracts only symbols defined in the library (excluding external references).

[0044] The `-g` option displays only externally visible global symbols, thus filtering out internal implementation details and local symbols. Using the nm tool, all defined public function symbols can be directly extracted from the compiled binary file, unaffected by complex factors such as source code structure, namespaces, and macro definitions. This ensures the accuracy and completeness of API extraction, and all engine library API information is uniformly saved as a JSON-formatted api.json file.

[0045] Step 3: Analyze the library source code and test code to generate Clang AST and LLVM IR. The generate_ast_ir function of the Builder class generates AST and IR for the target library and unit tests by calling the astirbuilder.build_ast_ir function.

[0046] In the generate_ast_ir function, the system uses the ProcessPoolExecutor process pool to process the AST / IR generation tasks of multiple libraries and unit tests in parallel. The process_ast_ir helper function executes the generate_ast_ir function of each library or test in an independent process, thereby making full use of multi-core CPU resources and significantly improving the processing efficiency of large-scale projects.

[0047] In the `astirbuilder.build_ast_ir` function, the `BuildParser` is used to parse the build log (`build.log`), extract all build commands and link commands, and then locate the target executable file in the link commands based on the `target_name` parameter (such as the library name or test executable file name). It then iterates through all the target files (`.o` files) linked by the executable file and finds the corresponding build command for each target file.

[0048] In the `astirbuilder.emit` function, the original compilation command is modified using a regular expression (e.g., `re.compile(r"^(clang\+\+|clang) ").sub(rf"\g <1> The command `{emit_opt} ", cmd)` adds the `-emit-ast` or `-emit-llvm` option before the original command, and uses the `-O0 -g` option to preserve debugging information and unoptimized code structure. The generated AST file is saved in `.ast` format, and the IR bitcode file is saved in `.bc` format. Finally, the `llvm-link` tool is used to link all bitcode files into a single `.bc` file, and the `make_build_db` function generates a `build_db` JSON file (containing the path to the bc file, a list of AST file paths, and the project directory), which is then stored in the `output / build_db` directory.

[0049] Step 4: Analyze the Clang AST and LLVM IR of the library source code using multiple types of analyzers (memory, array, constant, direction, file path, loop, number of arguments, type). The `generate_target_report` function of the `Builder` class performs multi-dimensional static analysis of the AST and IR of each library by calling the `target_analyzer` tool. In the `analyze` function of `TargetLibAnalyzer`, the system first loads the AST and IR data, and then sequentially creates and executes eight different types of analyzers:

[0050] Parameter Direction Analyzer: By analyzing StoreInst and LoadInst instructions and function calls, it determines the data flow of parameters and classifies them into Dir_In (input, read only), Dir_Out (output, write only), or Dir_InOut (input and output, both read and write), using bit flags (such as 0x100 for Dir_In and 0x010 for Dir_Out) for combined representation.

[0051] Memory allocation size analyzer: Identifies the parameters used to specify the memory allocation size by tracing the Def-Use chain to analyze whether parameters are passed to memory allocation functions (such as malloc, _Znaj, _Znam) as allocation size parameters.

[0052] Array Analyzer: By analyzing the GetElementPtr command and loop information (using LoopAnalysis and ScalarEvolutionAnalysis), it identifies array parameters and their corresponding length parameters, determines whether the array is a variable-length array, and establishes the relationship between array parameters and length parameters.

[0053] Constant Analyzer: Uses Clang ASTMatcher (such as `varDecl().bind("Tag")`) to extract constant declarations from the AST, supporting the extraction of constant values ​​of integer, floating-point, string and array types, providing constant value information for subsequent seed generation.

[0054] File path analyzer: By tracking the use of string parameters in function calls, it identifies the parameters passed to file operation functions (such as fopen, freopen, open, open64) and determines whether the parameters are used as file paths.

[0055] Loop Analyzer: By analyzing the ICmpInst instruction and the loop exit condition, it identifies whether parameters are involved in loop control, especially comparison operands used as loop exit conditions, and records loop depth information.

[0056] Parameter count analyzer: By matching function declarations in the Clang AST and function definitions in the LLVM IR, it extracts the number of function parameters and the starting index of the parameters (considering struct return values ​​and the this pointer of C++ member functions), and establishes a mapping from function names to parameter information.

[0057] Type Analyzer: Uses ASTMatcher (such as `enumDecl().bind("Tag")`) to extract enumeration type definitions from the AST, including enumeration names and all enumeration values, providing type information for subsequent type constraint generation.

[0058] Finally, all analysis results are merged into a target library analysis report in JSON format (e.g., `{"AllocSize":{...}, "Array": {...}, "Direction": {...}, ...}`), and saved in the result directory.

[0059] Step 5: As Figure 3 As shown, the AST and IR of the target library, unit test code, and API analysis report are stored as a vector library, and test code for the uncovered APIs is generated using a large model.

[0060] The TargetAPIFinder function of UTAnalyzer dynamically analyzes the function call relationships of unit test code in LLVM IR, extracts the set of all actually executed API functions (by analyzing CallBase instructions and function call graphs), and then calculates the API coverage difference (total API set minus dynamically executed API set) to identify API functions not covered by unit tests.

[0061] Unit test code, AST and IR, and API analysis reports are processed into blocks based on semantics. Each code unit, such as function, class definition, and test case, is treated as an independent block. CodeBERT is used to convert each code block into a high-dimensional vector representation. These vectors encode the syntactic structure, semantic information, and contextual features of the code. All vectorized code blocks are stored in the FAISS vector database to form a searchable data vector library.

[0062] For each API function that is not covered by the test, the function signature, parameter types, return type and other information of the API are first vectorized, and then the cosine similarity algorithm is used to search for similar code blocks and AST fragments in the vector database.

[0063] The top-5 most similar code blocks with a similarity threshold of 0.6 or higher are selected as input. The function information of the API not covered, the retrieved similar code blocks, the corresponding AST structure, and the structured prompt words are combined and input into the large language model (GPT-4o). The prompt words contain contextual information such as API function signatures, parameter constraint information (from the multi-dimensional analysis results in step 4), similar code examples, and target test framework requirements, guiding the model to generate test code that conforms to the specifications and the corresponding compilation commands.

[0064] The generated test code is compiled and verified. If compilation fails, the system will send the compilation error message, failure reason analysis and original prompt words back to the large model for iterative optimization and generation until test code that can be successfully compiled is generated or the maximum number of generation attempts is reached.

[0065] For successfully generated test code, repeat step 3 to generate the corresponding Clang AST and LLVMIR.

[0066] Step 6: Use multiple type analyzers (type, dependency, root definition) to analyze the Clang AST and LLVM IR of the test code. The generate_ut_report function of the Builder class performs multi-dimensional static analysis of the AST and IR of each unit test by calling the ut_analyzer tool.

[0067] In the `analyze` function of `UTAnalyzer`, the system first uses the `TCExtractor` (creating the corresponding extractor based on the test framework type, such as `GoogleTest` or `Boost.Test`) to extract all unit test cases from the AST, and then creates and executes three different types of analyzers in sequence:

[0068] Type analyzer: Uses Clang ASTMatcher (such as `enumDecl().bind("Tag")`) to traverse all AST cells, match and extract data type declarations, and for each data type, extract its full name and all member variables, including name and corresponding numerical information.

[0069] Dependency Analyzer: By analyzing unit test source code and AST, it extracts structural information and dependencies from source code files. Using the `getMainFuncDecl` function with an ASTMatcher (e.g., `functionDecl(hasName("main"), isDefinition())`), it locates the `main` function declaration and extracts its complete location information (file path, offset, length). Then, it uses the `getEndOffset` function to obtain the end offset of each source file. The `getIncludes` function analyzes all included header files in the source files. It iterates through the `SLocEntry` (Source LocationEntry) of the `SourceManager`, checking for the validity of its `IncludeLoc` (include location) and whether it belongs to the main file for each file entry. Finally, it uses `Rewriter` to extract the complete text of the include directives (including `#include ...`).<header.h> (or in the format `#include "header.h"`), and organize all header file paths in the order they are included.

[0070] Root Definition Analyzer: This analyzer traces API call parameters back to their original definitions using Root Definition Analysis (RDA) technology. First, it uses TargetAPIFinder to find all calls to the target API function in LLVM IR. Then, it uses RDAnalyzer for deep tracing. RDAnalyzer's findRootDefinitions function starts from the Use object and traces upwards along the Def-Use chain. It also uses the traverseFunction function to trace within functions (supporting both forward and backward tracing). RDAnalyzer identifies various types of root definitions.

[0071] Local variable declarations, including uninitialized variables; global variables; function parameters; constant values.

[0072] The DefMapGenerator converts all collected root definition nodes into Definition objects, which contain information such as the definition's location, type, AST value, and array length association.

[0073] Step 7: As Figure 4 As shown, the fuzz test driver source code and meta-seed are generated. The generate_fuzz_driver function of the Builder class calls the fuzz_generator tool to generate the fuzz test driver source code file and initial seed file.

[0074] In the `generate` function of the Generator, the target library analysis report and test analysis report are loaded, and the `initFuzzers` function is called to create a Fuzzer object for each unit test. The Fuzzer object encapsulates the unit test information, API call information, and input definition information. The system assigns a unique name to each Fuzzer object (using the `assignUniqueName` function, in the format `{UTName}_{APIName}_Test`). Then, the `generateFuzzSource` function is called to generate the fuzz test-driven source code.

[0075] In the `generate` function of `FuzzerSrcGenerator`, the original unit test source code directory is completely copied to the output directory (using `util::copy(SrcDir, FuzzerSrcPath, true)`), preserving the directory structure and file organization of the source code. Then, a `ProtobufMutator` object is created to generate the fuzzing entry point and Protobuf description file. `ProtobufMutator` adds Protobuf field definitions to each `FuzzInput` using the `addInput` function, generating the `FuzzArgsProfile.proto` file (containing the structured definitions of all input parameters that need fuzzing). Then, the `genEntry` function is called to generate the `fuzz_entry.cc` file, which contains the main fuzzing entry function (using the `DEFINE_PROTO_FUZZER` macro, an interface provided by libprotobuf-mutator). In the entry function, variable declarations and assignment statements are generated for each input parameter, and the values ​​from the Protobuf messages are copied to the corresponding C++ variables. Finally, the `enterAutofuzz()` function is called to execute the modified unit test logic.

[0076] The UTModify class is responsible for modifying the original unit test code to integrate fuzz testing logic. It generates the declarations of fuzz test variables (including basic type variables and array length variables) through the generateFuzzVarDeclarations function, and replaces the initialization of variables in the original unit test with values ​​obtained from the fuzz test input through the generateFuzzVarReplacements function (using the generateAssignStatement function to generate assignment statements, the generateTop function to inject necessary header files (including autofuzz.h and Protobuf-generated header files) and global variable setting functions at the beginning of the source file, and the generateBottom function to inject the enterAutofuzz ​​function (which wraps the execution logic of the original unit test and handles the setting of global variables) at the end of the source file. All code modifications are implemented through Clang's Replacement mechanism to ensure the accuracy and traceability of the modifications.

[0077] While generating source code, the system also generates an initial seed file (meta-seed) through CorpusGenerator. The generatorCorpus function of the generator iterates through all Fuzzer objects and calls the generate function of CorpusGenerator for each Fuzzer. CorpusGenerator extracts ASTValue (constant value) from the definition of FuzzInput. For root definitions that are not array lengths, the system converts ASTValue into a string in Protobuf text format, supporting integer, floating-point, string, enumeration and other types. For array types, the system generates different formats according to the element type (byte type or non-byte type) (byte arrays generate string format, other arrays generate list format). These values ​​come directly from the constant definitions in the unit test, providing high-quality initial input for fuzzing and significantly improving the efficiency and code coverage of fuzzing.

[0078] Step 8: Compile the fuzz test driver to generate an executable program. The build_fuzz_driver function of the Builder class calls the driverbuilder.build_driver function to compile the generated fuzz test driver source code into an executable program.

[0079] The `build_driver` function first uses `DriverParser` to parse the generated driver source code directory to identify all drivers that need to be compiled (each driver corresponds to a unit test case). Then, it uses the `FuzzerBuilder` class for compilation. `FuzzerBuilder` first parses the raw compilation log (`build.log`) using `BuildParser` to extract all compilation and linking commands, establishing a mapping between source files and compilation commands. For each driver, the system performs the following compilation steps:

[0080] First, compile the Protobuf description file. Use the protoc tool (`protoc --cpp_out={proto_path} --proto_path={proto_path} FuzzArgsProfile.proto`) to generate FuzzArgsProfile.pb.cc and FuzzArgsProfile.pb.h files, which contain the serialization and deserialization code for Protobuf messages.

[0081] Compile the files fuzz_entry.cc and FuzzArgsProfile.pb.cc. Based on the original compilation commands, use the get_compile_cmds_for_autofuzz ​​function to generate new compilation commands. This function replaces the source file paths with fuzz_entry.cc and FuzzArgsProfile.pb.cc, adds the include path `-I / usr / local / include / libprotobuf-mutator` to support the libprotobuf-mutator header file, uses the clang++ compiler (replacing the original clang or gcc), and adds fuzzing-related flags during compilation: `-DBOOST_TEST_NO_MAIN -DAUTOFUZZ` (disables the main function of the testing framework and enables automatic fuzzing macros), and `-fsanitize=address,fuzzer-no-link` (enables AddressSanitizer memory detection and libFuzzer support, but does not link the fuzzer library). The compiled object files are saved in the build_path / avas_out directory.

[0082] To compile the modified unit test source code, the system uses the get_compile_cmd_by_src function to find the compilation command corresponding to the original unit test source file, and then uses the compile function to execute the compilation. During compilation, fuzz testing-related flags and optional coverage flags are also added to ensure that all source code files are compiled correctly.

[0083] Finally, the `get_link_cmd` function is used to generate the linking commands, obtaining the linking commands for the original unit tests. Then, all compiled object files are added (including `fuzz_entry.cc.o`, `FuzzArgsProfile.pb.cc.o`, and the modified unit test object files). The linked static library files are replaced with the `fuzzer` or `profile` version of the library (such as `lib{libname}_fuzzer.a` or `lib{libname}_profile.a`, which were compiled in step 1). The linking options for the `libprotobuf-mutator` library are added (`-L / usr / local / lib -l:libprotobuf-mutator-libfuzzer.a -l:libprotobuf-mutator.a`). The option `-l:libprotobuf.a` adds the linking option for the libFuzzer library (`-fsanitize=address,fuzzer`, which links the libFuzzer library). The linked executable file is saved in the output / profiles directory (profile mode) or the output / fuzzers directory (fuzzer mode), and the executable file name is the same as the driver name.

[0084] Step 9: As Figure 5 As shown, the fuzzing process is executed automatically by the fuzz.py script in the helper directory, which collects coverage data. The execute_fuzz_for_all_executables function is the core component of fuzzing execution. It traverses all executable files in the output / profiles directory (by checking the executable permissions `stat().st_mode & 0o111`) and creates an independent test environment for each executable file.

[0085] For each executable file, the system first recursively searches for and copies the corresponding initial seed file (in the format `{executable_stem}.corpus`) from the result / test / {target} directory using the copy_corpus_files function. These seed files are the initial input in Protobuf text format generated in step 7. The system copies the seed files to a seed library directory (in the format `corpus_{executable.stem}`) specifically created for the current executable file, which serves as the initial input seed library for libFuzzer.

[0086] Then, the `run_fuzz_test` function is called to perform the actual fuzz test. This function first sets the `LLVM_PROFILE_FILE` environment variable (in the format `{executable.stem}.profraw`) to specify the storage path for the coverage data file. This is the format required by the LLVM code coverage tool, which is used to automatically collect code coverage information during program execution.

[0087] The system uses the `subprocess.run` function to call the executable file, passing in the command-line arguments for `libFuzzer`:

[0088] The corpus directory path (libFuzzer reads the initial seed file in this directory and saves newly discovered test cases back to this directory during the testing process).

[0089] The `-max_total_time={max_total_time}` parameter specifies the maximum execution time for the fuzz test (in seconds; the default is 300 seconds).

[0090] The `-print_final_stats=1` parameter requires libFuzzer to print statistics (including the number of test cases executed, code coverage, number of crashes found, etc.) at the end of the test.

[0091] The `-ignore_crashes=1` parameter instructs libFuzzer to continue execution when it encounters a crash instead of exiting immediately (this is very important for long-running fuzzing, as it ensures that multiple different vulnerabilities are found).

[0092] The `-timeout=120` parameter sets the maximum execution time of a single test case to 120 seconds (to prevent excessive test case execution time from reducing the efficiency of fuzz testing).

[0093] The `-artifact_prefix={info_dir} / ` parameter specifies the directory prefix for storing crash and timeout test cases (libFuzzer will save the test cases that trigger crashes or timeouts as files, with the filename containing the crash type and the hash value of the test case).

[0094] During execution, libFuzzer continuously mutates the input data (structured input based on Protobuf format), generates new test cases, executes the fuzz test driver, and monitors program behavior (detecting memory errors through AddressSanitizer and code coverage through libFuzzer's instrumentation). For test cases that trigger new code paths, libFuzzer adds them to the seed library as new seeds. For test cases that trigger crashes or timeouts, libFuzzer saves them to the directory specified by artifact_prefix.

[0095] During fuzzing, LLVM's code coverage tool continuously collects coverage data and writes it to a profraw file. This file uses a sparse format, recording only the actual execution locations and times, significantly reducing file size. After fuzzing is complete, the system calls the generate_coverage_report function to generate a coverage report. The report contains coverage statistics for each source file, including total lines, covered lines, and coverage percentage, helping developers understand the coverage effect of fuzzing and identify key code areas that need focused testing.

[0096] Through the complete construction and testing process described above, the method of this invention can automatically convert existing unit tests into high-quality fuzz test drivers. It achieves significant improvements in code coverage and vulnerability discovery capabilities in complex C++ projects such as physics simulation libraries. Compared with the traditional method of manually writing fuzz test drivers, the degree of automation and testing efficiency are improved by several times, verifying the effectiveness of the automated fuzz test driver generation strategy based on the combination of multi-dimensional static analysis, code vectorization retrieval, and large language models.

Claims

1. A defect detection method for a mutation-based physics simulation engine, characterized in that, The specific steps are as follows: The first stage is the target library analysis stage; the source code of the target library is subjected to in-depth static analysis, and the physical simulation library is built to generate compilation logs and static libraries. The nm tool is used to extract the parameter attribute information of API functions, and the library source code and unit test code are analyzed to generate Clang AST and LLVM IR. Various types of analyzers are used to conduct a comprehensive analysis of each API parameter from multiple dimensions. The second stage is the test code expansion stage; API functions not called by unit tests are extracted, and the library code analysis data Clang AST and LLVM IR, as well as the vectorized library code, are stored in the vector library. By analyzing the unit test code of the physics simulation engine, API call patterns, parameter construction logic and test scenario characteristics are extracted. Based on the similarity retrieval function of the vector library, APIs not covered by unit tests are matched, and the top-5 code snippets with a cosine similarity of 0.6 or higher are selected as reference samples. Then, the API semantic information, parameter attribute features, and the retrieved reference samples are input into the large model to generate test code extensions adapted to the data type of the physics simulation engine. At the same time, the corresponding compilation commands are automatically generated, and the syntax is checked and the compilation is verified by combining the Clang toolchain to ensure that the extended test code can be executed independently, forming a complete test resource package of "basic test code + extended supplementary logic", providing a structured and reusable code foundation for subsequent root definition analysis and fuzz test-driven generation. The third stage is the unit test analysis stage; test cases are identified from the AST of the test code, and multiple analyzers are used to perform multi-dimensional analysis of the unit test code: The fourth stage is the fuzz test-driven generation stage; based on the analysis results of the first three stages, fuzz test-driven code is generated. The root definitions are mapped to fuzz test inputs. A unique ID is assigned to each root definition, and a corresponding Protobuf field is generated to establish a mapping relationship between input variables and API parameters. Then, a fuzz test entry function is generated. The fuzz test entry function extracts values ​​from the Protobuf field and assigns them to variables. Finally, the test execution function is called. The test execution function is injected into the unit test code. The test execution function creates an automated test class that inherits from the original test class and executes the test logic independently. The unit test code is modified to replace the variable definitions with values ​​obtained from the fuzz test inputs. This process involves precise source code modification while maintaining the code format and comments. The generated fuzz test driver code uses the same compilation configuration as the original unit test, ensuring that the generated driver can correctly link the target library and dependencies; The compilation process generates executable fuzz test programs, with each test case corresponding to an independent executable file; The fifth stage is the test execution and result analysis stage; fuzz testing is run automatically using the libFuzzer fuzzer; each test driver creates an independent corpus directory, and the initial seed file is copied into the corpus as the starting point for fuzz testing; During test execution, the fuzzing engine continuously mutates the input data based on coverage, generates new test cases, executes tests, and monitors program behavior; it also sets up error handling mechanisms, including timeout control, crash capture, and exception handling, to ensure the stability of the testing process. When crashes, assertion failures, or other abnormal behaviors are detected, the test cases that triggered the problem are automatically saved to a dedicated directory for easy analysis and reproduction later. Simultaneously, code coverage analysis is integrated to collect coverage data during test execution and generate detailed coverage reports. These reports not only include overall coverage statistics but also provide detailed coverage information at the function and file levels, helping testers understand the depth and breadth of their tests. After testing is completed, a comprehensive test report is generated, including metrics such as the number of vulnerabilities discovered, code coverage, test execution time, and the number of test cases generated, providing quantitative data for evaluating test effectiveness.

2. The method for detecting defects in a mutation-based physics simulation engine according to claim 1, characterized in that, The specific analysis types in the first phase are as follows: Parameter orientation analysis: Identifying whether a parameter is an input, output, or input-output; Memory allocation size analysis: Identify whether the parameter is used as the allocation size parameter in the memory allocation function; Array analysis: Identify array parameters and their corresponding length parameters, and determine whether the array is a variable-length array; Constant analysis: Extract constant declarations from the AST, including constant values ​​of integer, floating-point, string, and array types; File path analysis: Determines whether a parameter is used as a file path; Loop analysis: Analyzes whether function parameters participate in loop control; Parameter count analysis: Analyzes the number of parameters and their starting positions in the function; Type analysis: Extract the enumeration type definition from the AST, including the enumeration name and all enumeration values.

3. The method for detecting defects in a mutation-based physics simulation engine according to claim 1, characterized in that, The analyzer in the third stage is as follows: Type analyzer: Extracts the enumeration type declaration, including the enumeration name and information about all enumeration members; Dependency Analyzer: Analyzes unit test source code and AST, extracts structural information and dependencies of source code files, including the location of the main function, the end offset of the source file, and the list of included header files; Root Definition Analyzer: Employs root definition analysis technology to trace the definition source of API call parameters in each test case; Root definition analysis uses forward data flow analysis to trace the source of parameters backward from the API call point, tracing upward along the Def-Use chain until the root definition of variable declarations, constant values, function return values, or global variable pairs is found. This process establishes a precise mapping relationship between API parameters and their definition locations in the test code, ultimately generating an analysis report containing root definition information for each parameter of each API call in each unit test, providing accurate input construction information for subsequent fuzz test-driven generation.

Images