Gazebo defect automatic discovery method based on coupling analysis and coverage rate guidance

Abstract

The invention belongs to the field of software testing, and particularly relates to a Gazebo defect automatic discovery method based on coupling analysis and coverage rate guidance, which is an automatic defect detection method based on a coupling analysis and coverage rate guidance mechanism and is suitable for defect discovery in a Gazebo simulation platform. According to the method, the high-coupling module and the key coupling point in the Gazebo source code to be tested are accurately identified through static coupling analysis, so that test resources can be preferentially concentrated in the most complex and most error-prone interaction area. According to the method, static coupling analysis and a dynamic coverage rate guiding mechanism are combined. Code lines and function coverage are paid attention to, and it is ensured that interaction paths between complex modules are fully tested by recognizing the coupling relation. According to the method and the device, automatic generation and optimization of the test case are realized, the dependence on a manual writing scene of a test expert is reduced, and the threshold and the cost of an automatic test are reduced.

Description

Technical Field

[0001] This invention belongs to the field of software testing, specifically relating to an automatic defect discovery method for Gazebo based on coupling analysis and coverage guidance. It is an automated defect detection method based on coupling analysis and coverage guidance mechanism, suitable for defect discovery in the Gazebo simulation platform. Background Technology

[0002] Gazebo is a widely used 3D robot simulation platform capable of simulating realistic physical interactions in a virtual environment, supporting functional testing of robots such as perception, motion, and dynamics. As the scale and complexity of robots and simulation systems continue to increase, their software packages often involve highly coupled modules, complex communication interfaces, dynamic scene interactions, and multi-threaded processing. This complexity introduces a higher risk of defects, especially in cross-module interactions and boundary conditions, where potential bugs are more likely to occur.

[0003] Traditional defect detection methods, such as fuzzing based on random input, have the following limitations in highly coupled simulation environments like Gazebo:

[0004] Inefficient input data: Randomly generated test cases are prone to mismatch with the expected input format of the program, resulting in test scenarios that cannot fully cover critical code paths;

[0005] Insufficient coverage: Due to the inability to customize test cases for the characteristics of highly coupled modules, some key interaction paths and crash triggering conditions cannot be detected;

[0006] The range of bug types is narrow: Traditional fuzz testing is mostly aimed at crash detection, but there are a lot of "logical correctness defects" in the simulation environment (such as violations of physical constraints and interface protocol errors). Although these problems do not cause the program to exit abnormally, they will damage the reliability of the system function.

[0007] Unable to dynamically adjust strategies: Existing methods struggle to optimize test case generation strategies based on real-time test coverage, resulting in low testing efficiency.

[0008] Therefore, there is an urgent need for a defect detection method that combines source code coupling analysis, automated test case generation, and real-time coverage guidance mechanisms, which can continuously and automatically discover explicit and implicit defects in the Gazebo simulation environment. Summary of the Invention

[0009] This invention aims to provide a method for automatically detecting software package defects in the Gazebo simulation environment. By analyzing the coupling points between the source code function under test and external components, and combining large model code generation and coverage guidance, efficient defect detection is achieved, including explicit (crash-type) and implicit (functional) bugs.

[0010] The technical solution of the present invention:

[0011] The automatic defect discovery method for Gazebo based on coupling analysis and coverage guidance involves the following steps:

[0012] Step (1) Source code coupling analysis and test priority determination: Perform static analysis on the source code of the Gazebo simulation software package, extract the calling relationship, shared variables and dependency relationship between modules, and identify the key coupling points in the source code functions; then calculate the coupling index and sort them from high to low according to the index value to form a test priority sequence, and prioritize the highly coupled modules at the head of the test queue.

[0013] Step (2) Existing test execution and coverage information collection: Execute the existing test cases provided by Gazebo, and at the same time use coverage tools (such as lcov / gcov) to collect code execution data, analyze the coverage of the source code by the official test, identify the lines and functions that are not covered or have low coverage, i.e., code with zero coverage or few coverage, and sort them in ascending order of coverage, giving priority to identifying lines and functions with zero coverage or few coverage.

[0014] Step (3) Construct structured prompts: Set the prompt template as follows:

[0015] Please generate targeted test code patches for the Gazebo simulation platform, focusing on covering highly coupled modules in areas with low coverage.

[0016] Test target information:

[0017] Core function: [function name]

[0018] Source file path: [Source file path]

[0019] Coupling Index: [Coupling Value]

[0020] Key external dependencies: [List of external dependency functions / variables]

[0021] Code area to be covered: [Lines of code or function names with low coverage]

[0022] Test Requirements: Based on the above information, generate a Google Test code patch that can be directly integrated into existing test suites. The test should focus on verifying: 1. Boundary conditions and exception handling for key parameters; 2. Fault tolerance of external dependency interfaces; 3. Functional correctness under specific configuration scenarios; 4. Integrity of data structures and state transitions.

[0023] Output format: Only output C++ test code, without any explanatory text.

[0024] The coupled modules analyzed in step (1) and the code units with zero coverage or few coverage times identified in step (2) are integrated according to their priority order to generate prompt information containing key test targets, and the prompt words are filled into the template to guide the generation of test code.

[0025] Step (4) Large model-driven test code generation and implantation: Based on the prompt words constructed in step (3), the large language model is called to generate targeted test case code patches and apply them to the original test to enhance the test intensity of highly coupled and low-coverage areas.

[0026] Step (5) Coverage-guided iteration and defect determination: Set coverage thresholds (row coverage >= 80%, function coverage >= 90%). If the current test coverage does not reach the threshold, repeat steps (3) to (4) to continuously generate and execute new test cases and gradually improve coverage; monitor program behavior in real time during test execution and record all exception and defect information.

[0027] The beneficial effects of this invention are:

[0028] 1. Highly targeted testing with significantly improved efficiency: Static coupling analysis accurately identifies highly coupled modules and key coupling points in the Gazebo source code under test, allowing testing resources to be prioritized for the most complex and error-prone interaction areas. This overcomes the shortcomings of traditional fuzz testing's blind randomness, improving the targeting and effectiveness of test cases from the source and significantly shortening the defect discovery cycle.

[0029] 2. Achieving Both Depth and Breadth of Coverage: This invention combines static coupling analysis with a dynamic coverage guidance mechanism. It not only focuses on line-of-code and function coverage but also ensures that complex inter-module interaction paths are fully tested by identifying coupling relationships. This approach can reach hidden defects such as deep interaction boundaries and condition races that are difficult to cover with traditional methods, achieving a more comprehensive detection of systemic defects in simulation software.

[0030] 3. More comprehensive defect detection dimensions: This invention can not only effectively discover explicit defects that cause program crashes and exits, but also greatly improve the overall reliability of the simulation system by generating test cases for specific interface protocols and functional logic.

[0031] 4. Adaptive optimization capability: A closed-loop feedback mechanism with coverage as the target is introduced. By analyzing coverage gaps in real time and driving the large model to generate new test code, the testing process can self-evolve, continuously exploring uncovered code areas, thus forming an adaptive and self-expanding automated testing process, effectively improving the adequacy of testing.

[0032] 5. High level of intelligence, reducing manpower burden: By utilizing the code generation capabilities of large models, complex analysis results such as highly coupled points and low-coverage code segments are automatically transformed into effective test cases, realizing the automated generation and optimization of test cases, reducing the reliance on test experts to manually write scenarios, and lowering the threshold and cost of automated testing. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the automatic defect discovery method for Gazebo based on coupling analysis and coverage guidance of the present invention. Detailed Implementation

[0034] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0035] The automatic defect discovery method for Gazebo based on coupling analysis and coverage guidance proposed in this invention can be applied in the Gazebo component library to automatically analyze and test plugin source code, such as the LiftDrag plugin in this embodiment.

[0036] The LiftDrag plugin implements the simulation of aerodynamic lift and drag. Its source code structure is relatively complex, containing a large number of external dependent functions and component calls, making it very suitable as an example to illustrate the execution flow of this invention.

[0037] I. Overall Process Overview

[0038] like Figure 1 As shown, the method of the present invention includes the following main processes:

[0039] 1. Source code coupling analysis and test priority determination: The source code structure of the Gazebo component library to be tested is analyzed to identify the relationships between modules such as interface calls, data sharing and functional dependencies, and to locate the key coupling points in the system; the coupling degree of each function is calculated and a test priority sequence is established based on the coupling strength.

[0040] 2. Existing test execution and coverage information collection: Run the test case set built into Gazebo for the source code under test, use coverage tools (such as lcov / gcov) to collect code execution data, analyze the coverage of the source code by existing tests, and identify lines of code and functions that are not covered or have low coverage.

[0041] 3. Test case generation strategy construction: Design a structured prompt word generation template, integrate the information of highly coupled modules in step (1) with the low coverage code area in step (2) to form prompt content with clear test guidance, and provide input basis for the automatic generation of subsequent test code.

[0042] 4. Large Model-Driven Test Code Generation and Integration: Based on the prompts generated in the previous step, the large language model is used to automatically generate test case patches for specific code regions, and the generated effective test code is integrated into the Gazebo testing environment to achieve targeted test enhancement for high-risk regions.

[0043] 5. Coverage guidance and defect logging: Feedback optimization, execute a new round of testing and update coverage data until the set threshold is reached; at the same time, collect error logs, abnormal events or logical deviations during the testing process.

[0044] The entire process forms a closed-loop system driven by static analysis, optimized by dynamic feedback, and generated by automatic testing, enabling continuous defect discovery and verification in complex simulation environments.

[0045] II. Detailed Explanation of Each Step

[0046] 1. Source code coupling analysis and test priority determination

[0047] It calculates the coupling between each function in a single source file and external functions / interfaces, using the function as the smallest unit of analysis.

[0048] Example of an analysis object: The main functions in the LiftDrag.cc file include:

[0049] LiftDragPrivate::Load()

[0050] LiftDragPrivate::Update()

[0051] LiftDrag::Configure()

[0052] LiftDrag::PreUpdate()

[0053] (1) Static analysis steps

[0054] Using Python scripting tools, abstract syntax tree parsing is performed on the file to automatically extract coupling points:

[0055] (a) List of function definitions (including scope, parameters and types, and return value);

[0056] (b) Other functions called within the function;

[0057] (c) The origin of the called function (current file / external file / library function / Gazebo interface).

[0058] For example, the parsing result of LiftDragPrivate::Load():

[0059] ExtCallsf: 18 (such as: entitiesFromScopedName, EntityComponentManager::EntityHasComponentType, math::Vector3d::Normalize)

[0060] ExtRefsf: 12 (such as: components::Link, components::Joint, kNullEntity,gzerr)

[0061] in:

[0062] ExtCallsf: The number of times function f calls the external function.

[0063] ExtRefsf: The number of times function f accesses external variables or classes.

[0064] (2) Calculate the function coupling index

[0065] For each function, calculate the function-level coupling degree.

[0066]

[0067] In the formula, a and b are weights (empirical coefficients: a=0.7, b=0.3); TotalCallsf is the sum of the number of times function f calls external functions and the number of times the function accesses external variables or classes; and Depthf is the depth of function f in the call.

[0068] (3) Structured output of results:

[0069] The coupling analysis report was generated in JSON format, and a portion of it is shown below:

[0070] {

[0071] "source": "LiftDrag.cc",

[0072] "functions": [

[0073] {

[0074] "name": "LiftDragPrivate::Load",

[0075] "external_calls": 18,

[0076] "external_refs": 12,

[0077] "coupling_index": 0.76,

[0078] "related_external": [

[0079] "sdf::ElementPtr::Get",

[0080] "sdf::ElementPtr::HasElement",

[0081] "sdf::ElementPtr::GetElement",

[0082] "entitiesFromScopedName",

[0083] "EntityComponentManager::EntityHasComponentType",

[0084] "math::Vector3d::Normalize",

[0085] "gzerr",

[0086] "gzwarn"

[0087] ],

[0088] "risk_level": "High"

[0089] } ]

[0091] }

[0092] After generating the above information for each function, the information is prioritized based on the coupling degree index, with information with high coupling degree being given priority to enter the prompt word construction stage.

[0093] 2. Existing test execution and coverage information collection

[0094] (1) Execute the original test file of LiftDrag (LiftDrag_TEST.cc) and record the coverage using the lcov / gcov tool.

[0095] (2) Generate coverage information in JSON format and extract uncovered rows and functions.

[0096] 3. Construct structured prompt words

[0097] By fusing coupling analysis with coverage information, prompt word templates for large models are generated and populated.

[0098] This context template based on coupling analysis provides a clearer direction when generating test code for large models, improving the effectiveness of patches.

[0099] The example generated template is as follows:

[0100] Please generate targeted test code for the following highly coupled functions:

[0101] Function: LiftDragPrivate::Load

[0102] Source file: src / LiftDrag.cc

[0103] Coupling analysis results:

[0104] External dependency functions: sdf::ElementPtr::Get / HasElement, entitiesFromScopedNam

[0105] Coupling index: 0.76

[0106] Key external references: EntityComponentManager, components::Link, components::Joint

[0107] Test coverage targets:

[0108] (a) Scenario where the test function parameters _ecm and _sdf are empty.

[0109] (b) Testing abnormal scenarios of control_joint_name configuration

[0110] (c) Boundary conditions for cover vector normalization (forward / upward)

[0111] (d) Validate entity type checking mechanism (Link / Joint type validation)

[0112] 4. Large-model-driven test code generation and implementation

[0113] The system inputs the prompts from the previous step into the large language model. The large model, combining coverage gaps and coupling, generates tests targeting the test coverage objective, for example:

[0114] TEST(LiftDragCrashReproTest, DynamicAddLiftDragExpectCrash)

[0115] {

[0116] ServerConfig serverConfig

[0117] serverConfig.SetSdfString(

[0118] " <sdf version="1.8"> <world name="default" / > < / sdf> ");

[0119] Server server(serverConfig);

[0120] server.SetUpdatePeriod(std::chrono::nanoseconds(1000000));

[0121] std::thread serverThread([&]() {

[0122] server.Run(true, 500, false);

[0123] });

[0124] std::this_thread::sleep_for(std::chrono::milliseconds(200));

[0125] gz::msgs::EntityPlugin_V req;

[0126] / / Add the LiftDrag plugin to the invalid entity, corresponding to test target 1

[0127] req.mutable_entity()->set_id(0);

[0128] auto *plugin = req.add_plugins();

[0129] plugin->set_name("gz::sim::systems::LiftDrag");

[0130] plugin->set_filename("gz-sim-lift-drag-system");

[0131] plugin->set_innerxml(

[0132] "<air_density> 1000< / air_density> \n"

[0133] " <cla> 1.2535816618911175< / cla> \n"

[0134] "<cla_stall>1.4326647564469914< / cla_stall>\n"

[0135] " <cda> 0< / cda> \n"

[0136] "<cda_stall>1.4326647564469914< / cda_stall>\n"

[0137] "<alpha_stall>1.396< / alpha_stall>\n"

[0138] " <a0> 0< / a0> \n"

[0139] " <area> 0.27637\n"

[0140] " <upward> 0.7071067811865476 0 -0.7071067811865475< / upward> \n"

[0141] " <forward> 0.7071067811865475 0 0.7071067811865476< / forward> \n"

[0142] "<link_name>propeller< / link_name>\n"

[0143] " <cp> 0 0.35 0< / cp> ");

[0144] gz::transport::Node node;

[0145] gz::msgs::Boolean rep;

[0146] bool serviceSuccess = false;

[0147] bool transportOk = node.Request(

[0148] " / world / default / entity / system / add",

[0149] req,

[0150] 2000u,

[0151] rep,

[0152] serviceSuccess);

[0153] ASSERT_TRUE(transportOk);

[0154] std::this_thread::sleep_for(std::chrono::seconds(1));

[0155] server.Run(true, 1000, false);

[0156] server.Running(false);

[0157] if (serverThread.joinable())

[0158] serverThread.join();

[0159] }

[0160] This test, through a dynamic loading process, focuses on verifying the critical coupling points between the LiftDrag plugin and external components such as SDF parsing and entity management under invalid value input scenarios, effectively detecting the risk of null pointer access during resource initialization. The generated test code can be added to the original test file to enhance coverage of null checks at coupling points.

[0161] 5. Coverage-guided iteration and defect determination

[0162] This step is a feedback optimization phase of the entire method, aiming to drive continuous improvement of the testing process through real-time coverage data, while recording defect information (bug Info) to achieve automated defect discovery.

[0163] During test execution, this method automatically captures program crashes (explicit defects), functional anomalies or logical deviations (latent defects), and log output anomalies. After executing the optimized test, the following crashes occur and are recorded, with the specific crash stack as follows:

[0164] Stack trace (most recent call last):

[0165] #13 Object " / home / momo / w_s / install / lib / libgz-sim9-gz.so.9.1.0", at0x7653d030b9e3, in runServer

[0166] #12 Object " / home / momo / w_s / install / lib / libgz-sim9.so.9", at0x7653cca0b591, in gz::sim::v9::Server::Run(bool, unsigned long, bool)

[0167] #11 Object " / home / momo / w_s / install / lib / libgz-sim9.so.9", at0x7653cca787b2, in

[0168] #10 Object " / home / momo / w_s / install / lib / libgz-sim9.so.9", at0x7653ccab1d33, in gz::sim::v9::SimulationRunner::Run(unsigned long)

[0169] #9 Object " / home / momo / w_s / install / lib / libgz-sim9.so.9", at0x7653ccab2d38, in gz::sim::v9::SimulationRunner::Step(gz::sim::v9::UpdateInfo const&)

[0170] #8 Object " / home / momo / w_s / install / lib / libgz-sim9.so.9", at0x7653ccaaef6a, in gz::sim::v9::SimulationRunner::UpdateSystems()

[0171] #7 Object " / home / momo / w_s / install / lib / gz-sim-9 / plugins / libgz-sim-lift-drag-system.so", at 0x7653b4302616, in

[0172] #6 Object " / home / momo / w_s / install / lib / gz-sim-9 / plugins / libgz-sim-lift-drag-system.so", at 0x7653b42fcb33, in

[0173] #5 Object " / home / momo / w_s / install / lib / libgz-sim9.so.9", at0x7653cc86dd54, in std::pair<double, bool> sdf::v15::Element::Get <double>(std::__cxx11::basic_string<char, std::char_traits <char>, std::allocator <char>> const&, double const&) const

[0174] #4 Object " / home / momo / w_s / install / lib / libgz-sim9.so.9", at0x7653cc876c94, in std::pair<double, bool> sdf::v15::Element::Get <double>(std::vector<sdf::v15::Error, std::allocator <sdf::v15::error>>&, std::__cxx11::basic_string<char, std::char_traits <char>, std::allocator <char>>const&, double const&) const

[0175] #3 Object " / home / momo / w_s / install / lib / libsdformat15.so.15", at0x7653ca9d15d2, in sdf::v15::Element::GetAttribute(std::__cxx11::basic_string<char, std::char_traits <char>, std::allocator <char>> const&) const

[0176] #2 Object " / home / momo / w_s / install / lib / libsdformat15.so.15", at0x7653ca924b13, in

[0177] #1 Object " / home / momo / w_s / install / lib / libsdformat15.so.15", at0x7653ca93157b, in

[0178] #0 Object " / home / momo / w_s / install / lib / libsdformat15.so.15", at0x7653ca939a16, in

[0179] Segmentation fault (Address not mapped to object [0x18])

[0180] After the experiment, the defect was manually reproduced. Once the reproduction was successful, it was submitted to the developers for fixing.< / char> < / char> < / char> < / char> < / double> < / char> < / char> < / double>

Claims

1. A method for automatic defect discovery in Gazebo based on coupling analysis and coverage guidance, characterized in that, The specific steps are as follows: Step (1) Source code coupling analysis and test priority determination; Step (2) Existing test execution and coverage information collection; Step (3) Construct structured prompt words; Step (4) Generation and implementation of test code driven by large models; Step (5) Coverage-guided iteration and defect determination.

2. The automatic defect discovery method for Gazebo based on coupling analysis and coverage guidance according to claim 1, characterized in that, Step (1) is as follows: Perform static analysis on the source code of the Gazebo simulation software package, extract the calling relationship, shared variables and dependency relationship between modules, and identify the key coupling points in the source code functions; then calculate the coupling index and sort them from high to low according to the index value to form a test priority sequence, and prioritize the highly coupled modules at the head of the test queue.

3. The automatic defect discovery method for Gazebo based on coupling analysis and coverage guidance according to claim 1, characterized in that, Step (2) is as follows: Execute the existing test cases provided by Gazebo, and at the same time use the coverage tool to collect code execution data, analyze the coverage of the source code by the official test, identify the lines and functions that are not covered or have low coverage, i.e., code with zero coverage or few coverage, sort them in ascending order of coverage, and prioritize identifying lines and functions with zero coverage or few coverage.

4. The automatic defect discovery method for Gazebo based on coupling analysis and coverage guidance according to claim 1, characterized in that, Step (3) is as follows: Set the prompt word template as follows: Please generate targeted test code patches for the Gazebo simulation platform, focusing on covering highly coupled modules in areas with low coverage. Test target information: Core function: [function name] Source file path: [Source file path] Coupling Index: [Coupling Value] Key external dependencies: [List of external dependency functions / variables] Code area to be covered: [Lines of code or function names with low coverage] Test requirements: Based on the above information, generate a Google Test code patch that can be directly integrated into the existing test suite; Output format: Output only C++ test code, without any explanatory text; The coupled modules analyzed in step (1) and the code units with zero coverage or few coverage times identified in step (2) are integrated according to their priority order to generate prompt information containing key test objectives, and filled into the template to guide the generation of test code.

5. The automatic defect discovery method for Gazebo based on coupling analysis and coverage guidance according to claim 1, characterized in that, Step (4) is as follows: Based on the prompt words constructed in step (3), call the large language model to generate targeted test case code patches and apply them to the original test.

6. The automatic defect discovery method for Gazebo based on coupling analysis and coverage guidance according to claim 1, characterized in that, Step (5) is as follows: If the current test coverage does not reach the threshold, repeat steps (3) to (4) to continuously generate and execute new test cases and gradually improve the coverage; monitor program behavior in real time during test execution and record all abnormal and defect information.

7. The automatic defect discovery method for Gazebo based on coupling analysis and coverage guidance according to claim 1, characterized in that, In step (1), for each function, the function-level coupling degree is calculated. : ； In the formula, a and b are weights; ExtCallsf is the number of times function f calls the external function; ExtRefsf is the number of times function f accesses external variables or classes; TotalCallsf is the sum of the number of times function f calls the external function and the number of times the function accesses external variables or classes; and Depthf is the depth of function f in the call.

Images