ROS 2 package defect detection method based on large language model and coverage guidance

Abstract

The present invention belongs to the field of software testing and relates to a ROS 2 software package defect detection method based on a large language model and coverage guidance. The method extracts function information from the program to be tested and uses regular expressions to parse out information such as the function's definition and parameter characteristics. Verification criteria related to the program to be tested are pre-set and physical and document constraints are extracted from them. The extracted function information, physical and document constraints are integrated into precise information to be filled into a prompt word template and input into the large language model to generate an initial test seed. A mutation strategy is introduced to perform diversified mutation operations on the generated seed to generate test cases with higher diversity. A coverage guidance mechanism is used to optimize the test cases and guide the generation of new test cases by real-time coverage monitoring. The defect verification mechanism can detect explicit defects such as program crashes and identify implicit defects such as violations of physical constraints or document constraints.

Description

Technical Field

[0001] The present invention belongs to the field of software testing and relates to a ROS 2 software package defect detection method based on a large language model and coverage guidance, which can be used to automatically detect defects in ROS 2 software packages. Background Art

[0002] The Robot Operating System (ROS) is an open-source framework widely used in robotics software development. It aims to provide developers with a modular design, multi-language support, and a highly flexible development environment, simplifying the development process of complex robotic applications. With the rapid development and widespread adoption of robotics technology, ROS 2, its successor, has been comprehensively optimized for real-time performance, security, distributed architecture, and multi-platform compatibility to better meet the needs of robotics development in complex environments. ROS 2 integrates numerous powerful software packages, such as localization and mapping, path planning, perception processing, and simulation, which provide critical support for multifunctional robotic collaboration. However, the implementation of these packages often involves highly complex code logic and cross-module interactions, and their accuracy and reliability directly impact the safety, stability, and functional integrity of the entire robotic system. Therefore, the precise implementation of each function has a direct and significant impact on the overall development of the robotic system.

[0003] Existing methods for detecting defects in ROS 2 software packages primarily rely on fuzz testing. Fuzz testing is an automated testing technique widely used in software testing. It aims to discover hidden defects or vulnerabilities by inputting large amounts of random or semi-structured data into the target program, triggering various boundary conditions and abnormal scenarios in the system. Its core concept is to use unexpected inputs to cover code paths that are difficult to access with traditional testing methods, thereby revealing potential functional defects, security issues, and performance bottlenecks. Fuzz testing is particularly adept at detecting crashes in software systems caused by improper boundary handling, such as illegal access, memory leaks, segmentation faults, or resource exhaustion.

[0004] The effectiveness of fuzz testing relies heavily on the diversity and coverage of generated test cases. However, since fuzz testing is often based on randomly generated input data, these inputs may not fully meet the format or syntax requirements of the target program, resulting in a low success rate for test cases in actual execution. Furthermore, randomly generated inputs struggle to cover all critical functional paths or edge cases, which can result in potential functional defects going untriggered and undetected. This problem is particularly pronounced in complex systems with highly modular and distributed architectures like ROS 2, where randomly generated test cases often fail to cover the system's entire input space or critical interaction paths. Furthermore, while fuzz testing primarily focuses on detecting explicit defects such as program crashes, correctness-related defects are also prevalent in the actual operation of ROS 2 software packages. These defects include violations of data or functional constraints, as well as physical constraints such as kinematic and dynamic constraints. While these issues may not cause program crashes, they can significantly impact system reliability and functionality. Summary of the Invention

[0005] To address the aforementioned issues, this paper proposes a ROS 2 software package defect detection method based on a large language model and coverage guidance. This method extracts function information from the program under test and uses regular expressions to parse relevant information, such as the function's definition and parameter characteristics. This method addresses the problem of randomly generated input failing to meet the target program's format or syntax requirements, thereby improving the validity of input data and the success rate of test case execution. Furthermore, verification criteria related to the program under test are pre-set, and physical and document constraints are extracted from them, providing clear guidance for detecting potential correctness defects. The extracted function information, physical and document constraints are integrated into precise information, which is then populated into a prompt word template and input into the large language model to generate initial test seeds. To further improve test coverage and the ability to trigger potential defects, 15 mutation strategies are introduced to perform diverse mutation operations on the generated seeds to generate test cases with greater diversity. Furthermore, during test execution, a coverage guidance mechanism is used to optimize test cases. Real-time coverage monitoring guides the generation of new test cases, thereby gradually improving coverage and test efficiency. At the same time, through the defect verification mechanism, the present invention can not only detect explicit defects such as program crashes, but also identify implicit defects such as violations of physical constraints or document constraints.

[0006] The technical solution of the present invention:

[0007] A ROS 2 package defect detection method based on a large language model and coverage guidance, with the following specific steps:

[0008] Step (1) Select the program to be tested, extract the code file D from it, and use regular expressions to extract function information F. Predefined verification criteria are set, including physical constraints and the official documentation of the program to be tested, from which physical constraints P and document constraints W are extracted respectively. The extracted function information F, physical constraints P, and document constraints W are constructed together into precise information.

[0009] Step (2) constructs a prompt word template to be filled: "Given the function code to be tested: <function information>, physical constraint information: <physical constraint>, and program constraint information to be tested: <document constraint>, please generate a seed for the program to be tested that meets the input format requirements and can trigger the physical constraint or document constraint." Use the precise information obtained in step (1) to fill in the prompt word template, specifically: use the function information F in the precise information to fill in "<function information>", the physical constraint P to fill in "<physical constraint>", and the document constraint W to fill in "<document constraint>". After the filling is completed, a series of natural language prompts are generated and input into a pre-selected large language model to automatically generate seeds, and all generated seeds are stored in the seed pool S.

[0010] Step (3) extracts each seed from the seed pool S generated in step (2) and applies a predefined mutation strategy to each seed to generate mutation seeds. Each mutation strategy modifies different aspects of the seed to generate a more diverse set of mutation seeds. The resulting mutation seeds are then stored in the mutation seed pool M.

[0011] Step (4) extracts each mutation seed from the mutation seed pool M generated in step (3) as a test case, and inputs these test cases into the program to be tested for execution. After the test case runs, the coverage information C generated during the execution process and the output information B of the program to be tested are collected.

[0012] Step (5) pre-sets the coverage threshold T, and evaluates the coverage information C and output information B generated in step (4) respectively. For the coverage information C, determine whether the coverage information C collected during the execution of the input test case is greater than or equal to the pre-set coverage threshold T; if it is greater than or equal to the coverage threshold T, then the mutation seed of the test case is retained and stored in the mutation seed pool M, so that a new mutation seed can be generated as a test case for testing in combination with the mutation strategy; if it is less than the coverage threshold T, then the mutation seed of the test case is discarded. For the output information B, defect verification is performed on the output information B; if the verification is a defect, a defect report is generated and the mutation seed of the test case is retained and stored in the mutation seed pool M, so that a new mutation seed can be generated as a test case for testing in combination with the mutation strategy.

[0013] Furthermore, step (1) specifically includes the following steps:

[0014] 1-1) Extract a code file D from the program to be tested, and use a regular expression to extract the function name, parameter name, parameter type, and parameter quantity from the code file D to form function information F.

[0015] 1-2) Set predefined verification criteria, which consists of physical constraints and official documentation of the program under test.

[0016] 1-3) For physical constraints, they are pre-set to be related to the physical constraints that appear in the actual usage scenarios of the ROS 2 software package, namely kinematic constraints, dynamic constraints, geometric constraints, and environmental constraints, which together constitute the physical constraint P. The four physical constraints are as follows:

[0017] a) Kinematic constraints: These constrain the relationship between the position, velocity, and acceleration of a system as it moves.

[0018] b) Dynamic constraints: ensure that the system meets requirements such as force balance and energy conservation during movement.

[0019] c) Geometric constraints: specify the shape, position and spatial relationship between components of the system.

[0020] d) Environmental constraints: The behavior of the system is determined by the physical conditions or rules of the external environment.

[0021] 1-4) For the official document to be tested, the functional constraints and data constraints related to the program to be tested are extracted from it and together constitute the document constraints W. The two document constraints are as follows:

[0022] a) Functional constraints: The output results, operation processes and interaction methods that the program should achieve under specific input conditions to ensure that the system behavior meets the expected functional specifications.

[0023] b) Data constraints: Restrictions on data values, types, structures, or relationships during program execution to ensure that the program's input, output, and intermediate data meet expected legality and consistency.

[0024] 1-5) The extracted function information F, physical constraints P and document constraints W are jointly constructed into precise information.

[0025] Furthermore, step (2) specifically includes the following steps:

[0026] 2-1) Pre-build a standardized prompt word template for input into the large language model. The format of the template is: "Given the function code to be tested: <function information>, physical constraint information: <physical constraint>, and program constraint information to be tested: <document constraint>, please generate a seed for the program to be tested that meets the input format requirements and can trigger physical constraints or document constraints."

[0027] 2-2) Use the precise information obtained in step (1) to fill in the prompt word template. Use the function information F in the precise information to fill in the "<function information>" in the prompt word template, use the physical constraint P to fill in the "<physical constraint>" in the prompt word template, and use the document constraint W to fill in the "<document constraint>" in the prompt word template. After the filling is completed, the generated prompt word template is input as natural language into the pre-selected large language model. The large language model will generate seeds based on the input prompt words, and finally, all the generated seeds are stored in the seed pool S.

[0028] Furthermore, step (3) specifically includes the following steps:

[0029] 3-1) Define three mutation algorithms to combine into 15 mutation strategies. The three mutation algorithms are as follows:

[0030] a) Bit Flip Mutation Algorithm: In computers, data is represented by binary bits (0 and 1). Bit flipping is to select a bit in the binary representation of the data and change 0 to 1 or vice versa.

[0031] b) Arithmetic mutation algorithm: Perturbations are introduced by performing arithmetic operations (such as addition, subtraction, multiplication, division, etc.) on the target value.

[0032] c) Interesting Value Mutation Algorithm: Interesting values ​​typically include boundary values ​​(such as maximum and minimum values), special values ​​(such as zero, negative numbers, and NaN), and common values ​​that are meaningful in specific contexts. The core idea of ​​the algorithm is to replace the target value with a predefined "interesting value" to simulate extreme or boundary conditions.

[0033] The 15 mutation strategies are combined as follows:

[0034] a) Mutation strategy 1: bit flip mutation.

[0035] b) Mutation strategy 2: arithmetic mutation.

[0036] c) Mutation strategy 3: interesting value mutation.

[0037] d) Mutation strategy 4: first perform bit-flip mutation, then perform arithmetic mutation.

[0038] e) Mutation strategy 5: first perform bit-flip mutation, then perform interesting value mutation.

[0039] f) Mutation strategy 6: Perform arithmetic mutation first, then bit flip mutation.

[0040] g) Mutation strategy 7: Perform arithmetic mutation first, then interesting value mutation.

[0041] h) Mutation strategy 8: first perform interesting value mutation, then perform bit flip mutation.

[0042] i) Mutation strategy 9: first perform interesting value mutation, then perform arithmetic mutation.

[0043] j) Mutation strategy 10: first perform bit-flip mutation, then perform arithmetic mutation, and finally perform interesting value mutation.

[0044] k) Mutation strategy 11: first perform bit-flip mutation, then perform interesting value mutation, and finally perform arithmetic mutation.

[0045] l) Mutation strategy 12: Perform arithmetic mutation first, then bit flip mutation, and finally interesting value mutation.

[0046] m) Mutation strategy 13: Perform arithmetic mutation first, then interesting value mutation, and finally bit flip mutation.

[0047] n) Mutation strategy 14: First perform interesting value mutation, then arithmetic mutation, and finally bit flip mutation.

[0048] o) Mutation strategy 15: First perform interesting value mutation, then bit flip mutation, and finally arithmetic mutation.

[0049] 3-2) Extract each seed from the seed pool S one by one, apply the 15 pre-defined mutation strategies to each extracted seed, and generate 15 mutation seeds. All generated mutation seeds are stored in the mutation seed pool M.

[0050] Furthermore, step (4) specifically includes the following steps:

[0051] 4-1) Extract each mutation seed from the mutation seed pool M generated in step (3) as a test case, and input these test cases into the program to be tested for execution.

[0052] 4-2) During the execution of the test case, a coverage collection tool is used to collect coverage information C during the execution of the test case, and at the same time, collect output information B of the program under test during the execution of the test case.

[0053] Furthermore, step (5) specifically includes the following steps:

[0054] 5-1) A coverage threshold T is preset to evaluate the coverage information C.

[0055] 5-2) Determine whether the coverage information C during the execution of the input test case is greater than or equal to the preset coverage threshold T. If it is, the mutation seed for this test case is retained and stored in the mutation seed pool, allowing subsequent generation of new mutation seeds as test cases using the 15 mutation strategies. If it is less than the coverage threshold T, the mutation seed for this test case is discarded.

[0056] 5-3) For output information B, defect verification is performed on it. Verification is performed based on whether a program crash occurs, whether physical constraint P is violated, and whether document constraint W is violated. If the verification indicates a defect, a defect report is generated and the mutation seed used for this test case is retained and stored in the mutation seed pool. This can then be combined with the 15 mutation strategies to generate new mutation seeds as test cases for testing.

[0057] Compared with the prior art, the present invention has the following advantages and effects:

[0058] The method of the present invention can efficiently and automatically detect defects in ROS 2 software packages. By combining function information in the program under test with constraints related to the program under test, this method constructs precise information to populate prompt word templates. This populated prompt word template is then input into a large language model, significantly improving the efficiency and quality of seed generation by the large language model. Furthermore, multiple mutation strategies are utilized to enhance the diversity of generated seeds, significantly improving the comprehensiveness of defect detection. Furthermore, a coverage guidance mechanism is employed to optimize test case coverage, ensuring that all paths in the program under test are covered as comprehensively as possible, achieving a more thorough testing effect.

[0059] Unlike traditional fuzz testing, which primarily detects program crashes, this method can detect correctness defects that violate both physical constraints (such as kinematic and dynamic constraints) and document constraints (such as functional and data constraints), thus providing a more comprehensive coverage of various defect types. By incorporating a large language model, this method significantly reduces manual intervention, lowers defect detection costs, and significantly improves test automation and detection accuracy.

[0060] At the same time, this method is highly scalable and can flexibly integrate different large language models to adapt to diverse testing needs. This feature not only improves user convenience but also lowers the threshold for technical application, enabling more developers to quickly adopt and deploy this technology, thereby effectively enhancing the overall stability and security of the ROS 2 software package. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1The figure is a flowchart of the ROS 2 software package defect detection method based on a large language model and coverage guidance of the present invention.

[0062] Figure 2 It is a subgraph of the precise information extraction process in the ROS 2 software package defect detection method guided by a large language model and coverage in the present invention.

[0063] Figure 3 It is a process subgraph generated by seeds in the ROS 2 software package defect detection method guided by a large language model and coverage in the present invention.

[0064] Figure 4 It is a flow subgraph of seed mutation in the ROS 2 software package defect detection method based on a large language model and coverage guidance of the present invention.

[0065] Figure 5 This is a flow subgraph of the testing phase of the ROS 2 software package defect detection method based on a large language model and coverage guidance in the present invention. DETAILED DESCRIPTION

[0066] The method of the present invention is described in detail below with reference to the accompanying drawings, technical solutions and embodiments.

[0067] like Figure 1As shown, the ROS 2 software package defect detection method based on a large language model and coverage guidance of the present invention proceeds as follows: First, function information F is extracted from the program under test. At the same time, physical constraints P and document constraints W are extracted from the verification standard to fill in prompt word templates that can accurately express the requirements. The filled prompt word templates are then input into a pre-selected large language model to generate seeds, which are then stored in a seed pool S. Secondly, seeds are extracted one by one from the seed pool S, and a pre-defined mutation strategy is selected for mutation operation to generate mutation seeds, which are then stored in a mutation seed pool M. Finally, mutation seeds are extracted one by one from the mutation seed pool M and input into the program under test as test cases. The program under test executes the test cases, generating coverage information C and output information B during the execution process. The coverage information C and output information B are then evaluated. With respect to the coverage information C, it is determined whether the coverage information C collected during the execution of the input test case is greater than or equal to a preset coverage threshold T. If the coverage is greater than or equal to the coverage threshold T, the mutation seed for this test case is retained and stored in the mutation seed pool M, allowing it to be subsequently combined with the mutation strategy to generate new mutation seeds for test cases. If the coverage is less than the coverage threshold T, the mutation seed for this test case is discarded. Defect verification is performed on the output information B. If the verification indicates a defect, a defect report is generated and the mutation seed for this test case is retained and stored in the mutation seed pool M, allowing it to be subsequently combined with the mutation strategy to generate new mutation seeds for test cases.

[0068] The following uses the ROS 2 software package TF2 as an example to explain the implementation details of each process. The specific implementation is as follows:

[0069] (1) Figure 2As shown, precise information extraction is performed. First, function information from the program under test is extracted by reading the source code file and applying regular expressions. This process begins by selecting code file D from the program under test and matching function declarations with appropriate regular expressions. Key information, such as the function name, parameter names, parameter types, and number of parameters, is extracted to construct complete function information F. Regular expressions can be used to match function signatures to extract the return value type, function name, and parameter list. The extracted function parameter list is further parsed into parameter names and types. If multiple parameters exist, they are segmented for subsequent analysis. Next, predefined verification criteria are set, including physical constraints and official documentation information for the program under test. Regarding physical constraints, the verification criteria prescribe constraints relevant to the actual application scenarios of the ROS 2 software package, including kinematic constraints, dynamic constraints, geometric constraints, and environmental constraints. These four constraints together constitute physical constraints P. Regarding the official documentation for the program under test, functional and data constraints related to the program under test are extracted, forming document constraints W. Finally, the extracted function information F, physical constraints P, and document constraints W are used to construct precise information.

[0070] 1.1. Use regular expressions to extract function information:

[0071] a) For the code file static_transform_broadcaster.py, in order to parse these function information, you need to design a regular expression to match the function definition and extract relevant information.

[0072] b) Extract the function name by matching the "def" keyword and the function name that follows it. The regular expression is "def(\w+)". "\w+" matches function names consisting of letters, numbers, and underscores.

[0073] c) Extract function parameters. Use a regular expression to match the parameters in the function definition. Parameters are usually enclosed in parentheses, such as (param1, param2). A regular expression like "\(([^)]*)\)" will match the contents of the parentheses (i.e., the parameter list) in the function definition.

[0074] d) Extract parameter names and types. For each parameter, if there is a type hint (such as param: Type), split it into type and parameter name. If there is no type hint, only the parameter name is extracted.

[0075] e) Use the above regular expression to match the code, extracting the name and parameter list of each function using the "def(\w+)\(([^)]*)\)" regular expression. Further process the extracted parameter list, breaking it down into individual parameters and checking whether it contains type hints. If so, separate the type and name; if not, extract only the parameter name. Count the number of parameters for each function to determine the number of parameters required.

[0076] f) The extracted function name, parameter name, parameter type, and number of parameters are stored in a dictionary to construct function information F.

[0077] 1.2. Extract physical constraints:

[0078] a) Extract kinematic constraints. In the TF2 software package, coordinate transformations involve changing position and orientation between different reference coordinate systems. These transformations must satisfy certain kinematic constraints. Translations cannot exceed the robot's working range, and rotation angles must satisfy the rationality of the physical system.

[0079] b) Extracting dynamic constraints: In ROS 2, robots often need to interact with the external environment, such as for obstacle avoidance, control, or mission execution. In TF2 applications, coordinate transformations must take into account both the external forces and constraints acting on the robot to ensure the rationality of the transformation.

[0080] c) Extracting geometric constraints. In the TF2 software package, geometric constraints are reflected in the spatial configuration between coordinate systems. When TF2's tf2::TransformBroadcaster is used to publish coordinate system transformations, it is necessary to ensure that the transformation is based on the accurate geometric configuration.

[0081] d) Extracting environmental constraints. In ROS 2, environmental constraints may involve the physical conditions under which the robot interacts with the external environment. During the use of TF2, when the robot detects an obstacle or encounters complex terrain, it needs to adjust its path planning based on the environmental conditions, so that the robot can accurately consider the physical constraints of the external environment when performing coordinate transformations.

[0082] e) The extracted kinematic constraints, dynamic constraints, geometric constraints and environmental constraints are jointly constructed into physical constraints F.

[0083] 1.3. Extract document constraints:

[0084] a) Extracting functional constraints. The TF2 software package requires that coordinate transformation should be real-time. Especially when performing sensor data fusion in a dynamic environment, the delay of coordinate transformation must not exceed the threshold set by the system.

[0085] b) Extract data constraints. The rotation quaternion (x, y, z, w) should conform to the quaternion specification, that is, x^2+y^2+z^2+w^2=1.

[0086] c) The extracted functional constraints and data constraints are jointly constructed into document constraints F.

[0087] 1.4. Constructing accurate information:

[0088] The extracted function information F, physical constraints P, and document constraints W are jointly constructed into precise information.

[0089] (2) Figure 3 As shown, first, a prompt word template is constructed with the following structure: "Given the code of the function to be tested: <function information>, physical constraint information: <physical constraint>, and program constraint information to be tested: <document constraint>, please generate a seed for the program to be tested that meets the input format requirements and can trigger the physical or document constraints." When constructing this prompt word template, the extracted precise information is processed to ensure that all information, including function information F, physical constraint P, and document constraint W, is accurate, thereby fully reflecting the code's context and operating conditions. Subsequently, the function information F from the precise information is used to fill in "<function information>", the physical constraint P fills in "<physical constraint>", and the document constraint W fills in "<document constraint>". The constructed prompt word template is automatically input into a pre-selected large language model to automatically generate seeds. Taking the function StaticTransformBroadcaster as an example, DeepSeek Coder V2 is selected as the large language model for seed generation. Based on the constructed prompt word template, the model automatically generates seeds related to the function to be tested, and all generated seeds are stored in a seed pool S.

[0090] After building this prompt word template and feeding it into the large language model, the resulting seed looks like this:

[0091]

[0092]

[0093] (3) Figure 4 Seed mutation is shown in Figure 1. First, 15 mutation strategies are predefined, covering a variety of different mutation operations to ensure that seeds can be effectively mutated from multiple perspectives. Then, each seed is extracted from the seed pool S one by one, and the 15 mutation strategies are applied to each extracted seed, generating 15 mutation seeds. All of these mutation seeds are stored in the mutation seed pool M.

[0094] 3.1. Define three mutation algorithms and combine them into 15 mutation strategies. The three mutation algorithms are as follows:

[0095] a) Bit flip mutation algorithm:

[0096]

[0097]

[0098] b) Arithmetic mutation algorithm:

[0099] def arithmetic(value):

[0100] return value+random.choice([-1000,1000])

[0101] c) Interesting value mutation algorithm:

[0102]

[0103] Among the 15 mutation strategies, mutation strategy 8 is taken as an example. It first performs interesting value mutation and then bit flip mutation. The details are as follows:

[0104]

[0105] 3.2. Extract each seed from the seed pool S one by one, select 15 predefined mutation strategies and perform 15 different mutation operations on them one by one, thereby generating 15 mutation seeds, and store all the generated mutation seeds in the mutation seed pool M. Take the seeds generated in (2) as an example, select mutation strategy 8, first perform interesting value mutation, and then perform bit flip mutation. The details are as follows:

[0106] seed:

[0107]

[0108] Mutation Seeds:

[0109]

[0110] (4) Figure 5As shown, the testing phase is carried out. Each mutation seed is extracted from the mutation seed pool M as a test case, and these test cases are input into the program under test for execution. During the test case execution, a coverage collection tool is used to collect coverage information C during the test case execution process. At the same time, the output information B of the program under test during the test case execution process is collected. A coverage threshold T is pre-set to evaluate the coverage information C. Based on the coverage information C, it is determined whether the coverage information C during the input test case execution process is greater than or equal to the pre-set coverage threshold T. If it is greater than or equal to the coverage threshold T, the mutation seed used as the test case is retained and stored in the mutation seed pool M so that a new mutation seed can be generated as a test case for testing in combination with the mutation strategy. If it is less than the coverage threshold T, the mutation seed used as the test case is discarded. For the output information B, defect verification is performed on the output information B. Verification is performed to determine whether a program crash occurs, whether physical constraints P are violated, and whether document constraints W are violated. If the verification is a defect, a defect report is generated and the mutation seed of the test case is retained and stored in the mutation seed pool so that a new mutation seed can be generated as a test case for testing in the future by combining the mutation strategy.

[0111] 4.1. Extract each mutation seed from the mutation seed pool M one by one as a test case in the testing phase and input it into TF2 for execution. Take static_transform_broadcaster.py as an example, which is the API interface for broadcast transformation in TF2.

[0112] Specifically:

[0113] Enter the test case, which is as follows:

[0114]

[0115]

[0116] Call the API interface static_transform_broadcaster.py to perform broadcast conversion. The corresponding code implementation content is as follows:

[0117]

[0118]

[0119] 4.2. Pre-set the coverage threshold T to 90%, and use the coverage tool to collect coverage information C. Taking Python code as an example, use the coverage tool to collect coverage. The coverage tool tracks the execution of each line of code during code execution and eventually generates a coverage report. Determine whether the coverage information C during the execution process is greater than or equal to the pre-set coverage threshold T. If it is greater than or equal to the coverage threshold T, the mutation seed used as the test case will be retained and stored in the mutation seed pool M, so that new mutation seeds can be generated as test cases for testing in combination with 15 mutation strategies. If it is less than the coverage threshold T, the mutation seed used as the test case will be discarded. The details are as follows:

[0120] The following are the basic commands for using the coverage tool:

[0121] coverage run test_TF2.py

[0122] This command runs the test case test_TF2.py and collects coverage data during code execution. Once the test case is finished executing, you can view the coverage report with the following command:

[0123] coverage report

[0124] This will display a command-line report showing code coverage information for each file, including which lines were executed and which lines were not.

[0125] The specific code implementation is as follows:

[0126]

[0127]

[0128] 4.3. During the defect verification process, the output information B of the program under test during the test case is collected and then subjected to defect verification. Verification is performed to determine whether a program crash occurs, whether physical constraints P are violated, and whether document constraints W are violated. If a defect is detected, a defect report is generated and the mutation seed for that test case is retained and stored in the mutation seed pool. This can then be combined with 15 mutation strategies to generate new mutation seeds as test cases for testing. The details are as follows:

[0129] Program crash detection: During verification, you need to monitor the program's running status, especially the test case execution output, to determine whether there are crashes or abnormal behavior. Capture the program's abnormal output (for example, using Python's try-except mechanism). Monitor whether the program exits normally (i.e., whether the program has a normal exit code or any unhandled exceptions).

[0130] The specific code is as follows:

[0131]

[0132] Physical Constraint Verification: Verify that the output value complies with known physical laws and constraints based on different physical constraints. Perform rationality checks on coordinate transformations.

[0133] The specific code is as follows:

[0134] if abs(transform.translation.x)>max_translation_x:

[0135] report_defect("Violation of kinematic constraint:Translation exceededlimit.")

[0136] Document constraint validation: Check the rotation quaternion specification of the coordinate transformation (x 2 +y 2 +z 2 +w 2 =1) whether it meets the requirements.

[0137] The specific code is as follows:

[0138]

[0139] 4.4. Finally, the defect report will record all test cases that have passed defect verification and are assessed as defects, as well as their related output information B. These defects will be reproduced again, and once confirmed as defects, they will be submitted to the developer for verification and repair.

Claims

1. A ROS2 software package defect detection method based on a large language model and coverage guidance, characterized in that: The specific steps are as follows: Step (1) Select the program to be tested, extract the code file D from it, and use regular expressions to extract function information F; set predefined verification standards, including physical constraints and official documents of the program to be tested, and extract physical constraints P and document constraints W from them respectively; construct the extracted function information F, physical constraints P and document constraints W into precise information; Step (2) constructs a prompt word template to be filled: "Given the function code to be tested: <function information>, physical constraint information: <physical constraint>, and program constraint information to be tested: <document constraint>, please generate a seed for the program to be tested that meets the input format requirements and can trigger the physical constraint or document constraint."; Use the precise information obtained in step (1) to fill in the prompt word template, specifically: use the function information F in the precise information to fill in "<function information>", the physical constraint P to fill in "<physical constraint>", and the document constraint W to fill in "<document constraint>"; after completing the filling, generate a series of natural language prompts, input them into a pre-selected large language model to automatically generate seeds, and store all the generated seeds in the seed pool S; Step (3) extracts each seed from the seed pool S generated in step (2) one by one, and applies the predefined mutation strategy to perform mutation operations one by one to generate mutation seeds; the mutation seeds finally generated are all stored in the mutation seed pool M; Step (4) extracting each mutation seed from the mutation seed pool M generated in step (3) as a test case, and inputting these test cases into the program to be tested for execution; After the test case is finished running, the coverage information C generated during the execution and the output information B of the program under test are collected; Step (5) pre-sets a coverage threshold T, and evaluates the coverage information C and output information B generated in step (4) respectively; for the coverage information C, determines whether the coverage information C collected during the execution of the input test case is greater than or equal to the pre-set coverage threshold T; If the coverage rate is greater than or equal to the coverage threshold T, the mutation seed of the test case will be retained and stored in the mutation seed pool M, so that new mutation seeds can be generated as test cases for testing in combination with the mutation strategy. If the coverage rate is less than the coverage threshold T, the mutation seed of the test case will be discarded. For the output information B, defect verification is performed on the output information B. If the verification is a defect, a defect report is generated and the mutation seed of the test case is retained and stored in the mutation seed pool M, so that a new mutation seed can be generated as a test case for testing in the future in combination with the mutation strategy.

2. A ROS2 software package defect detection method based on a large language model and coverage guidance according to claim 1, characterized in that: Step (1) specifically includes the following steps: 1-1) extracting a code file D from the program to be tested, and using a regular expression to extract the function name, parameter name, parameter type, and parameter number from the code file D to form function information F; 1-2) Set predefined verification criteria, which consists of physical constraints and official documentation of the program under test; 1-3) For physical constraints, they are pre-set to be related to the physical constraints that appear in the actual usage scenarios of the ROS2 software package, namely kinematic constraints, dynamic constraints, geometric constraints, and environmental constraints, which together constitute the physical constraint P; 1-4) For the official document to be tested, the functional constraints and data constraints related to the program to be tested are extracted from it and together constitute the document constraints W; 1-5) The extracted function information F, physical constraints P and document constraints W are jointly constructed into precise information.

3. A ROS2 software package defect detection method based on a large language model and coverage guidance according to claim 1 or 2, characterized in that: Step (3) specifically includes the following steps: 3-1) Define three mutation algorithms to combine into 15 mutation strategies. The three mutation algorithms include bit flip mutation algorithm, arithmetic mutation algorithm, and interesting value mutation algorithm. The 15 mutation strategies are combined as follows: a) Mutation strategy 1: bit flip mutation; b) Mutation strategy 2: arithmetic mutation; c) Mutation strategy 3: interesting value mutation; d) Mutation strategy 4: first perform bit-flip mutation, then perform arithmetic mutation; e) Mutation strategy 5: first perform bit-flip mutation, then perform interesting value mutation; f) Mutation strategy 6: perform arithmetic mutation first, then bit flip mutation; g) Mutation strategy 7: perform arithmetic mutation first, then interesting value mutation; h) Mutation strategy 8: first perform interesting value mutation, then perform bit flip mutation; i) Mutation strategy 9: first perform interesting value mutation, then perform arithmetic mutation; j) Mutation strategy 10: first perform bit-flip mutation, then perform arithmetic mutation, and finally perform interesting value mutation; k) Mutation strategy 11: first perform bit-flip mutation, then perform interesting value mutation, and finally perform arithmetic mutation; l) Mutation strategy 12: first perform arithmetic mutation, then bit flip mutation, and finally interesting value mutation; m) Mutation strategy 13: first perform arithmetic mutation, then perform interesting value mutation, and finally perform bit flip mutation; n) Mutation strategy 14: first perform interesting value mutation, then perform arithmetic mutation, and finally perform bit flip mutation; o) Mutation strategy 15: first perform interesting value mutation, then bit flip mutation, and finally arithmetic mutation; 3-2) Extract each seed from the seed pool S one by one, apply the 15 pre-defined mutation strategies to each extracted seed, and generate 15 mutation seeds. All generated mutation seeds are stored in the mutation seed pool M.

Images