ROS 2 software package defect detection method based on large language model and coverage rate guidance

Abstract

The invention belongs to the field of software testing, and relates to an ROS 2 software package defect detection method based on a large language model and coverage rate guidance. According to the method, function information is extracted from a to-be-tested program, and a regular expression is used for analyzing function definition, parameter characteristics and other information; the method comprises the following steps: presetting verification standards related to a to-be-tested program, and extracting physical constraints and document constraints from the verification standards; integrating the extracted function information, physical constraint and document constraint into accurate information, filling a cue word template with the accurate information, inputting the cue word template into a large language model, and generating an initial test seed; a mutation strategy is introduced, and diversified mutation operation is performed on the generated seeds, so that test cases with higher diversity are generated; optimizing the test case by adopting a coverage rate guiding mechanism, and guiding to generate a new test case by monitoring the coverage rate in real time; through a defect verification mechanism, dominant defects such as program crash and the like can be detected, and hidden defects such as violation of physical constraints or violation of document constraints and the like can be identified.

Description

Technical Field

[0001] The invention belongs to the field of software testing, and relates to a ROS 2 software package defect detection method based on a large language model and coverage guidance, which can be used for automatically detecting defects in the ROS 2 software package. Background Art

[0002] The Robot Operating System (ROS) is an open source framework widely used in robot software development. It aims to provide developers with modular design, multi-language support and a highly flexible development environment, thereby simplifying the development process of complex robot applications. With the rapid development and widespread application of robot technology, ROS 2, as its successor, has been comprehensively optimized in terms of real-time performance, security, distributed architecture and multi-platform compatibility to better meet the needs of robot research and development in complex environments. ROS 2 integrates many powerful software packages, such as positioning and mapping, path planning, perception processing and simulation, which provide key support for the realization of multifunctional collaboration of robots. However, the implementation of these software packages often involves highly complex code logic and cross-module interaction, and their accuracy and reliability directly affect the safety, stability and functional integrity of the entire robot system. Therefore, the precise implementation of each function has a direct and important impact on the overall development of the robot system.

[0003] Existing ROS 2 software package defect detection methods mainly rely on fuzz testing. Fuzz testing is an automated testing technology widely used in the field of software testing. It aims to discover hidden defects or vulnerabilities by inputting a large amount of random or semi-structured data into the target program to trigger various boundary conditions and abnormal scenarios of the system. The core idea is to use unexpected inputs to cover code paths that are difficult to reach with traditional testing methods, thereby revealing potential functional defects, security issues and performance bottlenecks. Fuzz testing is particularly good at detecting crashes caused by improper boundary handling in software systems, such as illegal access, memory leaks, segmentation faults or resource exhaustion.

[0004] The effectiveness of fuzz testing depends largely on the diversity and coverage of generated test cases. However, since it is usually based on randomly generated input data, these inputs may not fully meet the format or syntax requirements of the target program, resulting in a low success rate of test cases in actual execution. In addition, it is difficult for randomly generated inputs to cover all key functional paths or boundary conditions, which may cause some potential functional defects to not be triggered and detected. This problem is particularly evident in complex systems with highly modular and distributed architectures such as ROS 2, where randomly generated test cases usually cannot cover the entire input space or key interaction paths of the system. In addition, fuzz testing mainly focuses on the detection of explicit defects such as program crashes, but in the actual operation scenarios of ROS 2 software packages, correctness-related defects are also widely present, such as errors that violate data constraints or functional constraints, and errors that violate physical constraints such as kinematic constraints or dynamic constraints. Although these problems do not cause program crashes, they will significantly affect the reliability and functionality of the system. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a ROS 2 software package defect detection method based on a large language model and coverage guidance. The method extracts function information from the program to be tested, and uses regular expressions to parse the definition of the function and related information such as parameter characteristics to solve the problem that randomly generated input is difficult to meet the format or syntax requirements of the target program, thereby improving the legitimacy of the input data and the execution success rate of the test case. At the same time, the verification standards related to the program to be tested are pre-set, and physical constraints and document constraints are extracted from them to provide clear guidance for detecting potential correctness defects. The extracted function information, physical constraints and document constraints are integrated into precise information to fill in the prompt word template and input into the large language model to generate initial test seeds. In order to further improve the test coverage and the ability to trigger potential defects, 15 mutation strategies are introduced to perform diversified mutation operations on the generated seeds to generate test cases with higher diversity. In addition, during the test execution process, the coverage guidance mechanism is used to optimize the test cases, and the coverage is monitored in real time to guide the generation of new test cases, thereby gradually improving the coverage and test efficiency. At the same time, through the defect verification mechanism, the present invention can not only detect explicit defects such as program crashes, but also identify implicit defects such as violations of physical constraints or violations of document constraints.

[0006] The technical solution of the present invention:

[0007] A ROS 2 package defect detection method based on a large language model and coverage guidance, the specific steps are as follows:

[0008] Step (1) Select the program to be tested, extract the code file D from it, and use regular expressions to extract function information F. And set predefined verification standards, including physical constraints and official documents of the program to be tested, and extract physical constraints P and document constraints W from them respectively. The extracted function information F, physical constraints P and document constraints W are jointly constructed into precise information.

[0009] Step (2) constructs a prompt word template to be filled: "Given the function code to be tested: <function information>, physical constraint information: <physical constraint>, and program constraint information to be tested: <document constraint>, please generate a seed for the program to be tested that meets the input format requirements and can trigger physical constraints or document constraints.". Use the precise information obtained in step (1) to fill in the prompt word template, specifically: use the function information F in the precise information to fill in "<function information>", the physical constraint P to fill in "<physical constraint>", and the document constraint W to fill in "<document constraint>". After filling, a series of natural language prompts are generated and input into the pre-selected large language model to automatically generate seeds, and all generated seeds are stored in the seed pool S.

[0010] Step (3) extracts each seed from the seed pool S generated in step (2) one by one, and applies the predefined mutation strategy to perform mutation operations one by one to generate mutation seeds. Each mutation strategy modifies different aspects of the seed to generate more diverse mutation seeds, and the mutation seeds generated in the end will all be stored in the mutation seed pool M.

[0011] Step (4) extracts each mutation seed from the mutation seed pool M generated in step (3) as a test case, and inputs these test cases into the program to be tested for execution. After the test case runs, the coverage information C generated during the execution process and the output information B of the program to be tested are collected.

[0012] Step (5) pre-sets a coverage threshold T, and evaluates the coverage information C and output information B generated in step (4) respectively. For coverage information C, determine whether the coverage information C collected during the execution of the input test case is greater than or equal to the pre-set coverage threshold T; if it is greater than or equal to the coverage threshold T, the mutation seed used as the test case is retained and stored in the mutation seed pool M, so that a new mutation seed can be generated as a test case for testing in combination with the mutation strategy; if it is less than the coverage threshold T, the mutation seed used as the test case is discarded. For output information B, defect verification is performed on output information B; if the verification is a defect, a defect report is generated and the mutation seed used as the test case is retained and stored in the mutation seed pool M, so that a new mutation seed can be generated as a test case for testing in combination with the mutation strategy.

[0013] Furthermore, step (1) specifically includes the following steps:

[0014] 1-1) Extract a code file D from the program to be tested, and use a regular expression to extract the function name, parameter name, parameter type and parameter quantity from the code file D to form function information F.

[0015] 1-2) Set predefined verification criteria, which consist of physical constraints and official documentation of the program under test.

[0016] 1-3) For physical constraints, they are pre-set to be related to the physical constraints that appear in the actual usage scenarios of the ROS 2 software package, namely kinematic constraints, dynamic constraints, geometric constraints, and environmental constraints, which together constitute the physical constraints P. The four physical constraints are as follows:

[0017] a) Kinematic constraints: restrict the relationship between the position, velocity and acceleration of a system as it moves.

[0018] b) Dynamic constraints: ensure that the system meets the requirements of force balance and energy conservation during movement.

[0019] c) Geometric constraints: specify the shape, position and spatial relationship between components of the system.

[0020] d) Environmental constraints: The behavior of the system is determined by the physical conditions or rules of the external environment.

[0021] 1-4) For the official document to be tested, the functional constraints and data constraints related to the program to be tested are extracted from it, and together constitute the document constraints W. The two document constraints are as follows:

[0022] a) Functional constraints: The output results, operation processes and interaction methods that the program should achieve under specific input conditions to ensure that the system behavior meets the expected functional specifications.

[0023] b) Data constraints: Restrictions on data values, types, structures, or relationships during program execution to ensure that the program's input, output, and intermediate data meet expected legality and consistency.

[0024] 1-5) The extracted function information F, physical constraints P and document constraints W are jointly constructed into precise information.

[0025] Furthermore, step (2) specifically includes the following steps:

[0026] 2-1) Pre-build a standardized prompt word template for input into the large language model. The format of the template is: "Given the code of the function to be tested: <function information>, physical constraint information: <physical constraint>, and the constraint information of the program to be tested: <document constraint>, please generate a seed for the program to be tested that meets the input format requirements and can trigger physical constraints or document constraints."

[0027] 2-2) Use the precise information obtained in step (1) to fill in the prompt word template. Use the function information F in the precise information to fill in "<function information>" in the prompt word template, use the physical constraint P to fill in "<physical constraint>" in the prompt word template, and use the document constraint W to fill in "<document constraint>" in the prompt word template. After filling, the generated prompt word template is input as natural language into the pre-selected large language model. The large language model will generate seeds based on the input prompt words, and finally, all the generated seeds are stored in the seed pool S.

[0028] Furthermore, step (3) specifically includes the following steps:

[0029] 3-1) Define three mutation algorithms to combine into 15 mutation strategies. The three mutation algorithms are as follows:

[0030] a) Bit Flip Mutation Algorithm: In computers, data is represented by binary bits (0 and 1). Bit flipping is to select a bit in the binary representation of data and change 0 to 1, or 1 to 0.

[0031] b) Arithmetic mutation algorithm: Perturbations are introduced by performing arithmetic operations (such as addition, subtraction, multiplication, division, etc.) on the target value.

[0032] c) Interesting value mutation algorithm: Interesting values ​​usually include boundary values ​​(such as maximum and minimum values), special values ​​(such as zero, negative numbers, NaN), and common values ​​that are meaningful in specific contexts. The core idea of ​​the algorithm is to replace the target value with a predefined "interesting value" to simulate extreme or boundary conditions.

[0033] The 15 mutation strategies are combined as follows:

[0034] a) Mutation strategy 1: bit flip mutation.

[0035] b) Mutation strategy 2: arithmetic mutation.

[0036] c) Mutation strategy 3: interesting value mutation.

[0037] d) Mutation strategy 4: first perform bit flip mutation, then perform arithmetic mutation.

[0038] e) Mutation strategy 5: first perform bit flip mutation, then perform interesting value mutation.

[0039] f) Mutation strategy 6: Perform arithmetic mutation first, then bit flip mutation.

[0040] g) Mutation strategy 7: Perform arithmetic mutation first, then interesting value mutation.

[0041] h) Mutation strategy 8: first perform interesting value mutation, then perform bit flip mutation.

[0042] i) Mutation strategy 9: First perform interesting value mutation, then perform arithmetic mutation.

[0043] j) Mutation strategy 10: first perform bit flip mutation, then arithmetic mutation, and finally interesting value mutation.

[0044] k) Mutation strategy 11: first perform bit flip mutation, then perform interesting value mutation, and finally perform arithmetic mutation.

[0045] l) Mutation strategy 12: Perform arithmetic mutation first, then bit flip mutation, and finally interesting value mutation.

[0046] m) Mutation strategy 13: Perform arithmetic mutation first, then interesting value mutation, and finally bit flip mutation.

[0047] n) Mutation strategy 14: First perform interesting value mutation, then arithmetic mutation, and finally bit flip mutation.

[0048] o) Mutation strategy 15: First perform interesting value mutation, then bit flip mutation, and finally arithmetic mutation.

[0049] 3-2) Each seed is extracted from the seed pool S one by one, and the pre-defined 15 mutation strategies are applied to each extracted seed one by one to perform mutation operations, thereby generating 15 mutation seeds, and all the generated mutation seeds are stored in the mutation seed pool M.

[0050] Furthermore, step (4) specifically includes the following steps:

[0051] 4-1) Extract each mutation seed from the mutation seed pool M generated in step (3) as a test case, and input these test cases into the program to be tested for execution.

[0052] 4-2) During the execution of the test case, a coverage collection tool is used to collect coverage information C during the execution of the test case, and at the same time, output information B of the program under test during the execution of the test case is collected.

[0053] Furthermore, step (5) specifically includes the following steps:

[0054] 5-1) A coverage threshold T is preset to evaluate the coverage information C.

[0055] 5-2) For the coverage information C, determine whether the coverage information C during the execution of the input test case is greater than or equal to the preset coverage threshold T. If it is greater than or equal to the coverage threshold T, the mutation seed used as the test case will be retained and stored in the mutation seed pool, so that new mutation seeds can be generated as test cases for testing in combination with 15 mutation strategies. If it is less than the coverage threshold T, the mutation seed used as the test case will be discarded.

[0056] 5-3) For output information B, defect verification is performed on output information B. Verification is performed based on whether a program crash occurs, whether physical constraints P are violated, and whether document constraints W are violated. If the verification is a defect, a defect report is generated and the mutation seed used as the test case is retained and stored in the mutation seed pool, so that new mutation seeds can be generated as test cases for testing in combination with 15 mutation strategies.

[0057] Compared with the prior art, the present invention has the following advantages and effects:

[0058] The method of the present invention can efficiently and automatically detect defects in the ROS 2 software package. The method constructs accurate information for filling in the prompt word template by combining the function information in the program to be tested with the constraints related to the program to be tested, and inputs the filled prompt word template into the large language model, thereby greatly improving the efficiency and quality of the large language model to generate seeds. At the same time, a variety of mutation strategies are used to enhance the diversity of generated seeds, significantly improving the comprehensiveness of defect detection. And a coverage guidance mechanism is used to optimize the coverage of test cases, cover all paths of the program to be tested as comprehensively as possible, and achieve a more thorough test effect.

[0059] Unlike traditional fuzz testing, which mainly detects program crashes, this method can detect correctness defects that violate physical constraints (such as kinematic and dynamic constraints) and document constraints (such as functional constraints and data constraints), thereby more comprehensively covering various types of defects. By combining a large language model, this method significantly reduces manual intervention, reduces defect detection costs, and greatly improves the degree of test automation and detection accuracy.

[0060] At the same time, this method has strong scalability and can flexibly integrate different large language models to adapt to diverse testing needs. This feature not only improves the user's operational convenience, but also lowers the threshold for technology application, allowing more developers to quickly adopt and deploy this technology, thereby effectively enhancing the overall stability and security of the ROS 2 software package. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1It is a flow chart of a ROS 2 software package defect detection method based on a large language model and coverage guidance according to the present invention.

[0062] Figure 2 It is a subgraph of the precise information extraction process in the ROS 2 software package defect detection method guided by a large language model and coverage in the present invention.

[0063] Figure 3 It is a process subgraph generated by seeds in the ROS 2 software package defect detection method guided by a large language model and coverage in the present invention.

[0064] Figure 4 It is a process subgraph of seed mutation in the ROS 2 software package defect detection method guided by a large language model and coverage in the present invention.

[0065] Figure 5 It is a flow subgraph of the test phase in the ROS 2 software package defect detection method guided by a large language model and coverage in the present invention. DETAILED DESCRIPTION

[0066] The method of the present invention is described in detail below in conjunction with the accompanying drawings, technical solutions and embodiments.

[0067] like Figure 1As shown, the ROS 2 software package defect detection method based on a large language model and coverage guidance of the present invention is carried out according to the following process: First, extract function information F from the program to be tested, and at the same time, extract physical constraints P and document constraints W from the verification standard to fill in the prompt word template that can accurately express the requirements. After that, the filled prompt word template is input into the pre-selected large language model to generate seeds, and the generated seeds are stored in the seed pool S. Secondly, extract seeds one by one from the seed pool S, and select a pre-defined mutation strategy for mutation operation to generate mutation seeds, and store these mutation seeds in the mutation seed pool M. Finally, extract mutation seeds one by one from the mutation seed pool M, input them as test cases into the program to be tested, and the program to be tested executes the test case, generates coverage information C and output information B during the execution process, and evaluates the coverage information C and output information B respectively. For the coverage information C, it is judged whether the collected coverage information C during the execution of the input test case is greater than or equal to the preset coverage threshold T. If it is greater than or equal to the coverage threshold T, the mutation seed of the test case is retained and stored in the mutation seed pool M, so that a new mutation seed can be generated as a test case in combination with the mutation strategy for testing. If it is less than the coverage threshold T, the mutation seed of the test case is discarded. For the output information B, a defect verification is performed on the output information B. If the verification is a defect, a defect report is generated and the mutation seed of the test case is retained and stored in the mutation seed pool M, so that a new mutation seed can be generated as a test case in combination with the mutation strategy for testing.

[0068] The following takes the ROS 2 software package TF2 as an example to explain the implementation details of each process in detail. The specific implementation methods are as follows:

[0069] (1) Figure 2As shown, accurate information extraction is performed. First, the function information in the program to be tested is extracted by reading the source code file and applying regular expressions. In this process, the code file D in the program to be tested is first selected, and the function declaration is matched using an appropriate regular expression to extract key information such as function name, parameter name, parameter type and number of parameters, and construct complete function information F. The function signature can be matched by regular expressions to extract the return value type, function name and parameter list. The extracted function parameter list is further parsed into parameter names and types. If there are multiple parameters, they are segmented for subsequent analysis. Next, a predefined verification standard is set, which includes physical constraints and official document information of the program to be tested. In terms of physical constraints, the verification standard presets constraints related to the actual application scenarios of the ROS 2 software package, including kinematic constraints, dynamic constraints, geometric constraints and environmental constraints, which together constitute physical constraints P. In terms of the official documents to be tested, functional constraints and data constraints related to the program to be tested are extracted from it, which together constitute document constraints W. Finally, the extracted function information F, physical constraints P and document constraints W are used to construct accurate information.

[0070] 1.1. Use regular expressions to extract function information:

[0071] a) For the code file static_transform_broadcaster.py, in order to parse these function information, you need to design a regular expression to match the function definition and extract relevant information.

[0072] b) Extract the function name by matching the "def" keyword and the function name part that follows it. The regular expression is "def(\w+)". Among them, "\w+" matches function names that consist of letters, numbers, and underscores.

[0073] c) Extract function parameters. Use regular expressions to match the parameter part in the function definition. Usually the parameters are contained in parentheses, such as (param1, param2). A regular expression such as "\(([^)]*)\)" will match the content in parentheses in the function definition (i.e. the parameter list).

[0074] d) Extract parameter names and types. For each parameter, if there is a type hint (such as param: Type), split it into type and parameter name. If there is no type hint, only the parameter name is extracted.

[0075] e) Use the above regular expression to match the code, and use the "def(\w+)\(([^)]*)\)" regular expression to extract the name and parameter list of each function. The extracted parameter list is further processed to break it down into individual parameters and check whether it contains type hints. If it does, the type and name are separated; if not, only the parameter name is extracted. By calculating the number of parameters of each function, the number of parameters required for each function is obtained.

[0076] f) The extracted function name, parameter name, parameter type, and number of parameters are stored in a dictionary to construct function information F.

[0077] 1.2. Extract physical constraints:

[0078] a) Extract kinematic constraints. In the TF2 software package, coordinate transformation involves position and attitude transformation between different reference coordinate systems. These transformations need to meet certain kinematic constraints. The translation cannot exceed the robot's working range, and the rotation angle should meet the rationality of the physical system.

[0079] b) Extracting dynamic constraints. In ROS 2, robots often need to interact with the external environment, such as avoiding obstacles, controlling or executing tasks. In TF2 applications, coordinate transformation must take into account the external forces and constraints on the robot to ensure the rationality of the transformation.

[0080] c) Extract geometric constraints. In the TF2 software package, geometric constraints are reflected in the spatial configuration between coordinate systems. When TF2's tf2::TransformBroadcaster is used to publish coordinate system transformations, it is necessary to ensure that the transformation is based on accurate geometric configurations.

[0081] d) Extract environmental constraints. In ROS 2, environmental constraints may involve the physical conditions when the robot interacts with the external environment. During the use of TF2, when the robot detects an obstacle or encounters complex terrain, it needs to adjust its path planning according to the environmental conditions, so that the robot can accurately consider the physical constraints of the external environment when performing coordinate transformations.

[0082] e) The extracted kinematic constraints, dynamic constraints, geometric constraints and environmental constraints are jointly constructed into physical constraints F.

[0083] 1.3. Extract document constraints:

[0084] a) Extracting functional constraints, the TF2 software package requires that the coordinate transformation should be real-time, especially when performing sensor data fusion in a dynamic environment, the delay of the coordinate transformation must not exceed the threshold set by the system.

[0085] b) Extract data constraints. The rotation quaternion (x, y, z, w) should conform to the quaternion specification, that is, x^2+y^2+z^2+w^2=1.

[0086] c) The extracted functional constraints and data constraints are jointly constructed into document constraints F.

[0087] 1.4. Construct accurate information:

[0088] The extracted function information F, physical constraints P, and document constraints W are jointly constructed into precise information.

[0089] (2) Figure 3 As shown in the figure, first, a prompt word template is constructed, and its structure is as follows: "Given the code of the function to be tested: <function information>, physical constraint information: <physical constraint>, and program constraint information to be tested: <document constraint>, please generate a seed for the program to be tested that meets the input format requirements and can trigger physical constraints or document constraints." When constructing the prompt word template, the extracted precise information is processed to ensure that all information including function information F, physical constraints P, and document constraints W are accurate, so as to fully reflect the background and running conditions of the code. Subsequently, the function information F in the precise information is used to fill in "<function information>", the physical constraint P is filled in "<physical constraint>", and the document constraint W is filled in "<document constraint>". The constructed prompt word template is automatically input into the pre-selected large language model to automatically generate seeds. Taking the function StaticTransformBroadcaster as an example, DeepSeek Coder V2 is selected as the large language model for seed generation. The model automatically generates seeds related to the function to be tested based on the constructed prompt word template, and all generated seeds are stored in the seed pool S.

[0090] After building this prompt word template and feeding it into the big language model, the resulting seed looks like this:

[0091]

[0092]

[0093] (3) Figure 4 As shown in Figure 1, seed mutation. First, 15 mutation strategies are predefined, covering a variety of different mutation operations to ensure that seeds can be effectively mutated from multiple angles. Then, each seed is extracted from the seed pool S one by one, and the 15 mutation strategies are applied to each extracted seed to generate 15 mutation seeds, which are then stored in the mutation seed pool M.

[0094] 3.1. Define three mutation algorithms and combine them into 15 mutation strategies. The three mutation algorithms are as follows:

[0095] a) Bit flip mutation algorithm:

[0096]

[0097]

[0098] b) Arithmetic mutation algorithm:

[0099] def arithmetic(value):

[0100] return value+random.choice([-1000,1000])

[0101] c) Interesting value mutation algorithm:

[0102]

[0103] Among the 15 mutation strategies, mutation strategy 8 is taken as an example. The interesting value mutation is performed first, and then the bit flip mutation is performed. The details are as follows:

[0104]

[0105] 3.2. Extract each seed from the seed pool S one by one, select 15 pre-defined mutation strategies for each seed, and perform 15 different mutation operations on it, thereby generating 15 mutation seeds, and store all the generated mutation seeds in the mutation seed pool M. Take the seeds generated in (2) as an example, select mutation strategy 8, first perform interesting value mutation, and then perform bit flip mutation. The details are as follows:

[0106] seed:

[0107]

[0108] Mutation seeds:

[0109]

[0110] (4) Figure 5As shown, the test phase is carried out. Each mutation seed is extracted from the mutation seed pool M as a test case one by one, and these test cases are input into the program to be tested for execution. During the execution of the test case, the coverage information C during the execution of the test case is collected using a coverage collection tool, and at the same time, the output information B of the program to be tested during the execution of the test case is collected. A coverage threshold T is pre-set to evaluate the coverage information C. For the coverage information C, it is determined whether the coverage information C during the execution of the input test case is greater than or equal to the pre-set coverage threshold T. If it is greater than or equal to the coverage threshold T, the mutation seed used as the test case is retained and stored in the mutation seed pool M, so that a new mutation seed can be generated as a test case for testing in combination with the mutation strategy. If it is less than the coverage threshold T, the mutation seed used as the test case is discarded. For the output information B, the output information B is verified for defects. Verify whether a program crash occurs, whether the physical constraint P is violated, and whether the document constraint W is violated. If the verification is a defect, a defect report is generated and the mutation seed used as the test case is retained and stored in the mutation seed pool so that a new mutation seed can be generated as a test case for testing in the future in combination with the mutation strategy.

[0111] 4.1. Extract each mutation seed from the mutation seed pool M one by one as a test case in the test phase and input it into TF2 for execution. Take static_transform_broadcaster.py as an example, which is the API interface for broadcast conversion in TF2.

[0112] Specifically:

[0113] Enter the test case, the test case is as follows:

[0114]

[0115]

[0116] Call the API interface static_transform_broadcaster.py to publish broadcast conversion. The corresponding code implementation content is as follows:

[0117]

[0118]

[0119] 4.2. Pre-set the coverage threshold T to 90%, and use the coverage tool to collect coverage information C. Taking Python code as an example, use the coverage tool to collect coverage. The coverage tool tracks the execution of each line of code during code execution and finally generates a coverage report. Determine whether the coverage information C during the execution process is greater than or equal to the pre-set coverage threshold T. If it is greater than or equal to the coverage threshold T, the mutation seed used as the test case will be retained and stored in the mutation seed pool M, so that new mutation seeds can be generated as test cases for testing in combination with 15 mutation strategies. If it is less than the coverage threshold T, the mutation seed used as the test case will be discarded. The details are as follows:

[0120] The following are the basic commands for using the coverage tool:

[0121] coverage run test_TF2.py

[0122] This command runs the test case test_TF2.py and collects coverage data during code execution. Once the test case is executed, you can view the coverage report with the following command:

[0123] Coverage report

[0124] This will display a command line report showing code coverage information for each file, including which lines were executed and which were not.

[0125] The specific code implementation is as follows:

[0126]

[0127]

[0128] 4.3. During the defect verification process, the output information B of the program to be tested is collected during the test case process and the defect is verified. Verification is performed on whether the program crashes, whether the physical constraint P is violated, and whether the document constraint W is violated. If the verification is a defect, a defect report is generated and the mutation seed of the test case is retained and stored in the mutation seed pool, so that new mutation seeds can be generated as test cases for testing in combination with 15 mutation strategies. The details are as follows:

[0129] Program crash detection: During the verification process, it is necessary to monitor the running status of the program, especially the execution output of the test case, to determine whether there is a crash or abnormal behavior. Capture the abnormal output of the program (such as using Python's try-except mechanism). Monitor whether the program exits normally (that is, whether the program has a normal exit code, or whether there are any unhandled exceptions).

[0130] The specific code is as follows:

[0131]

[0132] Physical constraint verification: Verify that the output value complies with known physical laws and constraints based on different physical constraints. Perform rationality checks on coordinate transformations.

[0133] The specific code is as follows:

[0134] if abs(transform.translation.x)>max_translation_x:

[0135] report_defect("Violation of kinematic constraint:Translation exceededlimit.")

[0136] Document constraint validation: Check the rotation quaternion specification of the coordinate transformation (x 2 +y 2 +z 2 +w 2 =1) Whether it meets the requirements.

[0137] The specific code is as follows:

[0138]

[0139] 4.4. Finally, the defect report will record all test cases that have passed defect verification and are evaluated as defects and their related output information B. These defects will be reproduced again, and after being confirmed as defects, they will be submitted to the developer for verification and repair.

Claims

1. A ROS2 software package defect detection method based on a large language model and coverage guidance, characterized in that: The specific steps are as follows: Step (1) Select a program to be tested, extract a code file D from it, and use a regular expression to extract function information F; and set a predefined verification standard, including physical constraints and official documents of the program to be tested, and extract physical constraints P and document constraints W from them respectively; and construct the extracted function information F, physical constraints P and document constraints W together into precise information; Step (2) constructs a prompt word template to be filled: "Given the function code to be tested: <function information>, physical constraint information: <physical constraint>, and program constraint information to be tested: <document constraint>, please generate a seed for the program to be tested that meets the input format requirements and can trigger physical constraints or document constraints."; Use the precise information obtained in step (1) to fill in the prompt word template, specifically: use the function information F in the precise information to fill in "<function information>", the physical constraint P to fill in "<physical constraint>", and the document constraint W to fill in "<document constraint>"; after completing the filling, generate a series of natural language prompts, input them into a pre-selected large language model to automatically generate seeds, and store all the generated seeds in a seed pool S; Step (3) extracts each seed from the seed pool S generated in step (2) one by one, and applies the predefined mutation strategy to perform mutation operations one by one to generate mutation seeds; the mutation seeds finally generated will all be stored in the mutation seed pool M; Step (4) extracting each mutation seed from the mutation seed pool M generated in step (3) as a test case, and inputting these test cases into the program to be tested for execution; After the test case runs, collect the coverage information C generated during the execution and the output information B of the program under test; Step (5) pre-sets a coverage threshold T, and evaluates the coverage information C and output information B generated in step (4) respectively; for the coverage information C, determines whether the coverage information C collected during the execution of the input test case is greater than or equal to the pre-set coverage threshold T; If it is greater than or equal to the coverage threshold T, the mutation seed of the test case will be retained and stored in the mutation seed pool M, so that new mutation seeds can be generated as test cases in combination with the mutation strategy for testing; if it is less than the coverage threshold T, the mutation seed of the test case will be discarded; for the output information B, the output information B will be defect verified; If the verification is a defect, a defect report is generated and the mutation seed used as the test case is retained and stored in the mutation seed pool M, so that a new mutation seed can be generated as a test case for testing in the future in combination with the mutation strategy.

2. According to claim 1, a ROS2 software package defect detection method based on a large language model and coverage guidance is characterized in that: Step (1) specifically includes the following steps: 1-1) extracting a code file D from the program to be tested, and using a regular expression to extract a function name, a parameter name, a parameter type, and a parameter quantity from the code file D to form function information F; 1-2) Set predefined verification criteria, which consist of physical constraints and official documentation of the program under test; 1-3) For physical constraints, they are pre-set to be related to the physical constraints that appear in the actual usage scenarios of the ROS2 software package, namely kinematic constraints, dynamic constraints, geometric constraints, and environmental constraints, which together constitute the physical constraints P; 1-4) For the official document to be tested, the functional constraints and data constraints related to the program to be tested are extracted from it, and together constitute the document constraints W; 1-5) The extracted function information F, physical constraints P and document constraints W are jointly constructed into precise information.

3. A ROS2 software package defect detection method based on a large language model and coverage guidance according to claim 1 or 2, characterized in that: Step (3) specifically includes the following steps: 3-1) Define three mutation algorithms to combine into 15 mutation strategies. The three mutation algorithms include bit flip mutation algorithm, arithmetic mutation algorithm and interesting value mutation algorithm; The 15 mutation strategies are combined as follows: a) Mutation strategy 1: bit flip mutation; b) Mutation strategy 2: arithmetic mutation; c) Mutation strategy 3: interesting value mutation; d) Mutation strategy 4: first perform bit flip mutation, then perform arithmetic mutation; e) Mutation strategy 5: first perform bit flip mutation, then perform interesting value mutation; f) Mutation strategy 6: perform arithmetic mutation first, then bit flip mutation; g) Mutation strategy 7: perform arithmetic mutation first, then interesting value mutation; h) Mutation strategy 8: first perform interesting value mutation, then perform bit flip mutation; i) Mutation strategy 9: first perform interesting value mutation, then perform arithmetic mutation; j) Mutation strategy 10: first perform bit flip mutation, then perform arithmetic mutation, and finally perform interesting value mutation; k) Mutation strategy 11: first perform bit flip mutation, then perform interesting value mutation, and finally perform arithmetic mutation; l) Mutation strategy 12: first perform arithmetic mutation, then bit flip mutation, and finally interesting value mutation; m) Mutation strategy 13: first perform arithmetic mutation, then perform interesting value mutation, and finally perform bit flip mutation; n) Mutation strategy 14: first perform interesting value mutation, then arithmetic mutation, and finally bit flip mutation; o) Mutation strategy 15: first perform interesting value mutation, then bit flip mutation, and finally arithmetic mutation; 3-2) Each seed is extracted from the seed pool S one by one, and the pre-defined 15 mutation strategies are applied to each extracted seed one by one to perform mutation operations, thereby generating 15 mutation seeds, and all the generated mutation seeds are stored in the mutation seed pool M.

Images