A ROS 2 defect detection method based on large language model and differential testing

Abstract

The present invention belongs to the field of software testing and relates to a ROS 2 defect detection method based on a large language model and differential testing, which can be used to automatically detect defects in ROS 2. This method constructs accurate prompt words by combining the semantic information of the program to be tested, thereby significantly improving the efficiency and quality of seed generation by the large language model. At the same time, a variety of mutation strategies are introduced to enhance the diversity of seeds, thereby improving the comprehensiveness of defect detection. The present invention supports differential testing of the same functional modules implemented in different programming languages. This differentiated testing method can effectively reveal behavioral inconsistencies caused by differences in implementation languages, ensure functional consistency between different language implementations, and thus improve the stability and reliability of the system. Through combination with a large language model, the present invention significantly reduces manual intervention, reduces the cost of defect detection, and greatly improves the degree of automation and accuracy of testing.

Description

Technical Field

[0001] The present invention belongs to the field of software testing and relates to a ROS 2 defect detection method based on a large language model and differential testing, which can be used to automatically detect defects in ROS 2. Background Art

[0002] The Robot Operating System (ROS) is an open-source framework for robotics software development, designed to simplify the development and implementation of robotic applications. As ROS continues to evolve, ROS 2, its second-generation version, optimizes its core architecture to meet the complex and diverse application requirements of modern robotics. However, the multi-module, multi-language nature of ROS 2 introduces significant risks of software defects, which can cause unforeseen failures and compromise the safety, reliability, and functional integrity of robotic systems. Currently, research on the security and reliability of ROS 2 focuses on static analysis and fuzz testing methods.

[0003] However, static analysis and fuzz testing methods have some shortcomings. Static analysis can produce false positives and be time-consuming, exposing the limitations of traditional testing methods. Fuzz testing has demonstrated certain advantages in catching software vulnerabilities, but its detection effectiveness relies on the diversity and coverage of input data. If the generated input data fails to hit critical paths or edge cases, potential issues may be missed. Fuzz testing is particularly difficult to generate test cases that adequately cover critical scenarios in complex systems like ROS 2, which has a multi-layered, distributed architecture. These limitations indicate that traditional static analysis and fuzz testing methods cannot fully meet the defect detection needs of ROS 2.

[0004] Against this backdrop, AI-powered large language models offer new possibilities for defect detection. Detection methods based on large language models leverage their understanding of natural language and context to automatically extract functional logic and contextual information from source code, generating high-coverage, high-quality test cases. This method combines large language models with differential testing, using seeds generated by the large language model as test cases to differentially test functional modules implemented in different languages ​​within ROS 2 to detect functional consistency issues. This method not only significantly improves the efficiency of ROS 2 defect detection but also broadens the depth and scope of detection, providing a new technical path for capturing unknown defects. Summary of the Invention

[0005] In order to solve the above problems, the present invention proposes a ROS 2 defect detection method based on a large language model and differential testing. First, the present invention uses the AST abstract syntax tree to obtain the public information D of the program to be tested. context and function information D function, in order to construct a prompt word template that can accurately express the needs. Then, the filled template is input into the pre-selected large language model to generate seeds, and the generated seeds are stored in the seed pool T all Secondly, from the seed pool T all Extract seeds one by one, and randomly select one of the 30 predefined mutation strategies to perform mutation operations, thereby generating different mutation seeds, and store these mutation seeds in the mutation seed pool T mutation Finally, two API interfaces with the same functions implemented by C++ and Python in the program to be tested are pre-selected, namely API interface 1 and API interface 2 (since the ROS 2 system is mainly developed using C++ and Python, this paper only tests the interfaces of these two mainstream programming languages ​​to analyze their differences in function implementation and performance). mutation Extract mutation seeds one by one, input them into the program to be tested as test cases, and execute them by pre-selected API interface 1 and API interface 2 respectively, and obtain the output information Output1 of API interface 1 and the output information Output2 of API interface 2. If the output information Output1 and the output information Output2 are inconsistent, the difference between the output information Output1 and the output information Output2 is evaluated using the pre-set threshold T. If the difference between the output information Output1 and the output information Output2 is greater than the threshold T, the detailed information is recorded and a relevant report is generated, and the mutation seeds that generate inconsistent and unreasonable test cases are retained in the seed pool T all In order to subsequently combine the mutation strategy to generate new mutation seeds as test cases for differential testing, thereby further improving the diversity of test cases and the quality of defect detection.

[0006] The technical solution of the present invention:

[0007] A ROS 2 defect detection method based on a large language model and differential testing, with the following steps:

[0008] Step (1) Select the program to be tested and extract the code file F from it. Generate its corresponding AST abstract syntax tree and obtain public information D context And function information D function .

[0009] Step (2) constructs a prompt word template to be filled: "Given <context information>, the code of the function to be tested: <function information>, please generate a high coverage seed for the function to be tested." Using the public information D obtained in step (1) context Fill in "<context information>", function information D functionFill in "<function information>" and finally obtain a series of natural language accurate prompts for inputting into the large language model, input them into the pre-selected large language model, thereby generating seeds, and all the generated seeds are stored in the seed pool T all middle.

[0010] Step (3) is to use the seed pool T generated in step (2) all Extract seeds one by one from the predefined mutation strategies, and then randomly select one of them for mutation operation. Each mutation strategy modifies different aspects of the seed to generate diverse mutation seeds, and the resulting mutation seeds are stored in the mutation seed pool T mutation middle.

[0011] Step (4) pre-selects two API interfaces with the same functions implemented by C++ and Python in the program to be tested, namely API interface 1 and API interface 2, and then selects the mutation seed pool T generated in step (3) mutation Extract mutation seeds as test cases, input these test cases into the program to be tested, execute them by pre-selected API interface 1 and API interface 2 respectively, and obtain API interface 1 output information Output1 and API interface 2 output information Output2.

[0012] Step (5) pre-sets a reasonable difference threshold T, and evaluates the output information Output1 and output information Output2 generated in step (4). It detects whether the output information Output1 and output information Output2 are consistent. If they are consistent, the mutation seed that generates the test case is discarded. If they are inconsistent, it further detects whether the difference between the output information Output1 and output information Output2 is less than or equal to the threshold T. If the difference is less than or equal to the threshold T, the mutation seed that generates the test case is discarded; if the difference is greater than the threshold T, a report is generated for the relevant information that is inconsistent and unreasonable, and the mutation seed that generates the test case is retained in the seed pool T all In this way, it is convenient to combine the mutation strategy to generate new mutation seeds as test cases for differential testing.

[0013] Furthermore, step (1) specifically includes the following steps:

[0014] 1-1) A code file F is extracted from the program to be tested.

[0015] 1-2) Generate the AST abstract syntax tree corresponding to the code file F, and extract the global variables and global constants in the code file F to form the public information D context .

[0016] 1-3) Traverse the function nodes in the AST abstract syntax tree, extract all function contents (including function name, return type, parameter list, etc.), and record the calling relationship within each function to form function information D function .

[0017] Furthermore, step (2) specifically includes the following steps:

[0018] 2-1) Construct a standardized prompt word template for input into the large language model. The format of the template is: "Given <context information>, the code of the function to be tested: <function information>, please generate a high coverage seed for the function to be tested."

[0019] 2-2) Using the extracted public information D context Fill in the "<context information>" in the prompt word template, and use the function information D function Fill in the "<function information>" in the prompt word template. After filling, the generated prompt word template is input as natural language into the pre-selected large language model. The large language model will generate high-coverage seeds based on the input prompt words, and finally, all the generated seeds are stored in the seed pool T all middle.

[0020] Furthermore, step (3) specifically includes the following steps:

[0021] 3-1) Define 6 mutation algorithms to combine into 30 mutation strategies. The 6 mutation algorithms are as follows:

[0022] a) Overflow mutation: By changing the value in the seed, the value exceeds its valid range.

[0023] b) Sign mutation: changes the sign of a value (positive to negative or negative to positive).

[0024] c) Boundary value mutation: Modify the value in the seed to the boundary value of the data type, such as the maximum value, minimum value, or a value close to the boundary value.

[0025] d) Precision mutation: By modifying the precision of floating-point numbers, the program's response to inputs of different numerical precision is tested.

[0026] e) Type mutation: Changing the data type. For example, converting an integer to a floating-point type, or vice versa.

[0027] f) Invalid number mutation: Replace the numbers in the seed with invalid numbers, such as NaN (Not-a-Number), infinity, invalid negative numbers (such as -0), or values ​​outside the valid range.

[0028] The 30 mutation strategies are combined as follows:

[0029] a) Mutation strategy 1: perform overflow mutation first, then sign mutation.

[0030] b) Mutation strategy 2: first perform overflow mutation, then perform boundary value mutation.

[0031] c) Mutation strategy 3: perform overflow mutation first, then perform precision mutation.

[0032] d) Mutation strategy 4: perform overflow mutation first, then type mutation.

[0033] e) Mutation strategy 5: perform overflow mutation first, then perform invalid number mutation.

[0034] f) Mutation strategy 6: perform sign mutation first, then overflow mutation.

[0035] g) Mutation strategy 7: perform sign mutation first, then perform boundary value mutation.

[0036] h) Mutation strategy 8: perform sign mutation first, then precision mutation.

[0037] i) Mutation strategy 9: perform symbol mutation first, then type mutation.

[0038] j) Mutation strategy 10: perform symbol mutation first, then perform invalid numeric mutation.

[0039] k) Mutation strategy 11: perform boundary value mutation first, then overflow mutation.

[0040] l) Mutation strategy 12: perform boundary value mutation first, then sign mutation.

[0041] m) Mutation strategy 13: First perform boundary value mutation, then perform accuracy mutation.

[0042] n) Mutation strategy 14: perform boundary value mutation first, then type mutation.

[0043] o) Mutation strategy 15: first perform boundary value mutation, then perform invalid number mutation.

[0044] p) Mutation strategy 16: Perform precision mutation first, then overflow mutation.

[0045] q) Mutation strategy 17: perform precision mutation first, then sign mutation.

[0046] r) Mutation strategy 18: First perform precision mutation, then perform boundary value mutation.

[0047] s) Mutation strategy 19: perform precision mutation first, then type mutation.

[0048] t) Mutation strategy 20: perform precision mutation first, then perform invalid number mutation.

[0049] u) Mutation strategy 21: Perform type mutation first, then overflow mutation.

[0050] v) Mutation strategy 22: perform type mutation first, then symbol mutation.

[0051] w) Mutation strategy 23: Perform type mutation first, then perform boundary value mutation.

[0052] x) Mutation strategy 24: Perform type mutation first, then precision mutation.

[0053] y) Mutation strategy 25: perform type mutation first, then perform invalid number mutation.

[0054] z) Mutation strategy 26: first perform invalid digital mutation, then perform overflow mutation.

[0055] aa) Mutation strategy 27: first perform invalid numeric mutation, then perform symbol mutation.

[0056] bb) Mutation strategy 28: First perform invalid number mutation, then perform boundary value mutation.

[0057] cc) Mutation strategy 29: first perform invalid number mutation, then perform precision mutation.

[0058] dd) Mutation strategy 30: first perform invalid numeric mutation, then perform type mutation.

[0059] 3-2) From the seed pool T all Extract each seed one by one, and randomly select one of the 30 predefined mutation strategies for each extracted seed to perform mutation operation, thereby generating mutation seeds, and storing the generated mutation seeds in the mutation seed pool T mutation middle.

[0060] Furthermore, step (4) specifically includes the following steps:

[0061] 4-1) Preselect two API interfaces with the same functions implemented by C++ and Python in the program to be tested, namely API interface 1 and API interface 2.

[0062] 4-2) Then, from the mutant seed pool T mutation Extract each mutation seed one by one as a test case.

[0063] 4-3) Use the same test case to input into the program to be tested, execute it by the pre-selected API interface 1 and API interface 2 respectively, and obtain the output information Output1 of API interface 1 and the output information Output2 of API interface 2.

[0064] Furthermore, step (5) specifically includes the following steps:

[0065] 5-1) Check whether the output information Output1 and output information Output2 are consistent. If they are consistent, the mutation seed that generated the test case will be discarded. Otherwise, proceed to the next step of difference evaluation.

[0066] 5-2) A reasonable difference threshold T is set based on the differences in data accuracy processing among different programs to reduce the impact of accuracy differences on experimental results.

[0067] 5-3) Evaluate the difference between the inconsistent output information Output1 and the output information Output2. If the difference is less than or equal to the preset threshold T, the mutation seed that generated the test case does not need to be retained. Although the test case triggers an inconsistent result, it is within a reasonable range. If the difference is greater than the set threshold T, it is considered an inconsistency and the difference is not within a reasonable abnormal situation. The relevant information is recorded and reported, and the mutation seed that generated the test case is retained in the seed pool T. all In this way, it is convenient to combine mutation strategies to generate new mutation seeds as test cases for differential testing, helping to identify potential abnormal situations and program defects, thereby improving the quality and coverage of test cases.

[0068] Compared with the prior art, the present invention has the following advantages and effects:

[0069] This paper proposes a ROS 2 defect detection method based on a large language model and differential testing, enabling efficient and automated detection of potential defects in the ROS 2 system. This method constructs precise prompt words by incorporating semantic information from the program under test, significantly improving the efficiency and quality of seed generation using the large language model. Furthermore, multiple mutation strategies are introduced to enhance seed diversity, thereby improving the comprehensiveness of defect detection. Given that ROS 2 contains multiple modules with identical functionality but implemented in different languages, this paper supports differential testing of modules with the same functionality implemented in different programming languages. This differential testing approach effectively reveals behavioral inconsistencies caused by implementation language differences, ensuring functional consistency across different language implementations, and thus improving system stability and reliability. By integrating this with the large language model, this paper significantly reduces manual intervention, lowers the cost of defect detection, and significantly improves the level of test automation and accuracy. Furthermore, this method is highly scalable, enabling flexible switching and integration of different large language models to accommodate diverse testing needs. This flexibility not only optimizes the user experience but also lowers the barrier to entry for technology adoption, enabling more developers to easily apply this technology and further enhancing the stability and security of the ROS 2 system. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a flow chart of the ROS 2 defect detection method based on a large language model and differential testing.

[0071] Figure 2 This is a subgraph of the context information extraction process in the ROS 2 defect detection method based on a large language model and differential testing in the present invention.

[0072] Figure 3 It is a process subgraph of seed generation in the ROS 2 defect detection method based on a large language model and differential testing of the present invention.

[0073] Figure 4 This is a flow subgraph of seed mutation in the ROS 2 defect detection method based on a large language model and differential testing of the present invention.

[0074] Figure 5 This is a flow subgraph of differential testing in the ROS 2 defect detection method based on a large language model and differential testing of the present invention. DETAILED DESCRIPTION

[0075] The method of the present invention is described in detail below with reference to the accompanying drawings, technical solutions and embodiments.

[0076] like Figure 1As shown in the figure, the ROS 2 defect detection method based on the large language model and differential testing of the present invention is performed as follows: first, a code file F is obtained from the program to be tested, context analysis is performed on the code file F, and its corresponding AST abstract syntax tree is generated using a specific processing algorithm, and common information D is automatically identified and extracted from it. context and function information D function Subsequently, these key information are integrated into the pre-set prompt word template to be filled, and the template content is transmitted to the pre-selected large language model. The large language model generates seeds based on these precise prompts and stores the generated seeds in the seed pool T all Secondly, from the seed pool T all Extract seeds one by one, randomly select one of the pre-set mutation strategies to perform mutation operations on them, generate diverse mutation seeds, and store the mutated seeds in the mutation seed pool T mutation Then, from the mutant seed pool T mutation Extract mutation seeds one by one and input them into the program to be tested as test cases for execution. Finally, evaluate the output information after execution. If the output information is consistent, it will be discarded as the mutation seed of the test case. If the output information is inconsistent, proceed to the next step of difference evaluation. In the difference evaluation stage, if the difference in the output information is greater than the pre-set reasonable threshold T, the relevant information will be recorded and reported, and the mutation seed of the test case will be retained in the seed pool T all In this way, it is convenient to combine the mutation strategy to generate new mutation seeds as test cases for differential testing. If the difference in the output information is less than or equal to the threshold T, it is not considered an anomaly and the mutation seed used as the test case is discarded.

[0077] The following uses the coordinate conversion program TF2 in ROS 2 as an example to explain the implementation details of each process. The specific implementation is as follows:

[0078] (1) Figure 2As shown, context information extraction is performed. First, the code file F is extracted from the program to be tested, and an AST abstract syntax tree is generated for the code file F. According to the language type of the code file F to be tested, the corresponding AST abstract syntax tree generation tool is selected. Taking C++ and Python as examples, for C++ code files, tools such as Clang are used to generate the corresponding AST abstract syntax tree. For Python code files, the ast module that comes with Python is used to generate the corresponding AST abstract syntax tree. Use a traversal algorithm to traverse all nodes in the abstract syntax tree to find global variables and constants. Global variables are usually declared outside the class or outside the function and are not in any local scope. Global variables marked with keywords such as static or extern are identified, especially in C++ code. To identify global constants, keywords such as const or constexpr are usually used. By identifying the nodes of the abstract syntax tree, global variables and global constants in the code and other related public information are extracted, and this information is integrated into public information D context On this basis, all function definitions and declarations are searched in the AST abstract syntax tree to obtain the complete definition content of each function. At the same time, each function call relationship is retrieved, and the complete content of each function and its call relationship information are stored in the function information D function middle.

[0079] 1.1. Generate AST abstract syntax tree of code file:

[0080] a) Use the Clang tool to build the AST abstract syntax tree for the code file static_transform_broadcaster.h.

[0081] b) Use the AST abstract syntax tree to identify all top-level functions and build a function list, such as the StaticTransformBroadcaster function in the code file, which is specifically used to broadcast static coordinate transformation information.

[0082] c) By identifying the nodes of the AST abstract syntax tree generated by the code file, extract the global constants and global variables used in the file. The global constants are " / tf_static" and "StaticBroadcasterQoS()", and the global variables are "publisher_" and "net_message_".

[0083] d) Integrate its global variables and global constants into public information D context .

[0084] e) By identifying the AST abstract syntax tree nodes generated by the code file and traversing the generated AST abstract syntax tree nodes in the function list, the definitions of the relevant functions in the file are extracted. For example, the definition of the tested function StaticTransformBroadcaster is extracted as follows:

[0085]

[0086]

[0087] 1.2. Extract function content and function call relationship:

[0088] a) By identifying the definition node of each function through the AST abstract syntax tree, the internal implementation part of the function can be accessed through the child nodes of the AST abstract syntax tree.

[0089] b) Function calls are usually represented by CALL_EXPR nodes, which are achieved by traversing all CALL_EXPR nodes in the AST abstract syntax tree and recording the calling of each function, especially which function calls which other functions.

[0090] 1.3. Build context information:

[0091] a) Store the complete content and calling relationship of the function into function information D function middle..

[0092] b) Public information D context and function information D function The context information required for seed generation is integrated into the build. This enables efficient retrieval of relevant function information during the subsequent seed generation phase, ensuring that the code behavior is fully analyzed and fully covered.

[0093] (2) Figure 2 As shown, first, a prompt word template is designed, whose structure is: "Given <context information>, the code of the function to be tested: <function information>, please generate a seed with high coverage for the function to be tested." When constructing the prompt word template, the extracted context information is accurately processed to ensure that the public information D is included. context and function information D functionAll details including the code are accurate, so as to fully reflect the background and execution conditions of the code. Once the prompt word template is built, the system will automatically input it into the selected large language model for processing to generate the corresponding seeds. Taking the function StaticTransformBroadcaster as an example, DeepSeek Coder V2 is selected as the large language model for seed generation. The model automatically generates seeds related to the tested function based on the constructed prompt word template. These seeds contain test cases with high coverage. The generated seeds are stored in the seed pool T all in preparation for subsequent mutation operations.

[0094] After building this prompt word template and feeding it into the large language model, the resulting seed looks like this:

[0095]

[0096]

[0097] (3) Figure 3 As shown in Figure 2, seed mutation. First, 30 mutation strategies are predefined, covering different mutation operations, to ensure that seeds can be effectively mutated from multiple perspectives. Then, from the seed pool T all Extract seeds one by one and randomly select one of the predefined 30 mutation strategies for mutation operation. Mutation seeds generated by mutation operation are stored in mutation seed pool T mutation For subsequent differential testing.

[0098] 3.1. Define 6 mutation algorithms and combine them into 30 mutation strategies. The 6 mutation algorithms are as follows:

[0099] a) Overflow mutation:

[0100]

[0101] b) Symbol mutation:

[0102] def sign_mutation(seed:Union[int,float])->Union[int,float]:

[0103] """Sign mutation: Change the sign of a value (positive to negative or negative to positive)"""

[0104] return-seed

[0105] c) Boundary value mutation:

[0106]

[0107]

[0108] d) Accuracy mutation:

[0109] e) Type of mutation:

[0110] f) Invalid digital mutation:

[0111] Among the 30 mutation strategies, we take mutation strategy 6 as an example, the details are as follows:

[0112]

[0113] 3.2. From the seed pool T all Extract each seed one by one, randomly select one of the 30 predefined mutation strategies to perform mutation operation, thereby generating mutation seeds, and store all generated mutation seeds in the mutation seed pool T mutation Taking the seed generated in (2) as an example, we select mutation strategy 6 to perform mutation operation, first perform symbol mutation, then perform overflow mutation. The details are as follows:

[0114] seed:

[0115]

[0116]

[0117] Mutation Seeds:

[0118]

[0119]

[0120] (4) Figure 4 As shown, differential testing is performed. By selecting mutant seed pool T mutationExtract mutation seeds one by one as test cases, combine the selected API interfaces with the same functions implemented in C++ and Python in TF 2, perform differential testing, input the test cases into the selected API interfaces implemented in C++ and Python, obtain the output information after the execution of the two API interfaces, and evaluate the output information of the two. If the output information of the two is consistent, the mutation seed that generates the test case is discarded. If the output information is inconsistent, the next step of difference evaluation is carried out. In the difference evaluation stage, a reasonable difference threshold T is set in advance. If the difference value between the two is less than or equal to the threshold T, the mutation seed that generates the test case is discarded. If the difference value between the two is greater than the threshold T, the detailed relevant information is reported, and the mutation seed that generates the test case is retained in the seed pool T all In this way, it is convenient to combine the mutation strategy to generate new mutation seeds as test cases for differential testing.

[0121] 4.1. Mutate seed pool T one by one mutation Extract mutation seeds as test cases for differential testing and input them into TF2 APIs implemented in C++ and Python, respectively. The output is obtained and evaluated. For example, the C++ API static_transform_broadcaster.h and the Python API static_transform_broadcaster.py are both broadcast transformation APIs in TF2.

[0122] Specifically:

[0123] Using the same test case, the test case is as follows:

[0124]

[0125]

[0126] Call the C++ programming language to implement the API interface static_transform_broadcaster.h to publish broadcast conversion. The corresponding code implementation content is as follows:

[0127]

[0128]

[0129] Call the Python programming language to implement the API interface static_transform_broadcaster.py to publish broadcast conversion. The corresponding code implementation content is as follows:

[0130]

[0131] 4.2. Receive the output information Output1 of the API interface implemented by calling C++ and the output information Output2 of the API interface implemented by calling Python, as shown below:

[0132]

[0133] 4.3. Obtain output information Output1 and output information Output2, and evaluate output information Output1 and output information Output2. First, determine whether output information Output1 and output information Output2 are consistent. Use "==" to make the judgment. If they are consistent, discard the mutation seed as the test case. If they are inconsistent, proceed to the next step of evaluation, as shown below:

[0134]

[0135]

[0136] 4.4. If the output information Output1 and the output information Output2 are inconsistent, a difference evaluation is performed. According to the PEP 485 proposal introduced after Python 3.5, it is recommended to use an approximate equality method when comparing two floating-point numbers, rather than relying on exact equality judgment. This method allows two numerical values ​​and a tolerance error value (usually set to 10^-5) to be passed in. When the numerical difference is less than the error value, it is considered equal. Therefore, the difference threshold T is set to 10^-5 to reduce the impact of precision differences on experimental results. If the difference is less than or equal to the threshold T, it will be discarded as the mutation seed of the test case. Although it triggers the inconsistency between the two, the difference is reasonable. If the difference is greater than the threshold T, the information is reported, and the mutation seed of the test case is retained and stored in the seed pool for subsequent combination with the mutation strategy to generate a new mutation seed as a test case for subsequent differential testing, as shown below:

[0137]

[0138] 4.5. Finally, the report will record many test cases and difference information that trigger inconsistencies in the outputs of the two and the difference exceeds the preset reasonable threshold T. The abnormal situation will be reproduced again and submitted to the developer after being confirmed as a defect.

Claims

1. A ROS2 defect detection method based on a large language model and differential testing, characterized in that: The specific steps are as follows: Step (1) Select the program to be tested, extract the code file F from it; generate its corresponding AST abstract syntax tree, and obtain public information D context And function information D function ; Step (2) constructs a prompt word template to be filled: "Given <context information>, the code of the function to be tested: <function information>, please generate a high coverage seed for the function to be tested."; using the public information D obtained in step (1) context Fill in "<context information>", function information D function Fill in "<function information>" and finally obtain a series of natural language accurate prompts for inputting into the large language model, input them into the pre-selected large language model, thereby generating seeds, and all the generated seeds are stored in the seed pool T all middle; Step (3) is to use the seed pool T generated in step (2) all Extract seeds one by one, then randomly select one of the predefined mutation strategies for mutation operation; each mutation strategy modifies different aspects of the seed to generate diverse mutation seeds, and the resulting mutation seeds are stored in the mutation seed pool T mutation middle; Step (4) pre-selects two API interfaces with the same functions implemented by C++ and Python in the program to be tested, namely API interface 1 and API interface 2, and then selects the mutation seed pool T generated in step (3) mutation Extract mutation seeds as test cases, input these test cases into the program to be tested, execute them by pre-selected API interface 1 and API interface 2 respectively, and obtain the output information Output1 of API interface 1 and the output information Output2 of API interface 2; Step (5) pre-sets a reasonable difference threshold T, and evaluates the output information Output1 and output information Output2 generated in step (4), and detects whether the output information Output1 and output information Output2 are consistent. If they are consistent, the mutation seed generating the test case is discarded. If they are inconsistent, further detect whether the difference between the output information Output1 and output information Output2 is less than or equal to the threshold T; if the difference is less than or equal to the threshold T, the mutation seed generating the test case is discarded; if the difference is greater than the threshold T, a report is generated for the relevant information of the inconsistency and unreasonable difference, and the mutation seed generating the test case is retained in the seed pool T all In this way, it is convenient to combine the mutation strategy to generate new mutation seeds as test cases for differential testing.

2. The ROS2 defect detection method based on a large language model and differential testing according to claim 1, characterized in that: Step (3) specifically includes the following steps: 3-1) Define 6 mutation algorithms to combine into 30 mutation strategies. The 6 mutation algorithms are as follows: a) Overflow mutation: By changing the value in the seed, the value exceeds its valid range; b) Sign mutation: changes the sign of a value, from positive to negative or vice versa; c) Boundary value mutation: Modify the value in the seed to the boundary value of the data type; d) Precision mutation: By modifying the precision of floating-point numbers, the program's response to inputs with different numerical precision is tested; e) Type mutation: changing the type of data; f) Invalid number mutation: replace the numbers in the seed with invalid numbers; The 30 mutation strategies are combined as follows: a) Mutation strategy 1: first perform overflow mutation, then perform sign mutation; b) Mutation strategy 2: first perform overflow mutation, then perform boundary mutation; c) Mutation strategy 3: first perform overflow mutation, then perform precision mutation; d) Mutation strategy 4: first perform overflow mutation, then perform type mutation; e) Mutation strategy 5: first perform overflow mutation, then perform invalid digital mutation; f) Mutation strategy 6: perform sign mutation first, then overflow mutation; g) Mutation strategy 7: first perform symbol mutation, then perform boundary value mutation; h) Mutation strategy 8: perform symbol mutation first, then precision mutation; i) Mutation strategy 9: perform symbol mutation first, then type mutation; j) Mutation strategy 10: first perform symbol mutation, then perform invalid numeric mutation; k) Mutation strategy 11: perform boundary mutation first, then overflow mutation; l) Mutation strategy 12: first perform boundary value mutation, then perform sign mutation; m) Mutation strategy 13: first perform boundary value mutation, then perform precision mutation; n) Mutation strategy 14: first perform boundary value mutation, then type mutation; o) Mutation strategy 15: first perform boundary value mutation, then perform invalid number mutation; p) Mutation strategy 16: perform precision mutation first, then overflow mutation; q) Mutation strategy 17: perform precision mutation first, then sign mutation; r) Mutation strategy 18: first perform precision mutation, then perform boundary value mutation; s) Mutation strategy 19: perform precision mutation first, then type mutation; t) Mutation strategy 20: first perform precision mutation, then perform invalid number mutation; u) Mutation strategy 21: perform type mutation first, then overflow mutation; v) Mutation strategy 22: perform type mutation first, then symbol mutation; w) Mutation strategy 23: perform type mutation first, then perform boundary value mutation; x) Mutation strategy 24: perform type mutation first, then perform precision mutation; y) Mutation strategy 25: perform type mutation first, then perform invalid number mutation; z) Mutation strategy 26: first perform invalid digital mutation, then perform overflow mutation; aa) Mutation strategy 27: first perform null null mutation, then perform symbol mutation; bb) Mutation strategy 28: first perform invalid number mutation, then perform boundary value mutation; cc) Mutation strategy 29: first perform invalid digital mutation, then perform precision mutation; dd) Mutation strategy 30: first perform null null mutation, then perform type mutation; 3-2) From the seed pool T all Extract each seed one by one, and randomly select one of the 30 predefined mutation strategies for each extracted seed to perform mutation operation, thereby generating mutation seeds, and storing the generated mutation seeds in the mutation seed pool T mutation middle.

Images