ROS 2 defect detection method based on large language model and differential test

Abstract

The invention belongs to the field of software testing, and relates to an ROS 2 defect detection method based on a large language model and a differential test, which can be used for automatically detecting defects in ROS 2. According to the method, the accurate cue word is constructed by combining the semantic information of the to-be-tested program, so that the efficiency and the quality of generating the seed by the large language model are remarkably improved. Meanwhile, multiple mutation strategies are introduced to enhance the diversity of the seeds, so that the comprehensiveness of defect detection is improved. According to the method, differential testing on the same function module realized by using different programming languages is supported, the differential testing method can effectively reveal behavior inconsistency caused by language difference, the function consistency between different languages is ensured, and the stability and the reliability of the system are further improved. Through combination with a large language model, manual intervention is remarkably reduced, the defect detection cost is reduced, and the automation degree and precision of testing are greatly improved.

Description

Technical Field

[0001] The present invention belongs to the field of software testing, and relates to a ROS 2 defect detection method based on a large language model and differential testing, which can be used to automatically detect defects in ROS 2. Background Art

[0002] The Robot Operating System (ROS) is an open source framework for robot software development, which aims to simplify the development and implementation of robot applications. With the continuous development of ROS, ROS 2, as its second-generation version, optimizes the core architecture of ROS, aiming to meet the application requirements of modern robots in complex and diverse scenarios. However, the multi-module and multi-language features of ROS 2 bring significant risks of software defects, which may cause unforeseen failures and affect the safety, reliability and functional integrity of the robot system. At present, the research on the security and reliability of ROS 2 focuses on static analysis and fuzz testing methods.

[0003] However, static analysis and fuzz testing methods have some shortcomings. Static analysis can produce false positives and take a long time, exposing the limitations of traditional testing methods. Fuzz testing has certain advantages in capturing software vulnerabilities, but its detection effect depends on the diversity and coverage of input data. If the generated input data fails to touch the critical path or boundary conditions, potential problems may be missed. Especially in complex systems such as ROS 2 with a multi-level, distributed architecture, fuzz testing is difficult to generate test cases that cover enough critical scenarios. These limitations show that traditional static analysis and fuzz testing methods are difficult to fully meet the defect detection needs of ROS 2.

[0004] In this context, artificial intelligence big language models bring new possibilities for defect detection. The detection method based on big language models can use its understanding of natural language and context to automatically extract functional logic and context information from the source code, thereby generating test cases with high coverage and high quality. This method combines big language models with differential testing, and uses the seeds generated by big language models as test cases to perform differential testing on functional modules implemented in different languages ​​of ROS 2 to detect functional consistency issues. This method not only significantly improves the efficiency of ROS 2 defect detection, but also broadens the depth and scope of detection, providing a new technical path for capturing unknown defects. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a ROS 2 defect detection method based on a large language model and differential testing. First, the present invention uses the AST abstract syntax tree to obtain the public information D of the program to be tested. context and function information D function, in order to construct a prompt word template that can accurately express the needs. Then, the filled template is input into the pre-selected large language model to generate seeds, and the generated seeds are stored in the seed pool T all Secondly, from the seed pool T all Extract seeds one by one from the 30 predefined mutation strategies and randomly select one of them to perform mutation operation, thereby generating different mutation seeds, and store these mutation seeds in the mutation seed pool T mutation Finally, two API interfaces with the same functions implemented by C++ and Python in the program to be tested are pre-selected, namely API interface 1 and API interface 2 (since the ROS 2 system is mainly developed using C++ and Python, the present invention only tests the interfaces of these two mainstream programming languages ​​to analyze their differences in function implementation and performance), and then the mutation seed pool T is selected. mutation Extract mutation seeds one by one, input them into the program to be tested as test cases, execute them by pre-selected API interface 1 and API interface 2 respectively, and obtain output information Output1 of API interface 1 and output information Output2 of API interface 2. If output information Output1 and output information Output2 are inconsistent, use the pre-set threshold T to evaluate the difference between output information Output1 and output information Output2. If the difference between output information Output1 and output information Output2 is greater than threshold T, record detailed information to generate relevant reports, and retain the mutation seeds that generate inconsistent and unreasonable test cases to seed pool T all In order to subsequently combine the mutation strategy to generate new mutation seeds as test cases for differential testing, thereby further improving the diversity of test cases and the quality of defect detection.

[0006] The technical solution of the present invention:

[0007] A ROS 2 defect detection method based on a large language model and differential testing, the specific steps are as follows:

[0008] Step (1) Select the program to be tested and extract the code file F from it. Generate its corresponding AST abstract syntax tree and obtain public information D context And function information D function .

[0009] Step (2) constructs a prompt word template to be filled: "Given <context information>, the code of the function to be tested: <function information>, please generate a seed with high coverage for the function to be tested." Using the public information D obtained in step (1) context Fill in "<context information>", function information D functionFill in "<function information>", and finally obtain a series of natural language accurate prompts for inputting into the large language model, input into the pre-selected large language model, thereby generating seeds, and all the generated seeds are stored in the seed pool T all middle.

[0010] Step (3) The seed pool T generated from step (2) all Extract seeds one by one from the , and then randomly select one of the predefined mutation strategies for mutation operation. Each mutation strategy modifies different aspects of the seed to generate diverse mutation seeds, and the resulting mutation seeds are stored in the mutation seed pool T mutation middle.

[0011] Step (4) pre-selects two API interfaces with the same functions implemented by C++ and Python in the program to be tested, namely API interface 1 and API interface 2, and then selects the mutation seed pool T generated in step (3) mutation Extract mutation seeds as test cases, input these test cases into the program to be tested, execute them by pre-selected API interface 1 and API interface 2 respectively, and obtain API interface 1 output information Output1 and API interface 2 output information Output2.

[0012] Step (5) pre-sets a reasonable difference threshold T, and evaluates the output information Output1 and output information Output2 generated in step (4) to detect whether the output information Output1 and output information Output2 are consistent. If they are consistent, the mutation seed that generates the test case is discarded. If they are inconsistent, further detect whether the difference between the output information Output1 and output information Output2 is less than or equal to the threshold T. If the difference is less than or equal to the threshold T, the mutation seed that generates the test case is discarded; if the difference is greater than the threshold T, a report is generated for the inconsistent and unreasonable related information, and the mutation seed that generates the test case is retained in the seed pool T. all In this way, it is convenient to combine the mutation strategy to generate new mutation seeds as test cases for differential testing.

[0013] Furthermore, step (1) specifically includes the following steps:

[0014] 1-1) A code file F is extracted from the program to be tested.

[0015] 1-2) Generate the AST abstract syntax tree corresponding to the code file F, and extract the global variables and global constants in the code file F to form the public information D context .

[0016] 1-3) Traverse the function nodes in the AST abstract syntax tree, extract all function contents (including function name, return type, parameter list, etc.), and record the calling relationship within each function to form function information D function .

[0017] Furthermore, step (2) specifically includes the following steps:

[0018] 2-1) Construct a standardized prompt word template for input into a large language model. The format of the template is: "Given <context information>, the code of the function to be tested: <function information>, please generate a high coverage seed for the function to be tested."

[0019] 2-2) Using the extracted public information D context Fill in the "<context information>" in the prompt word template, and use the function information D function Fill in the "<function information>" in the prompt word template. After filling, the generated prompt word template is input as natural language into the pre-selected large language model. The large language model will generate seeds with high coverage based on the input prompt words, and finally, all the generated seeds are stored in the seed pool T all middle.

[0020] Furthermore, step (3) specifically includes the following steps:

[0021] 3-1) Define 6 mutation algorithms to combine into 30 mutation strategies. The 6 mutation algorithms are as follows:

[0022] a) Overflow mutation: By changing the value in the seed, the value exceeds its valid range.

[0023] b) Sign mutation: Change the sign of a value (positive to negative or negative to positive).

[0024] c) Boundary value mutation: Modify the value in the seed to the boundary value of the data type, such as the maximum value, minimum value, or a value close to the boundary value.

[0025] d) Precision mutation: By modifying the precision of floating-point numbers, the program's response to inputs of different numerical precision is tested.

[0026] e) Type mutation: changing the data type, for example, converting an integer to a floating point type, or converting a floating point type to an integer type.

[0027] f) Invalid number mutation: Replace the numbers in the seed with invalid numbers, such as NaN (Not-a-Number), infinity, invalid negative numbers (such as -0), or values ​​outside the valid range.

[0028] The 30 mutation strategies are combined as follows:

[0029] a) Mutation strategy 1: perform overflow mutation first, then sign mutation.

[0030] b) Mutation strategy 2: first perform overflow mutation, then perform boundary value mutation.

[0031] c) Mutation strategy 3: perform overflow mutation first, then perform precision mutation.

[0032] d) Mutation strategy 4: perform overflow mutation first, then type mutation.

[0033] e) Mutation strategy 5: perform overflow mutation first, then perform invalid digital mutation.

[0034] f) Mutation strategy 6: perform sign mutation first, then overflow mutation.

[0035] g) Mutation strategy 7: perform symbol mutation first, then perform boundary value mutation.

[0036] h) Mutation strategy 8: perform sign mutation first, then precision mutation.

[0037] i) Mutation strategy 9: perform symbol mutation first, then type mutation.

[0038] j) Mutation strategy 10: Perform symbol mutation first, then perform invalid numeric mutation.

[0039] k) Mutation strategy 11: perform boundary value mutation first, then overflow mutation.

[0040] l) Mutation strategy 12: first perform boundary value mutation, then perform sign mutation.

[0041] m) Mutation strategy 13: first perform boundary value mutation, then perform accuracy mutation.

[0042] n) Mutation strategy 14: first perform boundary value mutation, then perform type mutation.

[0043] o) Mutation strategy 15: first perform boundary value mutation, then perform invalid number mutation.

[0044] p) Mutation strategy 16: Perform precision mutation first, then overflow mutation.

[0045] q) Mutation strategy 17: perform precision mutation first, then sign mutation.

[0046] r) Mutation strategy 18: First perform precision mutation, then perform boundary value mutation.

[0047] s) Mutation strategy 19: perform precision mutation first, then type mutation.

[0048] t) Mutation strategy 20: Perform precision mutation first, then perform invalid number mutation.

[0049] u) Mutation strategy 21: Perform type mutation first, then overflow mutation.

[0050] v) Mutation strategy 22: Perform type mutation first, then symbol mutation.

[0051] w) Mutation strategy 23: Perform type mutation first, then perform boundary value mutation.

[0052] x) Mutation strategy 24: Perform type mutation first, then precision mutation.

[0053] y) Mutation strategy 25: Perform type mutation first, then perform invalid number mutation.

[0054] z) Mutation strategy 26: first perform invalid digital mutation, then perform overflow mutation.

[0055] aa) Mutation strategy 27: first perform invalid numeric mutation, then perform symbol mutation.

[0056] bb) Mutation strategy 28: first perform invalid number mutation, then perform boundary value mutation.

[0057] cc) Mutation strategy 29: First perform invalid number mutation, then perform precision mutation.

[0058] dd) Mutation strategy 30: first perform invalid number mutation, then perform type mutation.

[0059] 3-2) From the seed pool T all Each seed is extracted one by one, and for each extracted seed, one of the predefined 30 mutation strategies is randomly selected to perform mutation operation, thereby generating mutation seeds, and the generated mutation seeds are stored in the mutation seed pool T mutation middle.

[0060] Furthermore, step (4) specifically includes the following steps:

[0061] 4-1) Preselect two API interfaces with the same functions implemented by C++ and Python in the program to be tested, namely API interface 1 and API interface 2.

[0062] 4-2) Then from the mutation seed pool T mutation Extract each mutation seed one by one as a test case.

[0063] 4-3) Use the same test case to input into the program to be tested, and execute it by the pre-selected API interface 1 and API interface 2 respectively, and obtain the output information Output1 of API interface 1 and the output information Output2 of API interface 2.

[0064] Furthermore, step (5) specifically includes the following steps:

[0065] 5-1) Check whether the output information Output1 and the output information Output2 are consistent. If they are consistent, the mutation seed that generates the test case will be discarded, otherwise the next step of difference evaluation will be carried out.

[0066] 5-2) A reasonable difference threshold T is set according to the differences in data accuracy processing among different programs to reduce the impact of accuracy differences on experimental results.

[0067] 5-3) Evaluate the difference between the inconsistent output information Output1 and the output information Output2. If the difference is less than or equal to the preset threshold T, the mutation seed that generates the test case does not need to be retained. Although the test case triggers an inconsistent result, it is within a reasonable range. If the difference is greater than the set threshold T, it is considered to be an abnormal situation where the inconsistency and the difference are not reasonable. Record and report the relevant information, and retain the mutation seed that generates the test case in the seed pool T. all In this way, it is convenient to combine mutation strategies to generate new mutation seeds as test cases for differential testing, which helps to identify potential abnormal situations and program defects, thereby improving the quality and coverage of test cases.

[0068] Compared with the prior art, the present invention has the following advantages and effects:

[0069] The present invention proposes a ROS 2 defect detection method based on a large language model and differential testing, which can efficiently and automatically detect potential defects in the ROS 2 system. The method constructs accurate prompt words by combining the semantic information of the program to be tested, thereby significantly improving the efficiency and quality of the large language model to generate seeds. At the same time, multiple mutation strategies are introduced to enhance the diversity of seeds, thereby improving the comprehensiveness of defect detection. In view of the fact that ROS 2 contains multiple modules with the same functions but different implementation languages, the present invention supports differential testing of the same functional modules implemented in different programming languages. This differential testing method can effectively reveal the inconsistent behavior caused by the difference in implementation language, ensure the functional consistency between different language implementations, and thus improve the stability and reliability of the system. By combining with the large language model, the present invention significantly reduces manual intervention, reduces the cost of defect detection, and greatly improves the automation and accuracy of the test. In addition, the method of the present invention has high scalability and can flexibly switch and integrate different large language models to meet different testing requirements. This flexibility not only optimizes the user experience, but also lowers the threshold for technical application, enabling more developers to easily apply the technology, and further promotes the stability and security of the ROS 2 system. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a flow chart of the ROS 2 defect detection method based on a large language model and differential testing of the present invention.

[0071] Figure 2 It is a subgraph of the context information extraction process in the ROS 2 defect detection method based on a large language model and differential testing of the present invention.

[0072] Figure 3 It is a process subgraph of seed generation in the ROS 2 defect detection method based on a large language model and differential testing of the present invention.

[0073] Figure 4 It is a flow subgraph of seed mutation in the ROS 2 defect detection method based on a large language model and differential testing of the present invention.

[0074] Figure 5 It is a flow subgraph of differential testing in the ROS 2 defect detection method based on a large language model and differential testing of the present invention. DETAILED DESCRIPTION

[0075] The method of the present invention is described in detail below in conjunction with the accompanying drawings, technical solutions and embodiments.

[0076] like Figure 1As shown in FIG. 1 , the ROS 2 defect detection method based on a large language model and differential testing of the present invention is performed according to the following process: first, a code file F is obtained from the program to be tested, context analysis is performed on the code file F, a corresponding AST abstract syntax tree is generated using a specific processing algorithm, and public information D is automatically identified and extracted from the AST abstract syntax tree. context and function information D function Then, these key information are integrated into the pre-set prompt word template to be filled, and the template content is transmitted to the pre-selected large language model. The large language model generates seeds based on these precise prompts and stores the generated seeds in the seed pool T all Secondly, from the seed pool T all Extract seeds one by one, randomly select one of the pre-set mutation strategies to perform mutation operations on them, generate diverse mutation seeds, and store the mutated seeds in the mutation seed pool T mutation Then, from the mutant seed pool T mutation Extract mutation seeds one by one from the test case and input them into the program to be tested as test cases for execution. Finally, evaluate the output information after execution. If the output information is consistent, it will be discarded as the mutation seed of the test case. If the output information is inconsistent, proceed to the next step of difference evaluation. In the difference evaluation stage, if the difference in the output information is greater than the pre-set reasonable threshold T, the relevant information will be recorded and reported, and the mutation seed of the test case will be retained in the seed pool T all In this way, it is convenient to combine the mutation strategy to generate new mutation seeds as test cases for differential testing. If the difference in output information is less than or equal to the threshold T, it is not considered an abnormality and the mutation seed used as the test case is discarded.

[0077] The following takes the coordinate conversion program TF2 in ROS 2 as an example to explain the implementation details of each process in detail. The specific implementation is as follows:

[0078] (1) Figure 2As shown, context information extraction is performed. First, extract the code file F from the program to be tested, and generate an AST abstract syntax tree for the code file F. According to the language type of the code file F to be tested, select the corresponding AST abstract syntax tree generation tool. Taking C++ and Python as examples, for C++ code files, use tools such as Clang to generate its corresponding AST abstract syntax tree. For Python code files, use Python's built-in ast module to generate its corresponding AST abstract syntax tree. Use a traversal algorithm to traverse all nodes in the abstract syntax tree to find global variables and constants. Global variables are usually declared outside a class or function and are not in any local scope. Identify global variables marked with keywords such as static or extern, especially in C++ code. Identify global constants, usually using keywords such as const or constexpr. By identifying the nodes of the abstract syntax tree, extract the global variables and global constants in the code and other related public information, and integrate this information into public information D context On this basis, all function definitions and declarations are searched in the AST abstract syntax tree to obtain the complete definition content of each function. At the same time, each function call relationship is retrieved, and the complete content of each function and its call relationship information are stored in the function information D. function middle.

[0079] 1.1. Generate AST abstract syntax tree of code file:

[0080] a) Use the Clang tool to build the AST abstract syntax tree for the code file static_transform_broadcaster.h.

[0081] b) Use the AST abstract syntax tree to identify all top-level functions and build a function list, such as the StaticTransformBroadcaster function in the code file, which is specifically used to broadcast static coordinate transformation information.

[0082] c) By identifying the AST abstract syntax tree nodes generated by the code file, extract the global constants and global variables used in the file. The global constants are " / tf_static" and "StaticBroadcasterQoS()", and the global variables are "publisher_" and "net_message_".

[0083] d) Integrate the global variables and global constants into public information D context .

[0084] e) By identifying the AST abstract syntax tree nodes generated by the code file and traversing the generated AST abstract syntax tree nodes in the function list, the definition of the relevant functions in the file is extracted, such as the definition of the tested function StaticTransformBroadcaster is extracted as follows:

[0085]

[0086]

[0087] 1.2. Extract function content and function call relationship:

[0088] a) By identifying the definition node of each function through the AST abstract syntax tree, the internal implementation part of the function can be accessed through the child nodes of the AST abstract syntax tree.

[0089] b) Function calls are usually represented by CALL_EXPR nodes, which are achieved by traversing all CALL_EXPR nodes in the AST abstract syntax tree and recording the calling details of each function, especially which function calls which other functions.

[0090] 1.3. Build context information:

[0091] a) Store the complete content and calling relationship of the function into function information D function middle..

[0092] b) Public information D context and function information D function The integration builds the contextual information required for seed generation. This enables efficient retrieval of relevant function information in the subsequent seed generation phase, ensuring that the code behavior can be fully analyzed and fully covered.

[0093] (2) Figure 2 As shown in Figure 1, first, a prompt word template is designed, whose structure is: "Given <context information>, the code of the function to be tested: <function information>, please generate a seed with high coverage for the function to be tested." When constructing the prompt word template, the extracted context information is accurately processed to ensure that the public information D is included. context and function information D functionAll details including the code are accurate, so as to fully reflect the background and execution conditions of the code. Once the prompt word template is built, the system will automatically input it into the selected large language model for processing to generate the corresponding seeds. Taking the function StaticTransformBroadcaster as an example, DeepSeek Coder V2 is selected as the large language model for seed generation. The model automatically generates seeds related to the tested function based on the constructed prompt word template. These seeds contain test cases with high coverage. The generated seeds are stored in the seed pool T all in order to prepare for subsequent mutation operations.

[0094] After building this prompt word template and feeding it into the big language model, the resulting seed looks like this:

[0095]

[0096]

[0097] (3) Figure 3 As shown in Figure 1, seed mutation. First, 30 mutation strategies are predefined, covering different mutation operations, to ensure that seeds can be effectively mutated from multiple angles. Then, from the seed pool T all Extract seeds one by one from the 30 predefined mutation strategies and randomly select one of them for mutation operation. The mutation seeds generated by the mutation operation are stored in the mutation seed pool T mutation For subsequent differential testing.

[0098] 3.1. Define 6 mutation algorithms and combine them into 30 mutation strategies. The 6 mutation algorithms are as follows:

[0099] a) Overflow mutation:

[0100]

[0101] b) Symbol mutation:

[0102] def sign_mutation(seed:Union[int,float])->Union[int,float]:

[0103] """Sign mutation: Change the sign of a value (positive to negative or negative to positive)"""

[0104] return-seed

[0105] c) Boundary value mutation:

[0106]

[0107]

[0108] d) Accuracy mutation:

[0109] e) Type of mutation:

[0110] f) Invalid numeric mutation:

[0111] Among the 30 mutation strategies, mutation strategy 6 is taken as an example, and the details are as follows:

[0112]

[0113] 3.2. From the seed pool T all Extract each seed one by one, randomly select one of the 30 pre-defined mutation strategies to perform mutation operation, so as to generate mutation seeds, and store all the generated mutation seeds in the mutation seed pool T mutation Taking the seed generated in (2) as an example, we select mutation strategy 6 to perform mutation operation, first perform sign mutation, then perform overflow mutation. The details are as follows:

[0114] seed:

[0115]

[0116]

[0117] Mutation seeds:

[0118]

[0119]

[0120] (4) Figure 4 As shown, a differential test is performed. By selecting mutant seed pool T mutationExtract mutation seeds one by one as test cases, combine the selected API interfaces with the same functions implemented in C++ and Python in TF 2, perform differential testing, input the test cases into the selected API interfaces implemented in C++ and Python, obtain the output information after the execution of the two API interfaces, and evaluate the output information of the two. If the output information of the two is consistent, the mutation seed that generates the test case is discarded. If the output information is inconsistent, proceed to the next step of difference evaluation. In the difference evaluation stage, a reasonable difference threshold T is set in advance. If the difference value between the two is less than or equal to the threshold T, the mutation seed that generates the test case is discarded. If the difference value between the two is greater than the threshold T, the detailed relevant information will be reported, and the mutation seed that generates the test case will be retained in the seed pool T all In this way, it is convenient to combine the mutation strategy to generate new mutation seeds as test cases for differential testing.

[0121] 4.1. One by one from the mutation seed pool T mutation Extract mutation seeds as test cases for differential testing, input them into the API interfaces of the same functions implemented in C++ and Python in TF2, get the output results and evaluate them. Take the API interface static_transform_broadcaster.h implemented in C++ and the API interface static_transform_broadcaster.py implemented in Python as examples, both of which are API interfaces for broadcast conversion in TF2.

[0122] Specifically:

[0123] Using the same test case, the test case looks like this:

[0124]

[0125]

[0126] Call the C++ programming language to implement the API interface static_transform_broadcaster.h to publish broadcast conversion. The corresponding code implementation content is as follows:

[0127]

[0128]

[0129] Call the Python programming language to implement the API interface static_transform_broadcaster.py to publish broadcast conversion. The corresponding code implementation content is as follows:

[0130]

[0131] 4.2. Receive the output information Output1 of the API interface implemented by calling C++ and the output information Output2 of the API interface implemented by calling Python, as shown below:

[0132]

[0133] 4.3. Get the output information Output1 and output information Output2, and evaluate the output information Output1 and output information Output2. First, determine whether the output information Output1 and output information Output2 are consistent, and use "==" to make a judgment. If they are consistent, discard the mutation seed used as the test case. If they are inconsistent, proceed to the next step of evaluation, as shown below:

[0134]

[0135]

[0136] 4.4. If the output information Output1 and the output information Output2 are inconsistent, a difference evaluation is performed. According to the PEP 485 proposal introduced after Python 3.5, it is recommended to use an approximate equality method when comparing two floating-point numbers, rather than relying on precise equality judgment. This method allows two numerical values ​​and a tolerance error value (usually set to 10^-5) to be passed in. When the numerical difference is less than the error value, it can be considered equal. Therefore, the difference threshold T is set to 10^-5 to reduce the impact of precision differences on experimental results. If the difference is less than or equal to the threshold T, the mutation seed used as the test case will be discarded. Although it triggers the inconsistency between the two, the difference is reasonable. If the difference is greater than the threshold T, the information is reported, and the mutation seed used as the test case is retained and stored in the seed pool for subsequent combination with the mutation strategy to generate a new mutation seed as a test case for subsequent differential testing, as shown below:

[0137]

[0138] 4.5. Finally, the report will record many test cases and difference information that trigger inconsistency between the two outputs and the difference exceeds the preset reasonable threshold T. The abnormal situation will be reproduced again and submitted to the developer after being confirmed as a defect.

Claims

1. A ROS2 defect detection method based on a large language model and differential testing, characterized in that: The specific steps are as follows: Step (1) Select the program to be tested, extract the code file F from it, generate its corresponding AST abstract syntax tree, and obtain public information D context And function information D function ; Step (2) constructs a prompt word template to be filled: "Given <context information>, the code of the function to be tested: <function information>, please generate a seed with high coverage for the function to be tested."; use the public information D obtained in step (1) context Fill in "<context information>", function information D function Fill in "<function information>", and finally obtain a series of natural language accurate prompts for inputting into the large language model, input into the pre-selected large language model, thereby generating seeds, and all the generated seeds are stored in the seed pool T all middle; Step (3) The seed pool T generated from step (2) all Extract seeds one by one from the predefined mutation strategies, and then randomly select one from the predefined mutation strategies for mutation operation; each mutation strategy modifies different aspects of the seeds to generate diverse mutation seeds, and the resulting mutation seeds are stored in the mutation seed pool T mutation middle; Step (4) pre-selects two API interfaces with the same functions implemented by C++ and Python in the program to be tested, namely API interface 1 and API interface 2, and then selects the mutation seed pool T generated in step (3) mutation Extract mutation seeds as test cases, input these test cases into the program to be tested, execute them by the pre-selected API interface 1 and API interface 2 respectively, and obtain the output information Output1 of API interface 1 and the output information Output2 of API interface 2; Step (5) pre-sets a reasonable difference threshold T, and evaluates the output information Output1 and output information Output2 generated in step (4) to detect whether the output information Output1 and output information Output2 are consistent. If they are consistent, the mutation seed that generates the test case is discarded. If they are inconsistent, further detect whether the difference between the output information Output1 and output information Output2 is less than or equal to the threshold T; if the difference is less than or equal to the threshold T, the mutation seed that generates the test case is discarded; if the difference is greater than the threshold T, a report is generated for the relevant information that is inconsistent and unreasonable, and the mutation seed that generates the test case is retained in the seed pool T. all In this way, it is convenient to combine the mutation strategy to generate new mutation seeds as test cases for differential testing.

2. The ROS2 defect detection method based on a large language model and differential testing according to claim 1, characterized in that: Step (3) specifically includes the following steps: 3-1) Define 6 mutation algorithms to combine into 30 mutation strategies. The 6 mutation algorithms are as follows: a) Overflow mutation: By changing the value in the seed, the value exceeds its valid range; b) Sign mutation: change the sign of a value, from positive to negative or from negative to positive; c) Boundary value mutation: modify the value in the seed to the boundary value of the data type; d) Precision mutation: By modifying the precision of floating-point numbers, the program's response to inputs of different numerical precision is tested; e) Type mutation: changing the type of data; f) Invalid number mutation: replace the numbers in the seed with invalid numbers; The 30 mutation strategies are combined as follows: a) Mutation strategy 1: first perform overflow mutation, then perform sign mutation; b) Mutation strategy 2: first perform overflow mutation, then perform boundary value mutation; c) Mutation strategy 3: first perform overflow mutation, then perform precision mutation; d) Mutation strategy 4: first perform overflow mutation, then perform type mutation; e) Mutation strategy 5: first perform overflow mutation, then perform invalid digital mutation; f) Mutation strategy 6: first perform sign mutation, then perform overflow mutation; g) Mutation strategy 7: first perform symbol mutation, then perform boundary value mutation; h) Mutation strategy 8: first perform symbol mutation, then perform precision mutation; i) Mutation strategy 9: first perform symbol mutation, then perform type mutation; j) Mutation strategy 10: first perform symbol mutation, then perform invalid numeric mutation; k) Mutation strategy 11: first perform boundary value mutation, then perform overflow mutation; l) Mutation strategy 12: first perform boundary value mutation, then perform symbol mutation; m) Mutation strategy 13: first perform boundary value mutation, then perform precision mutation; n) Mutation strategy 14: first perform boundary value mutation, then type mutation; o) Mutation strategy 15: first perform boundary value mutation, then perform invalid number mutation; p) Mutation strategy 16: first perform precision mutation, then perform overflow mutation; q) Mutation strategy 17: first perform precision mutation, then perform sign mutation; r) Mutation strategy 18: first perform precision mutation, then perform boundary value mutation; s) Mutation strategy 19: first perform precision mutation, then perform type mutation; t) Mutation strategy 20: first perform precision mutation, then perform invalid digital mutation; u) Mutation strategy 21: perform type mutation first, then perform overflow mutation; v) Mutation strategy 22: perform type mutation first, then symbol mutation; w) Mutation strategy 23: first perform type mutation, then perform boundary value mutation; x) Mutation strategy 24: first perform type mutation, then perform precision mutation; y) Mutation strategy 25: first perform type mutation, then perform invalid number mutation; z) Mutation strategy 26: first perform invalid digital mutation, then perform overflow mutation; aa) Mutation strategy 27: first perform invalid numeric mutation, then perform symbol mutation; bb) Mutation strategy 28: first perform invalid digital mutation, then perform boundary value mutation; cc) Mutation strategy 29: first perform invalid number mutation, then perform precision mutation; dd) Mutation strategy 30: first perform invalid number mutation, then perform type mutation; 3-2) From the seed pool T all Each seed is extracted one by one, and for each extracted seed, one of the 30 pre-defined mutation strategies is randomly selected to perform mutation operation, thereby generating mutation seeds, and the generated mutation seeds are stored in the mutation seed pool T mutation middle.

Images