A ROS software package detection method and device based on fuzz testing

Abstract

The specification discloses a ROS software package detection method and device based on fuzz testing, including an initial seed generation step, a test case acquisition step and a test evaluation step. The initial seed generation step generates an initial seed according to the input type of the program to be tested and based on a seed generation strategy, performs sequence labeling and sorting on the initial seed, and uses the sorted seed queue as a seed pool. The test case acquisition step selects a seed in the seed pool according to a seed selection mechanism to obtain a selected seed, selects a mutation strategy according to a mutation strategy selection mechanism to obtain a selected mutation strategy, and mutates the selected seed based on the selected mutation strategy to obtain a test case. The test evaluation step injects the test case into the program to be tested for execution, collects coverage information and performs memory detection, evaluates the selected seed and the selected mutation strategy according to the coverage information and error information, and respectively feeds back to update the seed queue and the mutation priority attribute of the seed pool.

Description

Technical Field

[0001] This invention relates to the field of software package detection technology, and more specifically, to a method and apparatus for ROS software package detection based on fuzz testing. Background Technology

[0002] With the development of IT technology and intelligent industry, intelligent robots are being widely used in various fields. The Robot Operating System (ROS) is a widely used robot development platform in the field of intelligent robots and a crucial component of various intelligent robot control and task software systems. Although functionally similar to an operating system, ROS cannot run directly on the physical hardware of a computer like operating systems such as Windows. It relies on the existing operating system to provide robot development with many functions, including device drivers, visualization, numerous library functions, communication mechanisms, and package management. To meet the diverse needs of developers, ROS integrates a large number of software packages with varying functions, covering areas such as motion planning, robotic arm control, coordinate transformation, and image processing, helping developers solve many problems in the robot development process. However, if vulnerabilities exist in ROS software packages, the security of the entire intelligent robot will be threatened. Therefore, effective detection of vulnerabilities in ROS software packages is crucial. Summary of the Invention

[0003] This specification provides a method and apparatus for detecting ROS software packages based on fuzz testing, in order to overcome at least one technical problem existing in the prior art.

[0004] Firstly, according to embodiments of this specification, a ROS software package detection method based on fuzz testing is provided, comprising:

[0005] Initial seed generation steps: Generate initial seeds based on the input type of the program under test and the seed generation strategy; label and sort the initial seeds according to their sequence; and use the sorted seed queue as the seed pool.

[0006] Test case acquisition steps: Select a seed from the seed pool according to the seed selection mechanism to obtain the selected seed; select a mutation strategy according to the mutation strategy selection mechanism to obtain the selected mutation strategy; mutate the selected seed based on the selected mutation strategy to obtain the test case.

[0007] Test evaluation steps: Inject the test cases into the program under test for execution, collect coverage information and perform memory detection to obtain the coverage information and error information, evaluate the selected seed and the selected mutation strategy based on the coverage information and the error information and provide feedback respectively, so as to update the seed queue of the seed pool and the mutation priority attribute of the selected mutation strategy.

[0008] Optionally, if the program under test has at least one set of startup parameters,

[0009] Before the test case acquisition step, the ROS package detection method further includes: preprocessing the program under test, calculating the code space of each startup parameter in each group of startup parameters, and calculating the potential score of each startup parameter based on the code space of each startup parameter;

[0010] Prior to the test evaluation step, the ROS software package detection method further includes: selecting each set of startup parameters for the program under test according to the startup parameter selection mechanism to obtain the selected startup parameters;

[0011] The test evaluation steps specifically include: executing the program under test with the selected startup parameters and the test cases, collecting coverage information and performing memory detection to obtain the coverage information and error information, evaluating the selected seed, the selected mutation strategy, and the selected startup parameters based on the coverage information and the error information, and feeding back the evaluation results to update the seed queue of the seed pool, the mutation priority attribute of the selected mutation strategy, and the parameter priority attribute of the selected startup parameters.

[0012] Further optionally, the preprocessing of the program under test, calculating the code space of each startup parameter in each group of startup parameters, and calculating the potential score of each startup parameter based on the code space of each startup parameter, specifically includes:

[0013] Select a set of startup parameters from the program under test and perform a preprocessing step, the preprocessing step including:

[0014] The program under test is executed using the first startup parameter in the set of startup parameters and the pre-test case, the first coverage information is recorded, and a coverage information file is generated based on the first coverage information.

[0015] The program under test is executed with all startup parameters except the first startup parameter in the set of startup parameters and the pre-test cases respectively. The number of newly added lines of code and the number of branches corresponding to each startup parameter are recorded. The number of newly added lines of code and the number of branches are used as the code space information of the corresponding startup parameter. At the same time, the coverage information file is updated.

[0016] Roll back the updated overwrite information file to the state when only the first startup parameter was executed;

[0017] Clear all coverage information in the coverage information file, execute the program under test with the second startup parameter in the set of startup parameters and the pre-test case to obtain the second code space corresponding to the second startup parameter, then execute the program under test with the first startup parameter in the set of startup parameters and the pre-test case to obtain the first code space corresponding to the first startup parameter, and calculate the code space information of the first startup parameter based on the difference between the second code space and the first code space;

[0018] Based on the code space information of the first startup parameter and the code space information of all startup parameters in the set of startup parameters except the first startup parameter, the code space of each startup parameter in the set of startup parameters and the total code space of the set of startup parameters are calculated. The potential score of each startup parameter in the set of startup parameters is the ratio of the code space of each startup parameter to the total code space of the set of startup parameters.

[0019] Select the remaining groups of startup parameters that have not been selected in the program under test, and perform the preprocessing steps until the potential score of each startup parameter in all groups of startup parameters of the program under test is calculated.

[0020] Optionally, each startup parameter in all groups of startup parameters of the program under test has a parameter priority attribute and a usage count attribute. The parameter priority attribute includes a parameter priority score, which depends on the potential score and time score of the startup parameter. The usage count attribute includes a usage count value. The startup parameter selection mechanism specifically includes:

[0021] Each time a set of startup parameters for the program under test is selected, the startup parameter with the highest parameter priority score in each set of startup parameters is selected as the selected startup parameter. The usage count of the selected startup parameter is incremented by 1, and the parameter priority score of the selected startup parameter is reset to 0.

[0022] Calculate and update the parameter priority score of the remaining unselected startup parameters in each group of startup parameters. The formula for updating the parameter priority score of the remaining unselected startup parameters in each group of startup parameters is as follows:

[0023] S priority =S' priority +S time *(S potential +1)

[0024] Among them, S priorityS' represents the priority score of the updated startup parameters after the initial selection was not selected. Priority S represents the priority score of the unselected startup parameters before the update. time S represents the time score of the startup parameter that was not selected. potential This indicates the potential score for the startup parameter that was not selected.

[0025] Further optionally, the seed generation strategy specifically includes:

[0026] When the input type of the program under test is int, the Integers() function strategy is executed;

[0027] When the input type of the program under test is float, the Float() function strategy is executed;

[0028] When the input type of the program under test is array, the Lists() function strategy is executed;

[0029] When the input type of the program under test is complex, the Composite() function strategy is executed.

[0030] In the Integers(), Float(), Lists(), and Composite() function strategies, the user sets the value range for each type of data according to the data input requirements of the program under test.

[0031] Further optionally, all seeds in the seed pool have a seed priority attribute, which includes a seed priority score, the seed priority score being the sum of the seed performance score and the seed time score. The seed selection mechanism specifically includes:

[0032] All seeds in the seed pool are sorted in descending order according to their seed priority scores.

[0033] Each time a seed is selected from the seed pool, the seed at the head of the queue is selected as the chosen seed, and the chosen seed is removed from the queue.

[0034] The seed time score of all unselected seeds in the seed pool is increased by a first preset threshold, and the seed priority score of the unselected seeds is updated. The unselected seeds in the seed pool are then reordered in descending order according to the updated seed priority scores.

[0035] Further optionally, the mutation strategy includes a bitflip mutation algorithm, a bitexchange mutation algorithm, an arithmetic mutation algorithm, and an interesting mutation algorithm. Each mutation algorithm has a mutation priority attribute, which includes a mutation priority score, which is the sum of the mutation performance score and the mutation time score. The mutation strategy selection mechanism specifically includes:

[0036] Each time a mutation algorithm of the mutation strategy is selected, the mutation algorithm with the highest mutation priority score is selected as the selected mutation strategy, and the mutation priority score of the selected mutation strategy is reset to 0.

[0037] The seed time scores of the remaining unselected mutation algorithms are increased by a second preset threshold, and the mutation priority scores of the remaining unselected mutation algorithms are updated.

[0038] Optionally, the coverage information includes newline coverage information and new branch coverage information, and the error information includes program error information and memory detection error information. The step of evaluating the selected seed and the selected mutation strategy based on the coverage information and the error information, and feeding back the evaluations respectively, to update the seed queue of the seed pool and the mutation priority attribute of the selected mutation strategy, specifically includes:

[0039] The number of lines and branches of the new triggering code are calculated based on the coverage information, using the following formula:

[0040] D Line =N LineCov -O LineCov

[0041] D Branch =N BranchCov -O BranchCov

[0042] Among them, D Line N represents the line number of the newly triggered code. LineCov Indicates the line of code that will be overwritten after the current execution of the program under test, O LineCov D represents the lines of code that were overwritten before the current execution of the program under test. Branch N represents the number of branches that trigger the new code. BranchCov Indicates the code branch that will be overwritten after the current execution of the program under test, O BranchCov This indicates the code branch that was covered before the current execution of the program under test;

[0043] If both the program error message and the memory detection error message are empty, then the error score S ErrThe error score is 0; if at least one of the program error message and the memory detection error message is not empty, then the error score S is 0. Err =1;

[0044] The evaluation score of the test case is calculated based on the number of lines and branches of the newly triggered code and the error score. The evaluation score S of the test case is... testcase The calculation formula is:

[0045] S testcase =D Line ×1+D Branch ×2+S Err

[0046] Determine the evaluation score S of the test case. testcase Is it 0?

[0047] If not, the test case is inserted into the seed pool as a new seed, and the evaluation score S of the test case is set. testcase The seed priority score is used as the seed priority score for the new seed, and the evaluation score S of the test case is also used. testcase The mutation priority attribute of the selected mutation strategy is updated as the mutation priority score of the selected mutation strategy.

[0048] Further optionally, the coverage information includes newline coverage information and new branch coverage information, and the error information includes program error information and memory detection error information. The step of evaluating and feeding back the selected seed, selected mutation strategy, and selected startup parameters based on the coverage information and the error information to update the seed queue of the seed pool, the mutation priority attribute of the selected mutation strategy, and the parameter priority attribute of the selected startup parameters specifically includes:

[0049] The number of lines and branches of the new triggering code are calculated based on the coverage information, using the following formula:

[0050] D Line =N LineCov -O LineCov

[0051] D Branch =N BranchCov -O BranchCov

[0052] Among them, D Line N represents the line number of the newly triggered code. LineCov Indicates the line of code that will be overwritten after the current execution of the program under test, O LineCov D represents the lines of code that were overwritten before the current execution of the program under test. Branch N represents the number of branches that trigger the new code. BranchCovIndicates the code branch that will be overwritten after the current execution of the program under test, O BranchCov This indicates the code branch that was covered before the current execution of the program under test;

[0053] If both the program error message and the memory detection error message are empty, then the error score S Err The error score is 0; if at least one of the program error message and the memory detection error message is not empty, then the error score S is 0. Err =1;

[0054] Extract the usage count attribute of the selected startup parameters in each group of startup parameters of the program under test, and determine whether there is a startup parameter with a usage count value of 1 among the selected startup parameters;

[0055] If it exists, then the evaluation score S of the test case is... testcase for:

[0056] S testcase =S Err

[0057] If it does not exist, then the evaluation score S of the test case is... testcase for:

[0058] S testcase =D Line ×1+D Branch ×2+S Err

[0059] Determine the evaluation score S of the test case. testcase Is it 0?

[0060] If not, the test case is inserted into the seed pool as a new seed, and the evaluation score S of the test case is set. testcase The seed priority score is used as the seed priority score for the new seed, and the evaluation score S of the test case is also used. testcase The mutation priority attribute of the selected mutation strategy and the parameter priority attribute of the selected initiation parameter are updated as the mutation priority score of the selected mutation strategy and the parameter priority score of the selected initiation parameter.

[0061] Secondly, according to the embodiments of this specification, a ROS software package detection device based on fuzz testing is also provided, comprising:

[0062] The initial seed generation module is used to generate an initial seed based on the input type of the program under test and a seed generation strategy, and to perform sequence marking and sorting on the initial seeds. The sorted seed queue serves as the seed pool.

[0063] The test case acquisition module is used to select seeds in the seed pool according to the seed selection mechanism to obtain the selected seeds, and to select a mutation strategy according to the mutation strategy selection mechanism to obtain the selected mutation strategy. Based on the selected mutation strategy, the selected seeds are mutated to obtain test cases.

[0064] The test evaluation module is used to inject the test cases into the program under test for execution, collect coverage information and perform memory detection to obtain the coverage information and error information, evaluate the selected seed and the selected mutation strategy based on the coverage information and the error information and provide feedback respectively, so as to update the seed queue of the seed pool and the mutation priority attribute of the selected mutation strategy.

[0065] Thirdly, according to embodiments of this specification, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in the first aspect.

[0066] Fourthly, according to embodiments of this specification, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0067] Fifthly, according to embodiments of this specification, a computer program product is also provided, the computer program product including instructions that, when executed on a computer or processor, cause the computer or processor to perform the steps of the method described in the first aspect.

[0068] The beneficial effects of the embodiments in this specification are as follows:

[0069] By introducing two evaluation metrics for seeds—whether coverage can be improved and whether they can trigger program or memory errors—the quality of generated seeds is enhanced. Furthermore, this ROS package detection method integrates four seed generation strategies and four seed mutation strategies. During mutation, test cases are generated based on coverage, program errors, and memory errors, continuously injecting test cases into the program under test to expose its errors. In addition, for some programs under test with selective startup parameters, this method establishes a startup parameter selection mechanism, improving the testing efficiency for such programs. This method can consistently improve coverage and, compared to traditional fuzzing tools, is more effective at triggering vulnerabilities. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0071] Figure 1 This is a flowchart illustrating the ROS software package detection method based on fuzz testing provided in Embodiment 1 of this specification.

[0072] Figure 2 This is a flowchart illustrating the ROS software package detection method based on fuzz testing provided in Embodiment 2 of this specification.

[0073] Figure 3 A schematic diagram of the experimental process for applying the fuzz-based ROS software package detection method provided in the embodiments of this specification to the ROS default motion planning package OMPL;

[0074] Figure 4 This is a schematic diagram illustrating the coverage growth of demo_RigidBodyPlanningWithODESolverAndControls provided in the embodiments of this specification;

[0075] Figure 5 A schematic diagram illustrating the coverage growth of demo_VectorFieldConservative provided in the embodiments of this specification;

[0076] Figure 6 A schematic diagram illustrating the coverage growth of demo_Koules provided in the embodiments of this specification;

[0077] Figure 7 This is a diagram illustrating the comparison of vulnerability trigger rates provided in the embodiments of this specification.

[0078] Figure 8 A schematic diagram of the structure of the ROS software package detection device based on fuzz testing provided in the embodiments of this specification;

[0079] Figure 9 This is a schematic diagram illustrating the configuration of a computer device provided in the embodiments of this specification. Detailed Implementation

[0080] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this specification.

[0081] It should be noted that, unless otherwise specified, the embodiments and features described in this specification can be combined with each other. The terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this specification are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0082] Vulnerabilities in the Robot Operating System (ROS) can be broadly categorized into three types: The first type consists of vulnerabilities introduced by developers during program writing, mostly due to improper referencing of ROS communication mechanisms or software packages; the second type is vulnerabilities inherent in ROS itself; and the third type is vulnerabilities within ROS software packages. Because ROS and its software packages are frequently referenced by developers, the second and third types of vulnerabilities are far more dangerous than the first type.

[0083] In existing technologies, most security measures for ROS are achieved through static analysis. Static analysis is a method that directly analyzes the target code, examining its syntax, interfaces, processes, and structure to identify suspicious elements, such as incorrect pointer usage, non-standard method references, and incorrect recursion. Currently, static analysis has uncovered various vulnerabilities in ROS, including stack overflows, unsafe YAML file loading, and integer overflows. However, static analysis has a very high false positive rate, resulting in significant time and manpower being consumed in checking for false positives.

[0084] This specification employs fuzz testing (fuzzing) to dynamically test ROS software packages. Fuzz testing is a software testing technique that automatically or semi-automatically generates random data input into a program and monitors for program anomalies, such as crashes and assertion failures, to detect potential program errors, such as memory leaks. However, applying fuzz testing to ROS presents certain challenges: ROS software packages exhibit strong data correlation and have stricter requirements for input data format; some existing traditional fuzzing tools cannot directly test ROS software packages.

[0085] To address the problems in the prior art, this specification discloses a ROS software package detection method based on fuzzing. It employs a fuzzing method for ROS software packages, using coverage, program errors, and memory errors as guiding mutation test cases. By continuously injecting test cases into the program under test, errors are exposed, resulting in more reliable results and a lower false positive rate. Detailed explanations follow.

[0086] Figure 1 This document illustrates a ROS software package detection method based on fuzz testing, according to Embodiment 1 of this specification. Figure 1 As shown, the ROS software package detection method includes the following steps:

[0087] Initial seed generation step S110: Generate an initial seed based on the input type of the program under test and the seed generation strategy. Mark and sort the initial seeds according to their sequence. The sorted seed queue is used as the seed pool.

[0088] Before executing the fuzz test in Embodiment 1 of this specification, a subset of test cases needs to be generated as initial data for the seed pool. In one embodiment, the seed generation strategy specifically includes:

[0089] When the input type of the program under test is int (integer type), the Integers() function strategy is executed, that is, the Integers() function based on the computer programming language Python is called to generate test cases of integer type;

[0090] When the input type of the program under test is float (floating-point), the Float() function strategy is executed, that is, the Python-based Float() function is called to generate test cases of floating-point data.

[0091] When the input type of the program under test is array type, the Lists() function strategy is executed, that is, the Python-based Lists() function is called to generate test cases containing array type data of int and float.

[0092] When the input type of the program under test is complex, the Composite() function strategy is executed, that is, the strategy function corresponding to each basic type in the input type combination is called to combine the data of each type and generate test cases of multiple mixed types. Among them, the Composite() function strategy in Embodiment 1 of this specification allows users to specify the combination of input types.

[0093] In the Integers(), Float(), Lists(), and Composite() function strategies, users set the value range for each type of data according to the data input requirements of the program under test. This ensures that the seed generation strategy generates valid initial test cases, as some ROS packages have requirements for the value range of the input data.

[0094] This ROS package testing method implements three basic strategies and one composite strategy based on different input types of the program under test. It uses the Python-based `hypothesis` package to define several generation strategies to generate the initial seed. The `hypothesis` package is a high-level Python testing library primarily used for unit testing in Python, capable of rapidly generating a large number of extreme test cases.

[0095] After generating initial seeds based on the seed generation strategy, the initial seeds are sequence-marked to sort them, and the sorted seed queue serves as the seed pool. It's important to note that the priority of the generated initial seeds is 0, meaning there is no actual priority order. However, there must be a seed at the head of the queue in storage. Therefore, sorting the generated initial seeds is primarily to obtain the seed at the head of the queue. In Embodiment 1 of this specification, the method for sequence-marking the initial seeds is not limited; each initial seed can be assigned a sequence number randomly, or the sequence number can be set based on the generation time of the initial seed.

[0096] Test case acquisition step S120: Select a seed from the seed pool according to the seed selection mechanism to obtain the selected seed, and select a mutation strategy according to the mutation strategy selection mechanism to obtain the selected mutation strategy. Mutate the selected seed based on the selected mutation strategy to obtain the test case.

[0097] In Embodiment 1 of this specification, all seeds in the seed pool have a seed priority attribute. This seed priority attribute determines the priority of a seed when it is selected. The seed pool is a queue that stores seeds. In this queue, all seeds are arranged in descending order according to their seed priority attribute. Specifically, the seed priority attribute includes a seed priority score, which is the sum of the seed performance score and the seed time score. The seed performance score depends on the seed's past performance, and the seed time score depends on the length of time the seed has been in the seed pool. In other words, the seed priority score depends on the seed's past performance and the length of time it has been in the seed pool, so that seeds with good past performance are selected with priority.

[0098] In one specific embodiment, the seed selection mechanism includes the following rules:

[0099] A1: All seeds in the seed pool are sorted in descending order according to their seed priority score.

[0100] A2: Each time a seed is selected from the seed pool, the seed at the head of the queue is selected as the chosen seed, and the chosen seed is removed from the queue.

[0101] A3: The seed time score of all unselected seeds in the seed pool is increased by the first preset threshold, and the seed priority score of the unselected seeds is updated. The unselected seeds in the seed pool are then reordered in descending order according to the updated seed priority score.

[0102] During each mutation, the ROS software package's detection method selects the seed at the head of the queue in the seed pool, i.e., the seed with the highest seed priority score, removes it from the queue, and performs mutation and execution. Furthermore, the seed priority scores of all unselected seeds are increased, and their seed time scores are increased by a first preset threshold to ensure that every seed in the seed pool has a chance to mutate and execute. The first preset threshold can be a fixed value or a value set based on the time spent in the seed pool (the time interval since the last seed selection).

[0103] Furthermore, in future executions, if the selected seed performs well after mutation, the mutated seed will be assigned an initial seed priority score and inserted into the seed pool queue.

[0104] This ROS software package integrates four mutation strategies (i.e., four mutation algorithms): bitflip mutation, bitexchange mutation, arithmetic mutation, and interesting mutation. The bitflip mutation converts data into binary (default 64 bits) and then flips the binary data according to a step size and flip amount to complete the mutation. The bitexchange mutation is similar to the bitflip mutation, converting data into binary and swapping the high-order and low-order bits. The arithmetic mutation is an arithmetic calculation consisting of addition and subtraction operations. The interesting mutation uses pre-defined "interesting values" to replace parts of the data to achieve mutation. In practice, the INTERESTING_16 dataset from the American Fuzzy Lop (AFL) fuzzing tool can be used as the dataset for the interesting mutation algorithm.

[0105] For each seed requiring mutation, the ROS software package detection method selects one of the four mutation strategies mentioned above for mutation. Furthermore, if the mutated data exceeds the range of values ​​set by the program under test for the input data, the data will be reduced to ensure it conforms to the input specifications.

[0106] In Embodiment 1 of this specification, each mutation algorithm has a mutation priority attribute. This mutation priority attribute determines the priority of the mutation algorithm when selecting a mutation strategy, so as to select the better-performing mutation algorithm more frequently. Specifically, the mutation priority attribute includes a mutation priority score, which is the sum of the mutation performance score and the mutation time score. In a specific embodiment, the mutation strategy selection mechanism includes the following rules:

[0107] B1: Each time a mutation algorithm is selected as the mutation strategy, the mutation algorithm with the highest mutation priority score is selected as the selected mutation strategy, and the mutation priority score of the selected mutation strategy is reset to 0.

[0108] B2: The seed time scores of the remaining unselected mutation algorithms are increased by the second preset threshold, and the mutation priority scores of the remaining unselected mutation algorithms are updated.

[0109] When selecting a mutation algorithm, the ROS software package detection method chooses the mutation algorithm with the highest mutation priority score as the selected mutation strategy. This mutation algorithm is then used to mutate the selected seed to obtain test cases. Simultaneously, the selected mutation algorithm is recorded in the test cases. Furthermore, the mutation priority scores of the remaining unselected mutation algorithms are increased; that is, the seed time scores of the remaining unselected mutation algorithms are increased by a second preset threshold to ensure that each mutation algorithm has a chance to be used. The second preset threshold can be set to a fixed value, and this embodiment does not impose any restrictions on this.

[0110] Furthermore, the mutation priority score of the selected mutation algorithm will be temporarily set to 0. In future executions, the selected mutation algorithm will be assigned an initial mutation priority score based on the performance of the test cases.

[0111] Test evaluation step S130: Inject test cases into the program under test for execution, collect coverage information and perform memory detection to obtain coverage information and error information, evaluate the selected seed and selected mutation strategy based on coverage information and error information and provide feedback respectively, so as to update the seed queue of the seed pool and the mutation priority attribute of the selected mutation strategy.

[0112] In Embodiment 1 of this specification, coverage information and error information are collected by executing test cases. The effectiveness of the seed and its mutation algorithm is evaluated based on these information. Specifically, this ROS software package detection method introduces two evaluation metrics: coverage improvement and program exception triggering, to effectively assess the effectiveness of the seed and its mutation algorithm. Coverage information includes new line coverage and new branch coverage. The improvement in coverage is determined by these two factors. If a test case can trigger a new line of code or branch, it means that the test case has explored a previously unexplored path, and therefore, this test case is valuable. Error information includes program error messages and memory detection error messages. If a test case can trigger errors from the program or memory detection tool during execution, it also means that the test case is valuable. After a mutated seed has been executed, the effectiveness of the mutated seed and its mutation algorithm is evaluated using two metrics: coverage improvement and program exception triggering. If the test case is deemed valuable, the mutated seed can be retained in the seed pool to provide high-performing seeds for subsequent fuzzing. Similarly, the mutation algorithm used for the mutated seed will be assigned a new initial mutation priority score to ensure that subsequent fuzzing will select better-performing mutation algorithms more frequently. Thus, this ROS package detection method uses coverage, program errors, and memory errors as guiding mutation test cases to expose errors by continuously injecting test cases into the program under test.

[0113] In one specific embodiment, the detailed steps of the evaluation feedback in the test evaluation step S130 are as follows:

[0114] a) The number of lines and branches of the newly triggered code are calculated based on the coverage information. The calculation formula is as follows:

[0115] D Line =N LineCov -O LineCov

[0116] D Branch =N BranchCov -O BranchCov

[0117] Among them, D Line The variable N represents the line number of the newly triggered code (i.e., the increment of the covered code lines). LineCov This represents the number of lines of code covered after the current execution of the program under test (i.e., the set of all lines of code covered by this seed and all previous seeds), O LineCov D represents the set of code lines covered before the current execution of the program under test (i.e., the collection of code lines previously covered by the seed). BranchN represents the number of branches that trigger the new code (i.e., the increment of the covered code branch). BranchCov This represents the code branch covered after the current execution of the program under test (i.e., the set of all code branches covered by this seed and previous seeds), O BranchCov This indicates the code branches covered before the current execution of the program under test (i.e., the collection of code branches previously covered by the seed).

[0118] b) Determine if the program error message and memory detection error message are empty. If both are empty, then the error score S is... Err The error score is 0; if at least one of the program error message and the memory detection error message is not empty, the error score S is 0. Err The value is 1.

[0119] c) Calculate the evaluation score of the test case based on the number of lines and branches of the newly triggered code and the error score. The evaluation score S of the test case is... testcase The calculation formula is:

[0120] S testcase =D Line ×1+D Branch ×2+S Err

[0121] d) Determine the evaluation score S of the test cases. testcase If the value is 0, then insert the test case as a new seed into the seed pool, and assign the test case's evaluation score S. testcase The seed priority score is used as the new seed, and the evaluation score S of the test cases is also used. testcase As the mutation priority score of the selected mutation strategy, update the mutation priority attribute of the selected mutation strategy.

[0122] In this ROS package testing method, coverage and error information are collected by executing test cases. Then, the test cases are scored based on the collected information; for each new line of code triggered, the test case receives an evaluation score S. testcase Add 1 point; for each new branch triggered, the test case's evaluation score S increases. testcase Add 2 points. Furthermore, if the test case triggers a program or memory check error at runtime, the test case's evaluation score S will be increased. testcase Add 1 point and save this test case (i.e., the mutated seed) for future debugging. Through the above steps, a final evaluation score S for the test case can be obtained. testcaseIf the obtained evaluation score is not 0, the test case is considered valuable and will be inserted into the seed pool as a seed. Its evaluation score will be used as the seed priority score for that seed, and will also be used as the mutation priority score for the mutation algorithm used in this test. Finally, the set of code lines previously covered by seeds and the set of code branches previously covered by seeds will be updated.

[0123] Some programs under test in the ROS software package have optional startup parameters. For example, users need to select the planning algorithm or mode to use through startup parameters. These startup parameters have a high probability of affecting the program execution path, thereby affecting coverage. To address this issue... Figure 2 This document illustrates a fuzz-based ROS software package detection method according to Embodiment 2 of this specification. This method cleverly selects these startup parameters to improve testing efficiency. When the program under test has at least one set of startup parameters, such as... Figure 2 As shown, the ROS software package detection method includes the following steps:

[0124] Step S210: Generate an initial seed based on the input type of the program under test and the seed generation strategy. Mark and sort the initial seeds according to their sequence. The sorted seed queue is used as the seed pool.

[0125] Before the fuzz test is executed, the ROS software package detection method of Embodiment 2 of this specification generates a portion of test cases as initial data for the seed pool based on the seed generation strategy. For specific steps, please refer to the content of the initial seed generation step S110 in Embodiment 1 of the above specification. Step S210 of Embodiment 2 of this specification is the same as the initial seed generation step S110 in Embodiment 1 of the above specification, and will not be repeated here.

[0126] Step S220: Preprocess the program to be tested, calculate the code space of each startup parameter in each group of startup parameters, and calculate the potential score of each startup parameter based on the code space of each startup parameter.

[0127] In Embodiment 2 of this specification, the ROS package detection method preprocesses the program under test before testing begins, calculates the code space of each startup parameter in each set of startup parameters, and quantifies it using a potential score. The code space of a startup parameter refers to the number of lines and branches added to the code when the startup parameter is used for the first time, reflecting the size of the code space occupied by that code. Because startup parameters directly determine the path, such as directly specifying the algorithm to be used, this characteristic can lead to a significant increase in coverage when a startup parameter is used for the first time. This increase is essentially due to the initial exploration of that code space. The number of lines and branches added when the startup parameter is used for the first time reflects, to some extent, the size of the code space occupied by that code. The larger this code space, the more worthwhile it is to explore, the greater its potential, and the better its performance in future executions.

[0128] It should be noted that the startup parameters in this manual refer to parameters with selection properties. The code space of startup parameters refers to the amount of code in the path behind each parameter. It is measured by the number of lines and branches. The more lines and branches, the more code there is.

[0129] In one specific embodiment, a set of startup parameters from the program under test is selected, and a preprocessing step is performed to calculate the potential score of each startup parameter in that set. If the program under test has multiple sets of startup parameters, the remaining sets of startup parameters that have not yet been selected are selected, and the preprocessing step is performed again until the potential score of each startup parameter in all sets of startup parameters of the program under test is calculated. The preprocessing step specifically includes the following steps:

[0130] 1) Execute the program under test with the first startup parameter in a set of startup parameters and the pre-test case, record the first coverage information, and generate a coverage information file based on the first coverage information.

[0131] 2) Execute the program under test with all startup parameters except the first startup parameter in a set of startup parameters and pre-test cases respectively. Record the number of new lines of code and branches corresponding to each startup parameter. The number of new lines of code and branches are used as the code space information of the corresponding startup parameter. At the same time, update the coverage information file.

[0132] 3) Roll back the updated overwrite information file to the state when only the first startup parameter was executed.

[0133] 4) Clear all coverage information in the coverage information file, execute the program under test with the second startup parameter in a set of startup parameters and the pre-test case to obtain the second code space corresponding to the second startup parameter. Then, execute the program under test with the first startup parameter in a set of startup parameters and the pre-test case to obtain the first code space corresponding to the first startup parameter. Calculate the code space information of the first startup parameter based on the difference between the second code space and the first code space.

[0134] 5) Based on the code space information of the first startup parameter and the code space information of all startup parameters in the set of startup parameters except the first startup parameter, calculate the code space of each startup parameter in the set of startup parameters and the total code space of the set of startup parameters. The potential score of each startup parameter in the set of startup parameters is the ratio of the code space of each startup parameter to the total code space of the set of startup parameters.

[0135] In summary, for each set of startup parameters, firstly, the program is executed using the first parameter and test cases in that set, and coverage information is recorded. Then, the following operations are performed on all parameters except the first startup parameter: keeping the test cases unchanged, the program under test is re-executed using that startup parameter, and the number of newly added lines of code and branches is recorded as the code space information for that startup parameter. Finally, the coverage information file is rolled back to the state when only the first startup parameter was executed. For the code space of the first startup parameter, all coverage information needs to be cleared, the test cases remain unchanged, the second startup parameter and the first startup parameter are executed sequentially, and then the difference between the two is used to calculate the code space of the first startup parameter. Thus, the code space of each startup parameter in that set is calculated. The potential score of a startup parameter is obtained by dividing its code space by the code space of all startup parameters in that set. If the program under test has multiple sets of startup parameters, each set of startup parameters is selected and preprocessed separately, and the code space and potential score of each set of startup parameters are calculated sequentially. It is important to note that when calculating the code space and potential score of each set of startup parameters, the settings of other startup parameters in the program under test must remain unchanged. It is important to note that...

[0136] Step S230: Select a seed from the seed pool according to the seed selection mechanism to obtain the selected seed, select a mutation strategy according to the mutation strategy selection mechanism to obtain the selected mutation strategy, select each set of startup parameters of the program under test according to the startup parameter selection mechanism to obtain the selected startup parameters, mutate the selected seed based on the selected mutation strategy to obtain test cases.

[0137] The seed selection mechanism, mutation strategy selection mechanism, and four mutation algorithms in Embodiment 2 of this specification are the same as those in Embodiment 1 of the above specification. For details, please refer to the test case acquisition step S120 in Embodiment 1 of the above specification, which will not be repeated here.

[0138] In Embodiment 2 of this specification, after obtaining the potential score of each startup parameter through step S220, before executing the program under test, for any set of startup parameters, the ROS software package detection method selects each set of startup parameters according to the startup parameter selection mechanism. Each startup parameter in all sets of startup parameters of the program under test has a parameter priority attribute and a usage frequency attribute. The parameter priority attribute determines the priority of the startup parameter when selecting each set of startup parameters, so as to select startup parameters that have performed better in the past more frequently. The usage frequency attribute is used to record the number of times the startup parameter is used. Specifically, the parameter priority attribute includes a parameter priority score, which depends on the potential score and the parameter time score of the startup parameter, and the usage frequency attribute includes the number of times the parameter is used.

[0139] In one specific embodiment, the startup parameter selection mechanism includes the following rules:

[0140] C1: Each time a set of startup parameters for the program under test is selected, the startup parameter with the highest priority score in each set of startup parameters is selected as the selected startup parameter. The usage count of the selected startup parameter is incremented by 1, and the priority score of the selected startup parameter is reset to 0.

[0141] C2: Calculate and update the parameter priority score of the remaining unselected startup parameters in each group of startup parameters. The formula for updating the parameter priority score of the remaining unselected startup parameters in each group of startup parameters is as follows:

[0142] S priority =S' priority +S time *(S potential +1)

[0143] Among them, S priority S' represents the priority score of the updated startup parameters after the initial selection was not selected. priority S represents the priority score of the unselected startup parameters before the update. time S represents the time score of the startup parameter that was not selected. potential This indicates the potential score for the startup parameter that was not selected.

[0144] When selecting startup parameters for each group, this ROS package detection method chooses the startup parameter with the highest priority score as the selected startup parameter for that group. The priority scores of the remaining startup parameters in that group that were not selected are all increased. For details, please refer to the updated calculation formula for the priority scores of the remaining startup parameters in each group, as described above, to ensure that each startup parameter has a chance to be selected. Furthermore, the startup parameters in this ROS package detection method have a usage count attribute to determine whether a selected startup parameter is being used for the first time. This prevents misjudgments during the evaluation step due to a significant increase in coverage caused by the first use of a startup parameter.

[0145] Furthermore, the parameter priority score of the selected startup parameter in this round will be temporarily set to 0. In future executions, the selected startup parameter in this round will be assigned an initial parameter priority score based on the performance of the test cases.

[0146] Step S240: Execute the program under test with the selected startup parameters and test cases, collect coverage information and perform memory detection to obtain coverage information and error information, evaluate the selected seed, selected mutation strategy and selected startup parameters based on the coverage information and error information and provide feedback respectively, so as to update the seed queue of the seed pool, the mutation priority attribute of the selected mutation strategy and the parameter priority attribute of the selected startup parameters.

[0147] In Embodiment 2 of this specification, the coverage information includes new line coverage information and new branch coverage information, and the error information includes program error information and memory detection error information. Similar to Embodiment 1 of the specification above, both introduce two evaluation indicators: coverage improvement and program exception triggering, to effectively evaluate the effectiveness of the seed and the mutation algorithm used. Therefore, for any content not mentioned in Embodiment 2 of this specification, please refer to the content of test evaluation step S130 in Embodiment 1 of the specification above.

[0148] In one specific embodiment, the detailed steps for evaluating the feedback in step S240 are as follows:

[0149] I) Calculate the number of lines and branches of the newly triggered code based on the coverage information. The calculation formula is as follows:

[0150] D Line =N LineCov -O LineCov

[0151] D Branch =N BranchCov -O BranchCov

[0152] Among them, D LineThe variable N represents the line number of the newly triggered code (i.e., the increment of the covered code lines). LineCov This represents the number of lines of code covered after the current execution of the program under test (i.e., the set of all lines of code covered by this seed and all previous seeds), O LineCov D represents the set of code lines covered before the current execution of the program under test (i.e., the collection of code lines previously covered by the seed). Branch N represents the number of branches that trigger the new code (i.e., the increment of the covered code branch). BranchCov This represents the code branch covered after the current execution of the program under test (i.e., the set of all code branches covered by this seed and previous seeds), O BranchCov This indicates the code branches covered before the current execution of the program under test (i.e., the collection of code branches previously covered by the seed).

[0153] Ⅱ) Determine if the program error message and memory detection error message are empty. If both are empty, then the error score S is... Err The error score is 0; if at least one of the program error message and the memory detection error message is not empty, the error score S is 0. Err The value is 1.

[0154] Ⅲ) Extract the usage count attribute of the selected startup parameters in each group of startup parameters of the program under test, and determine whether there is a startup parameter with a usage count value of 1 among the selected startup parameters;

[0155] If it exists, then the evaluation score S of the test case is... testcase for:

[0156] S testcase =S Err

[0157] If it does not exist, then the evaluation score S of the test case is... testcase for:

[0158] S testcase =D Line ×1+D Branch ×2+S Err

[0159] IV) Determine the evaluation score S of the test cases. testcase If the value is 0, then insert the test case as a new seed into the seed pool, and assign the test case's evaluation score S. testcase The seed priority score is used as the new seed, and the evaluation score S of the test cases is also used. testcase As the mutation priority score of the selected mutation strategy and the parameter priority score of the selected initiation parameter, update the mutation priority attribute of the selected mutation strategy and the parameter priority attribute of the selected initiation parameter.

[0160] In this ROS package testing method, coverage and error information are collected by executing test cases. Then, the test cases are scored based on the collected information; for each new line of code triggered, the test case receives an evaluation score S. tastcase Add 1 point; for each new branch triggered, the test case's evaluation score S increases. tastcase Add 2 points. Furthermore, if the test case triggers a program or memory check error at runtime, the test case's evaluation score S will be increased. tastcase Add 1 point and save the test case (i.e., the mutated seed) for future debugging. If any startup parameter in the test case is being used for the first time, the test case's evaluation score S is increased. testcase The coverage score (i.e., the score calculated based on the number of lines and branches of newly triggered code) will be removed. After completing the above steps, a final evaluation score S for the test case can be obtained. tastcase If the obtained evaluation score is not 0, the test case is considered valuable and will be inserted into the seed pool as a seed. Its evaluation score will serve as the seed priority score for that seed, and will also be used as the mutation priority score for the mutation algorithm used in this test and the parameter priority score for the selected start-up parameters. Finally, the set of code lines previously covered by seeds and the set of code branches previously covered by seeds will be updated.

[0161] The above describes the process and working principle of the ROS software package detection method based on fuzz testing provided in this manual. The following section will combine... Figure 1 – Figure 6 This paper details the experiment of applying the fuzz-based ROS package detection method to the default ROS motion planning package OMPL (The Open Motion Planning Library).

[0162] To accommodate various versions of the Ubuntu operating system, ROS has released different versions. To ensure the timeliness of this experiment, this manual uses the noetic version, which has a maintenance period until 2025. The ROS package used for testing is the default motion planning package OMPL, version 1.5.2.

[0163] Because the OMPL package contains a large number of algorithms and their implementation is relatively complex, this experiment selects a portion of the programs in the OMPL package for testing. Since the demo programs in the OMPL package are completed by the official OMPL team, they can better guarantee the correctness of the use of OMPL algorithms and reduce false alarms. Moreover, for OMPL users, the demo programs are the most important reference when writing programs using OMPL. If there are errors in the demo programs, it may mislead users and cause unnecessary trouble. Therefore, this experiment modifies a portion of the demo programs in the OMPL package to make them test programs that can accept the test cases in the detection methods of this ROS software package.

[0164] like Figure 3 As shown, this experiment will analyze the input of the demo program to be used, and then modify the demo program to make it a test program that can receive test cases in the ROS package's testing methods. In the OMPL package, most programs use arrays and floating-point numbers as input types. This is mainly because the OMPL package's functionality primarily involves data computation, requiring users to set spatial states and then find suitable paths. In this process, floating-point numbers and arrays are needed to represent upper and lower bounds, starting positions, ending points, and obstacle ranges. After determining the input format, the initial seeds are obtained according to the determined input format through a seed generation strategy, and these seeds are then sequentially labeled and placed into a seed pool. When running the test program, the test program will read the data from the selected seeds for testing. To test more functions and algorithms, several demo programs will be selected for this experiment using this method.

[0165] Regarding coverage improvement:

[0166] In fuzzing, a crucial metric is the ability to cover more lines of code and branches. This experiment used several programs from the demo program. For programs other than demo_Koules (which took approximately 16 hours to run due to its longer runtime), the experiment employed the fuzzing-based ROS package detection method described in this manual for approximately 8 hours of fuzzing, collecting coverage information and executing a total of 28,645 test cases. The following examples from the demo program—demo_RigidBodyPlanningWithODESolverAndControls, demo_VectorFieldConservative, and demo_Koules—demonstrate the growth in line and branch coverage of the tested programs. In this experiment, the fuzzing-based ROS package detection method described in this manual generated and executed 9,540 test cases for these three programs. With the injection of test cases, the row and branch coverage of demo_RigidBo dyPlanningWithODESolverAndControls and demo_VectorFi-eldConservative increased as follows: Figure 4 and Figure 5 As shown.

[0167] like Figure 4 As shown, with the injection of test cases, the line coverage of `demo_RigidBodyPlanningWithODE-SolverAndControls` increased from 1511 lines to 1630 lines, an increase of approximately 7.88%, and the branch coverage increased from 829 to 938, an increase of approximately 13.15%. Figure 5 As shown, with the injection of test cases, the row coverage of `demo_VectorFieldConservat` increased from 1740 lines to 1866 lines, an increase of approximately 7.24%, and the branch coverage increased from 1376 to 1520, an increase of approximately 10.47%. Since the input of both programs consists entirely of specific coordinate values, and their input type is float, float cannot achieve the same significant coverage change effect as boolean input. Therefore, the improvement in row and branch coverage is relatively low. However, the ROS package detection method based on fuzzing in this specification can still achieve a code increase of approximately 7%–8% and a branch increase of 10%–14%. Therefore, when the input is generally coordinate data, this ROS package detection method can achieve higher coverage.

[0168] In the demo programs of the OMPL package, some programs, such as demo_Koules, input not only simple coordinate values ​​but also a mixture of algorithm selection, pattern selection, and coordinate values. When testing such programs, the mutation strategy selection mechanism from the ROS package's detection method was used for the pattern selection and algorithm selection parts of the input. This maximized the test coverage of algorithms and patterns with higher error rates. Because demo_Koules has a long single execution time, its test duration was 16 hours, during which 3573 test cases were generated and executed. With the injection of test cases, the growth of line coverage and branch coverage of demo_Koules is as follows: Figure 6 As shown.

[0169] like Figure 6 As shown, with the injection of test cases, the line coverage of `demo_Koules` increased from 1423 lines to 4352 lines, an increase of approximately 205.83%, and the branch coverage increased from 902 to 3446, an increase of approximately 282.04%. The reason for such a significant increase in line and branch coverage of `demo_Koules` is mainly because `demo_Koules` requires the selection of certain program startup parameters. Changing these startup parameters will cause significant changes to the classes, functions, and execution paths called during program execution, thus greatly affecting the program's line and branch coverage. Although these program startup parameters have a significant impact on coverage, their influence will greatly decrease once all combinations of these startup parameters are exhausted. At this point, float-type coordinate values ​​will become the main factor affecting coverage changes. Figure 6 As shown, even after exhausting most possible combinations of program startup parameters, this ROS package detection method can still achieve a relatively stable improvement in line and branch coverage through variations of float type inputs and a startup parameter selection mechanism. Therefore, under these conditions, this ROS package detection method can also achieve higher coverage.

[0170] In summary, the ROS software package detection method based on fuzz testing described in this specification can improve coverage by using mutant seeds.

[0171] Regarding the triggering of the seed vulnerability:

[0172] When evaluating the quality of generated seeds, besides coverage, another crucial metric is the number of seeds that successfully trigger vulnerabilities in the program. To assess the ability of this ROS software package to generate seeds that trigger vulnerabilities, this experiment introduced the traditional fuzzing tools AFL and libFu-zzer as control groups, conducting the same experiments as described above. In the experiment, the number of seeds generated and executed by these three tools, the number of seeds that could trigger vulnerabilities, and the percentage of seeds that could trigger vulnerabilities out of the total generated seeds were counted. The results are as follows: Figure 7 As shown in Table 1 below.

[0173]

[0174] Table 1. Specific data on vulnerability triggering.

[0175] Experimental results show that, in testing `demo_RigidBodyPlanningWithODESolverAndC-ontrols` and `demo_VectorFieldConservative`, the seeds generated by this ROS package detection method (RPFuzz) are slightly more effective at triggering vulnerabilities than AFL and libFuzzer. Compared to AFL and libFuzzer's direct mutation of the byte stream, this ROS package detection method considers the data type and value range when mutating data, resulting in more valid data and thus better vulnerability triggering and fewer false positives. However, the difference between this ROS package detection method and traditional tools is significant in `demo_Koules`. This is mainly because `demo_Koules` requires selecting certain program startup parameters, and this ROS package detection method can better select these parameters using its startup parameter selection mechanism, ensuring that vulnerable startup parameters are selected more frequently. In conclusion, the seeds generated by this ROS package detection method are more effective at triggering vulnerabilities in programs.

[0176] Regarding the triggering of the vulnerability:

[0177] In the above-mentioned triggering of the seed vulnerability, the vulnerability shown in Table 2 below was successfully triggered and two CVE numbers were obtained.

[0178]

[0179] Table 2 Vulnerabilities discovered by ROS software package detection methods

[0180] In Ompl / src / ompl / geometric / planners / prm / src / LazyPRM.cpp, the cloneState() function allocates memory. The freeMemory() function within the class is supposed to release the memory allocated by cloneState(). However, during the vulnerability reproduction process, the program still called cloneState() after the last call to freeMemory(), and this memory was not released, leading to a memory leak. To avoid this problem, call freeMemory() in the destructor. The vulnerability and its fix have been reported to the official team and confirmed, and a CVE number has been obtained for this vulnerability.

[0181] In Ompl / src / ompl / geometric / src / SimpleSetup.cpp, a pointer was not checked for null before use. During the reproduction of the vulnerability, a segmentation fault occurred when calling the function pointed to by the pointer because the pointer was null. The vulnerability has been reported to the official documentation.

[0182] In Ompl / src / ompl / geometric / planners / rrt / src / VFRRT.cpp, a reassignment of a pointer within a Motion object caused a memory leak in the memory space that the pointer originally pointed to. The leaked portion is not used in the program, so avoiding allocating that memory segment resolves the issue. The vulnerability and its fix have been reported to the official team and confirmed, and a CVE number has been obtained for this vulnerability.

[0183] In boost / random / uniform_on_sphere.hpp (Boost 1.58), due to programmer oversight, a break statement was not added to case 1 in the switch function, causing the program to continue running to case 2, resulting in a HeapBuffer Overflow.

[0184] In addition, the experiment also discovered some vulnerabilities in the official demo program, as shown in Table 3 below. Although these vulnerabilities exist in the demo program, their impact on daily production is relatively small. However, the demo program serves as an important reference for users when writing programs, and errors in it may lead users to learn incorrect usage methods, causing unnecessary trouble.

[0185]

[0186] Table 3 shows the vulnerabilities found in the demo program using the ROS software package detection method.

[0187] In summary, this specification discloses a fuzz-based ROS software package detection method, which can effectively discover vulnerabilities in ROS software packages. This method improves the quality of generated seeds by introducing two evaluation metrics: whether coverage can be improved and whether program errors or memory errors can be triggered. Furthermore, this method integrates four seed generation strategies and four seed mutation strategies. During mutation, test cases are generated based on coverage, program errors, and memory errors, continuously injecting test cases into the program under test to expose its errors. In addition, for some programs under test with selective startup parameters, this method also establishes a startup parameter selection mechanism to improve the testing efficiency of such programs.

[0188] Corresponding to the above method embodiments, this specification provides a ROS software package detection device based on fuzz testing, such as... Figure 8 As shown, the ROS software package detection device mainly includes: an initial seed generation module 31, a test case acquisition module 32, and a test evaluation module 33.

[0189] The initial seed generation module 31 is used to generate an initial seed based on the input type of the program under test and the seed generation strategy, to perform sequence marking and sorting on the initial seed, and the sorted seed queue serves as the seed pool.

[0190] The test case acquisition module 32 is used to select seeds in the seed pool according to the seed selection mechanism to obtain the selected seeds, and to select a mutation strategy according to the mutation strategy selection mechanism to obtain the selected mutation strategy. Based on the selected mutation strategy, the selected seeds are mutated to obtain test cases.

[0191] The test evaluation module 33 is used to inject test cases into the program under test for execution, collect coverage information and perform memory detection to obtain coverage information and error information, evaluate the selected seeds and selected mutation strategies based on coverage information and error information and provide feedback respectively, so as to update the seed queue of the seed pool and the mutation priority attribute of the selected mutation strategy.

[0192] Specific limitations regarding the fuzz-based ROS software package detection device can be found in the aforementioned limitations on the fuzz-based ROS software package detection method, and will not be repeated here. Each module in the aforementioned fuzz-based ROS software package detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0193] This specification also discloses a computer device, such as... Figure 9 As shown, it includes a memory 42 and a processor 41. The memory 42 stores a computer program, and the processor 41 executes the computer program to implement the steps of the above-described ROS software package detection method based on fuzz testing.

[0194] This specification also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described ROS software package detection method based on fuzz testing.

[0195] This specification also discloses a computer program product containing instructions that, when executed on a computer or processor, cause the computer or processor to perform the steps of the aforementioned ROS software package detection method based on fuzz testing.

[0196] It will be understood by those skilled in the art that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this specification. Furthermore, it will be understood by those skilled in the art that the modules in the apparatus of the embodiments may be distributed within the apparatus of the embodiments as described in the embodiments, or may be located in one or more apparatuses different from this embodiment with corresponding changes. The modules of the above embodiments may be combined into one module, or may be further divided into multiple sub-modules.

[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this specification, and are not intended to limit them. Although this specification has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the above embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this specification.

Claims

1. A method for detecting ROS software packages based on fuzz testing, characterized in that, include: Initial seed generation steps: Generate initial seeds based on the input type of the program under test and the seed generation strategy; label and sort the initial seeds according to their sequence; and use the sorted seed queue as the seed pool. Test case acquisition steps: Select a seed from the seed pool according to the seed selection mechanism to obtain the selected seed; select a mutation strategy according to the mutation strategy selection mechanism to obtain the selected mutation strategy; mutate the selected seed based on the selected mutation strategy to obtain the test case. Test evaluation steps: Inject the test cases into the program under test for execution, collect coverage information and perform memory detection to obtain the coverage information and error information, evaluate the selected seed and the selected mutation strategy based on the coverage information and the error information and provide feedback respectively, so as to update the seed queue of the seed pool and the mutation priority attribute of the selected mutation strategy. Wherein, if the program under test has at least one set of startup parameters, Before the test case acquisition step, the ROS package detection method further includes: preprocessing the program under test, calculating the code space of each startup parameter in each group of startup parameters, and calculating the potential score of each startup parameter based on the code space of each startup parameter; Prior to the test evaluation step, the ROS software package detection method further includes: selecting each set of startup parameters for the program under test according to the startup parameter selection mechanism to obtain the selected startup parameters; The test evaluation steps specifically include: executing the program under test with the selected startup parameters and the test cases, collecting coverage information and performing memory detection to obtain the coverage information and error information, evaluating the selected seed, the selected mutation strategy and the selected startup parameters based on the coverage information and the error information and feeding back the evaluation results respectively, so as to update the seed queue of the seed pool, the mutation priority attribute of the selected mutation strategy and the parameter priority attribute of the selected startup parameters; The preprocessing of the program under test, including calculating the code space of each startup parameter in each set of startup parameters and calculating the potential score of each startup parameter based on its code space, specifically includes: Select a set of startup parameters from the program under test and perform a preprocessing step, the preprocessing step including: The program under test is executed using the first startup parameter in the set of startup parameters and the pre-test case, the first coverage information is recorded, and a coverage information file is generated based on the first coverage information. The program under test is executed with all startup parameters except the first startup parameter in the set of startup parameters and the pre-test cases respectively. The number of newly added lines of code and the number of branches corresponding to each startup parameter are recorded. The number of newly added lines of code and the number of branches are used as the code space information of the corresponding startup parameter. At the same time, the coverage information file is updated. Roll back the updated overwrite information file to the state when only the first startup parameter was executed; Clear all coverage information in the coverage information file, execute the program under test with the second startup parameter in the set of startup parameters and the pre-test case to obtain the second code space corresponding to the second startup parameter, then execute the program under test with the first startup parameter in the set of startup parameters and the pre-test case to obtain the first code space corresponding to the first startup parameter, and calculate the code space information of the first startup parameter based on the difference between the second code space and the first code space; Based on the code space information of the first startup parameter and the code space information of all startup parameters in the set of startup parameters except the first startup parameter, the code space of each startup parameter in the set of startup parameters and the total code space of the set of startup parameters are calculated. The potential score of each startup parameter in the set of startup parameters is the ratio of the code space of each startup parameter to the total code space of the set of startup parameters. Select the remaining groups of startup parameters that have not been selected in the program under test, and perform the preprocessing steps until the potential score of each startup parameter in all groups of startup parameters of the program under test is calculated.

2. The ROS software package detection method based on fuzz testing according to claim 1, characterized in that, Each startup parameter in all groups of startup parameters of the program under test has a parameter priority attribute and a usage frequency attribute. The parameter priority attribute includes a parameter priority score, which depends on the potential score and time score of the startup parameter. The usage frequency attribute includes a usage frequency value. The startup parameter selection mechanism specifically includes: Each time a set of startup parameters for the program under test is selected, the startup parameter with the highest parameter priority score in each set of startup parameters is selected as the selected startup parameter. The usage count of the selected startup parameter is incremented by 1, and the parameter priority score of the selected startup parameter is reset to 0. Calculate and update the parameter priority score of the remaining unselected startup parameters in each group of startup parameters. The formula for updating the parameter priority score of the remaining unselected startup parameters in each group of startup parameters is as follows: ； in, This indicates the priority score of the updated startup parameters after the initial selection was not selected. This indicates the priority score of the unselected startup parameters before the update. This indicates the time score of the startup parameter that was not selected. This indicates the potential score for the startup parameter that was not selected.

3. The ROS software package detection method based on fuzz testing according to claim 1, characterized in that, The seed generation strategy specifically includes: When the input type of the program under test is int, the Integers() function strategy is executed; When the input type of the program under test is float, the Float() function strategy is executed; When the input type of the program under test is array, the Lists() function strategy is executed; When the input type of the program under test is complex, the Composite() function strategy is executed. In the Integers(), Float(), Lists(), and Composite() functions, the user sets the value range for each type of data according to the data input requirements of the program under test.

4. The ROS software package detection method based on fuzz testing according to claim 1, characterized in that, All seeds in the seed pool have a seed priority attribute, which includes a seed priority score, which is the sum of the seed performance score and the seed time score. The seed selection mechanism specifically includes: All seeds in the seed pool are sorted in descending order according to their seed priority scores. Each time a seed is selected from the seed pool, the seed at the head of the queue is selected as the chosen seed, and the chosen seed is removed from the queue. The seed time score of all unselected seeds in the seed pool is increased by a first preset threshold, and the seed priority score of the unselected seeds is updated. The unselected seeds in the seed pool are then reordered in descending order according to the updated seed priority scores.

5. The ROS software package detection method based on fuzz testing according to claim 1, characterized in that, The mutation strategies include bitflip mutation algorithm, bitexchange mutation algorithm, arithmetic mutation algorithm, and interesting mutation algorithm. Each mutation algorithm has a mutation priority attribute, which includes a mutation priority score, which is the sum of the mutation performance score and the mutation time score. The mutation strategy selection mechanism specifically includes: Each time a mutation algorithm of the mutation strategy is selected, the mutation algorithm with the highest mutation priority score is selected as the selected mutation strategy, and the mutation priority score of the selected mutation strategy is reset to 0. The seed time scores of the remaining unselected mutation algorithms are increased by a second preset threshold, and the mutation priority scores of the remaining unselected mutation algorithms are updated.

6. The ROS software package detection method based on fuzz testing according to claim 1, characterized in that, The coverage information includes newline coverage information and new branch coverage information, and the error information includes program error information and memory detection error information. The process of evaluating the selected seed and the selected mutation strategy based on the coverage information and the error information, and then feeding back the evaluations to update the seed queue of the seed pool and the mutation priority attribute of the selected mutation strategy, specifically includes: The number of lines and branches of the new triggering code are calculated based on the coverage information, using the following formula: ； ； in, The variable representing the line number of the newly triggered code. This indicates the line of code that will be overwritten after the current execution of the program under test. This indicates the lines of code that were overwritten before the current execution of the program under test. The variable representing the number of branches in the newly triggered code. This indicates the code branch that will be overwritten after the current execution of the program under test. This indicates the code branch that was covered before the current execution of the program under test; If both the program error message and the memory detection error message are empty, then the error score is... The error score is 0; if at least one of the program error message and the memory detection error message is not empty, then the error score is 0. =1; The evaluation score of the test case is calculated based on the number of lines and branches of the newly triggered code and the error score. The calculation formula is: ； Determine the evaluation score of the test cases. Is it 0? If not, the test case is inserted into the seed pool as a new seed, and the evaluation score of the test case is... The seed priority score is used as the seed priority score for the new seed, and the evaluation score of the test case is also used. The mutation priority attribute of the selected mutation strategy is updated as the mutation priority score of the selected mutation strategy.

7. The ROS software package detection method based on fuzz testing according to claim 1, characterized in that, The coverage information includes newline coverage information and new branch coverage information, and the error information includes program error information and memory detection error information. The process of evaluating and feeding back the selected seed, selected mutation strategy, and selected startup parameters based on the coverage information and the error information to update the seed queue of the seed pool, the mutation priority attribute of the selected mutation strategy, and the parameter priority attribute of the selected startup parameters specifically includes: The number of lines and branches of the new triggering code are calculated based on the coverage information, using the following formula: ； ； in, The variable representing the line number of the newly triggered code. This indicates the line of code that will be overwritten after the current execution of the program under test. This indicates the lines of code that were overwritten before the current execution of the program under test. The variable representing the number of branches in the newly triggered code. This indicates the code branch that will be overwritten after the current execution of the program under test. This indicates the code branch that was covered before the current execution of the program under test; If both the program error message and the memory detection error message are empty, then the error score is... The error score is 0; if at least one of the program error message and the memory detection error message is not empty, then the error score is 0. =1; Extract the usage count attribute of the selected startup parameters in each group of startup parameters of the program under test, and determine whether there is a startup parameter with a usage count value of 1 among the selected startup parameters; If it exists, then the evaluation score of the test case is... for: ； If it does not exist, then the evaluation score of the test case is... for: ； Determine the evaluation score of the test cases. Is it 0? If not, the test case is inserted into the seed pool as a new seed, and the evaluation score of the test case is... The seed priority score is used as the seed priority score for the new seed, and the evaluation score of the test case is also used. The mutation priority attribute of the selected mutation strategy and the parameter priority attribute of the selected initiation parameter are updated as the mutation priority score of the selected mutation strategy and the parameter priority score of the selected initiation parameter.

8. A ROS software package detection device based on fuzz testing, characterized in that, A method for implementing the ROS package detection method based on fuzz testing as described in any one of claims 1–7 includes: The initial seed generation module is used to generate an initial seed based on the input type of the program under test and a seed generation strategy, and to perform sequence marking and sorting on the initial seeds. The sorted seed queue serves as the seed pool. The test case acquisition module is used to select seeds in the seed pool according to the seed selection mechanism to obtain the selected seeds, and to select a mutation strategy according to the mutation strategy selection mechanism to obtain the selected mutation strategy. Based on the selected mutation strategy, the selected seeds are mutated to obtain test cases. The test evaluation module is used to inject the test cases into the program under test for execution, collect coverage information and perform memory detection to obtain the coverage information and error information, evaluate the selected seed and the selected mutation strategy based on the coverage information and the error information and provide feedback respectively, so as to update the seed queue of the seed pool and the mutation priority attribute of the selected mutation strategy.

Images