Machine learning guided automatic driving vehicle laser radar firmware fuzzing method

Abstract

The application discloses a kind of machine learning guided automatic driving vehicle laser radar firmware fuzz testing methods, and the method steps include: based on static analysis and artificial adjustment, obtain the final radar firmware interface data specification model;Based on the radar firmware input interface data specification model that has been constructed, generate effective test cases, train neural network model using test cases;The neural network model and radar firmware interface data specification model constructed in the above steps are combined, and the test cases that meet the conditions are guided to be generated and executed.The application is directed to the characteristics that the automatic driving vehicle laser radar firmware interface protocol is special and the internal state space is complex, proposes the automatic driving vehicle laser radar firmware fuzz testing method, by combining radar firmware interactive interface specification, guided fuzz testing using machine learning technology, efficiently generate test input that meets radar firmware interface specification and efficiently explore radar firmware state space to detect vulnerabilities.

Description

Technical Field

[0001] This invention relates to fuzz testing methods, belonging to the field of software engineering, and specifically to a machine learning-guided fuzz testing method for firmware of autonomous driving vehicle-mounted LiDAR. Background Technology

[0002] Radar (Radio Detection and Ranging) uses radio waves to detect and range targets. After nearly a century of development, radar system technology has continuously improved, especially with the advent of LiDAR (Light Detection and Ranging), which has greatly enhanced the system's target measurement, identification, and anti-interference capabilities. Radar has wide applications in many fields, one important application being vehicle perception of the external environment. Compared to cameras, radar's detection performance is less affected by complex external environments, hence the widespread deployment of automotive LiDAR. Especially in recent years, with the rise of autonomous driving technology, LiDAR has become a crucial component of autonomous driving perception modules, directly impacting the correct implementation of autonomous driving functions. The development process of a radar system mainly includes design, debugging, testing, production, and delivery stages. Among these, radar testing ensures that the developed radar product conforms to the design and improves the reliability and safety of the radar system itself, thus playing a pivotal role in modern radar system development. Automotive LiDAR connects to other electronic control units within the vehicle via a bus, and its software system receives and parses specific format data messages from other electronic control units, driving the radar's operation. Autonomous driving vehicle-mounted LiDAR firmware is implemented in C / C++. Compared to general C / C++ software, its code is more complex, its operating environment is more unique, and its interface protocols are more complex. Hidden serious defects such as data out-of-bounds errors and memory leaks, as well as incomplete functionalities, often go undetected. As a crucial component of the vehicle's perception module, any defects in the radar firmware can have serious consequences. Current radar firmware interface testing primarily employs methods that pre-generate test data by analyzing the interface design. Test pass / fail decisions typically require manual verification, resulting in weak defect detection capabilities. The large scale, algorithmic complexity, numerous configuration items, and complex testing conditions of radar firmware make testing a demanding and challenging task.

[0003] The core idea of ​​fuzzing is to generate a large number of valid or partially valid test inputs and send them to the running system under test. By monitoring the system's operating status, security violations and phenomena such as program crashes can be detected, thus achieving the purpose of vulnerability detection. This method is simple, easy to implement, and highly scalable, and is widely used in practice. However, fuzzing technology, especially feedback-guided gray-box fuzzing, faces many challenges when applied to radar firmware testing. First, radar system interaction interfaces (such as network protocol messages) have relatively strict constraints in terms of data type, syntax format, and value range. Fuzzing must fully consider the interaction interface specifications when generating test inputs to generate valid test inputs and effectively explore the multi-dimensional input space. Second, the size and complexity of radar system code are increasing day by day. The main goal of existing radar testing is to maximize code coverage and thus improve the probability of vulnerability detection. However, faced with the huge radar system code space, existing gray-box fuzzing methods based on genetic algorithms are often inefficient when exploring unknown state spaces. Vulnerability detection is random and blind, resulting in a huge waste of testing resources and low vulnerability detection efficiency. Summary of the Invention

[0004] This invention proposes a machine learning-guided fuzzy testing method for autonomous driving vehicle-mounted LiDAR firmware, aiming to solve the problems of existing testing methods for LiDAR firmware requiring extensive manual analysis to extract LiDAR firmware interface data specifications, and the resulting test cases being unable to efficiently explore the internal state space of the LiDAR firmware.

[0005] To solve the above problems, the present invention is achieved through the following technical solution:

[0006] A machine learning-guided method for firmware fuzz testing of autonomous driving vehicle-mounted LiDAR, characterized by the following steps:

[0007] Step 1) Construction of the firmware interface data specification model for autonomous driving LiDAR; Static analysis is performed based on the source code to obtain the LiDAR firmware interface data specification model; The LiDAR firmware interface data specification model obtained from the static analysis is manually adjusted to obtain the final LiDAR firmware interface data specification model;

[0008] Step 2) Model Training: Based on the final radar firmware input interface data reduction model obtained in Step 1), generate valid initialization seed test cases, and use the test cases as training data to train the neural network model. The input of the neural network model is a byte sequence of a single test case, and the output is a bit vector representing the program edge coverage.

[0009] Step 3) Test execution: Based on the neural network model constructed in step 2), and combined with the radar firmware interface data specification model constructed in step 1), guide the generation of test cases that conform to the radar firmware interface data specification and can fully explore the internal state space of the radar firmware, and execute them.

[0010] The aforementioned machine learning-guided firmware fuzzy testing method for autonomous driving LiDAR is characterized in that step 1) specifically includes the following steps:

[0011] Step 11) Extraction of radar firmware interface data specifications based on static analysis; by performing static analysis on the source code of autonomous vehicle-mounted LiDAR firmware, a control flow graph is constructed, and the data structure definitions used to represent program input in the code are identified, and the data type and value range of each data member are automatically extracted.

[0012] Step 12) Manual adjustment of data specification; providing a human-computer interaction interface so that users can freely adjust the input data specification obtained from static analysis in step 11) to obtain the final radar firmware input data specification model.

[0013] The aforementioned machine learning-guided firmware fuzzy testing method for autonomous driving LiDAR is characterized in that step 2) specifically includes the following steps:

[0014] Step 21) Training set generation; Based on the final radar firmware input interface data reduction model obtained in step 12), generate valid initialization seed test cases, send the test cases to the radar firmware for execution, and record the edge coverage information of the radar system program space fed back by the execution of the test cases.

[0015] Step 22) Model training: Based on the collected valid test cases and coverage feedback information, a machine learning algorithm is used for training and learning, thereby generating a neural network model for predicting the relationship between test inputs and edge coverage.

[0016] The aforementioned machine learning-guided firmware fuzzy testing method for autonomous driving LiDAR is characterized in that step 3) specifically includes the following steps:

[0017] Step 31) Gradient-guided mutation generation: Based on the seed set constructed in Step 2), for each test case, input it into the neural network model, calculate the gradient information of the model output layer with respect to the input layer, determine the bytes in a test input that may improve the coverage of the program under test and the mutation direction of the byte value according to the gradient information; combine the data reduction model output in Step 1) to generate test cases that conform to the radar firmware data specification, and send the test cases to the radar firmware for execution;

[0018] Step 32) Test execution and feedback information collection; construct the required operating environment for the radar firmware, drive the radar firmware to run, and send the generated variants to the radar firmware under test for execution; collect the coverage information and whether a crash was triggered during the test process. If the test does not reach the predetermined number of test rounds, add test cases that can improve coverage to the seed set, retrain the model, and start a new round of test execution; if the test reaches the predetermined number of test rounds, output the crashes triggered and the corresponding test cases as a test report.

[0019] Step 33) Model retraining: Retrain the model using the latest seed set, and then perform a new round of testing based on the retrained model.

[0020] By adopting the above technical solutions, the present invention can achieve the following beneficial effects:

[0021] The machine learning-guided fuzzy testing method for autonomous driving vehicle-mounted LiDAR firmware of the present invention mainly combines the radar firmware interaction interface specification and uses machine learning technology to guide fuzzy testing, efficiently generating test inputs that conform to the radar firmware interface specification, and efficiently exploring the radar firmware state space to detect vulnerabilities. Attached Figure Description

[0022] Figure 1 This is a system framework diagram of the machine learning-guided firmware fuzzy testing method for autonomous driving vehicle-mounted LiDAR according to an embodiment of the present invention.

[0023] Figure 2 This is the control flow diagram corresponding to the simple code of the radar firmware in the embodiment of the present invention.

[0024] Figure 3 This is a neural network structure diagram for guided fuzzy testing according to an embodiment of the present invention.

[0025] Figure 4 This is a diagram of the structure of a single neuron in a neural network used for guided fuzzy testing in an embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] like Figure 1 As shown, Figure 1 This is a system framework diagram of a machine learning-guided firmware fuzzy testing method for autonomous driving LiDAR in accordance with an embodiment of the present invention. The present invention includes the following steps:

[0028] Step 1) Construction of the data specification model; Step 1) combines static analysis and manual adjustment to construct the radar firmware interface data specification model, specifically including the following steps:

[0029] Step 11) Static Analysis: Perform static analysis on the radar firmware source code, construct a control flow graph, and simultaneously locate and parse the structure definitions used to represent program input in the source code to construct a radar firmware interface data reduction model. This invention targets C / C++ radar firmware with network protocol messages as the interface input type. It analyzes the input protocol message structure, traversing each member variable. If the member variable is of a basic C / C++ data type, it records the variable name, data type, and value range. If the member variable is of an array, it traverses and parses each element of the array. If the member variable is of a structure, it recursively performs static analysis on the structure to extract the data reduction model. Finally, the data reduction model constructed for the radar firmware input message can be represented by the set *(name i ,type i D i |1≤i≤n+ indicates that each element in the set corresponds to a member of a basic C / C++ data type in the protocol message, where name i For member identification, type i For its data type, D i D represents its range of values. i for type i The defined range of values.

[0030] Step 12) Manual Adjustment: The data reduction model obtained from static analysis can be further manually adjusted to meet specific testing needs. For radar firmware, the data reduction model built based on static analysis can be represented by the set *(name) i ,type i D i |1≤i≤n+ indicates that each element in the set corresponds to a member of a basic C / C++ data type in the protocol message, where name i For member identification, type i For its data type, D i D represents its range of values. i for type i The defined range of values. Because of type i The defined value range is the maximum value range for this data type. In some scenarios, such a data range is too broad, so a more precise value range can be set manually. If (name i ,type i D i D represents the constraints of the original data obtained from static analysis. iIf a new value range is set for the user, the final data constraint will be (name). i ,type i D i ′).

[0031] Step 2) Model Training; Step 2) simulates the branching behavior of the tested radar firmware by constructing and training a feedforward neural network, specifically including the following steps:

[0032] Step 21) Assume the input message received by the radar firmware contains n basic data types. First, based on the interface data specification model constructed in Step 1), generate a set of test cases X = *x1,x2,…,x N +, where x i Let x be an n-dimensional vector, corresponding to a test case. i In the process, it is sent to the firmware of the radar under test for execution, and the edge coverage y obtained by this test case is counted. i y i Let y be an i-dimensional vector. ij =1 indicates that this test case covers the j-th edge in the control flow graph of the radar firmware under test. For example, in Figure 2 In the radar firmware example code shown, its control flow graph contains 3 edges. RadarParam is a structure used to represent the input data of the radar firmware, which has 4 member variables. Each input is a four-dimensional vector, and the corresponding label is a three-dimensional vector. For RadarParam input data where all member variables are 1, the input vector is [1,1,1,1]. After it is input into the program under test and executed, it covers the e0 and e1 edges in the program's control flow graph. Therefore, its label vector is [1,1,0]. Thus, ([1,1,1,1],[1,1,0]) is a set of valid training data.

[0033] Step 22) A feedforward neural network consists of numerous neurons, each belonging to a different layer. Each neuron in a layer receives signals from the neurons in the previous layer and outputs a signal to the next layer. The first layer is called the input layer, the last layer is called the output layer, and the other intermediate layers are called hidden layers. Neurons in adjacent layers are fully connected, hence the name fully connected neural network. The fully connected neural network is represented as follows: Figure 3 As shown, the structure of each neuron is as follows: Figure 4As shown, the data flow of the entire feedforward neural network is as follows: each neuron in the first layer receives the input signal, then performs a weighted summation within its own neurons, and outputs the result to the next layer of neurons; the input to the neurons in the second layer comes from the output of the previous layer; and so on. Finally, after calculation by the intermediate neural networks, the result is output to the output layer, yielding the final result.

[0034] In this invention, each neuron in the input layer of the neural network model corresponds to the value of each basic type data member in the input that needs to be mutated. The output layer can be regarded as a bit vector, which indicates the edge coverage of the control flow graph of the radar firmware under test. The i-th neuron in the output layer being 1 indicates that the current test case covers the i-th edge in the control flow graph of the radar firmware under test.

[0035] The numerous (sample input, label) pairs in the seed training set constructed in step 21) are input into the neural network for training, and the trained neural network model is used in subsequent steps.

[0036] Step 3) Test Execution; As shown in Table 1, Algorithm 1 describes the overall process of the test execution phase, and the specific steps include:

[0037]

[0038] Table 1

[0039] 31) Gradient-guided mutation generation, specifically including gradient calculation and mutant generation.

[0040] Gradient calculation: The concept of gradient is based on the concepts of partial derivatives and directional derivatives. A partial derivative, simply put, is a variable that, for a multivariate function, is chosen as the independent variable and all other independent variables are kept constant; only the relationship between the dependent variable and the chosen independent variable is considered.

[0041] By calculating gradients, we can obtain the degree to which the change in the value of each data member in the input data affects the coverage of the edges we want to cover in the program control flow graph. Let the prediction generated by the neural network for the input x be f(x,θ). Take the i-th dimension f of f(x,θ). i (x,θ), which represents the neural network's prediction of whether the current input covers the i-th edge of the control flow graph of the radar firmware under test, then f i The gradient of (x, θ) with respect to the input x can be expressed as: G is the j-th dimension of data. j Then it represents f i (x,θ) with respect to x j The partial derivative of G, therefore G jThis can characterize the impact of the j-th data member in the test case on the coverage of the i-th edge in the control flow graph of the radar firmware under test. j The absolute value of G represents the degree of influence of the j-th data member in the test case on the coverage of the i-th edge in the control flow graph of the radar firmware under test, and determines whether to mutate this data member; j The positive or negative value represents the influence of the j-th data member in the test case on the coverage of the i-th edge in the control flow graph of the radar firmware under test. A positive value means that increasing the value of this data member is more likely to cover this edge, while a negative value means that decreasing the value of this data member is more likely to cover this edge.

[0042] Variant Generation: The variant generation stage primarily utilizes the gradient information calculated in the previous steps, combined with the radar firmware interface data specification extracted in the static analysis stage, to generate a large number of variants conforming to the interface data specification. Having obtained the gradient-related information in the preceding calculations, we know which data members in the seed are worth mutating, and what the best mutation direction is to improve coverage—whether to increase or decrease the value. The next step is to perform the mutation. For each data member in the seed that is determined to mutate, the upper and lower bounds of its value are extracted according to the radar firmware data specification constructed in step 1). The gradient sign corresponding to this data member is examined. If it is positive, it means that increasing the value of this data member is beneficial to improving coverage; therefore, the value of the corresponding member variable in the variant is a random value between the current value and the upper bound. If it is negative, it means that decreasing the value of this data member is beneficial to improving coverage; therefore, the value of the corresponding member variable in the variant is a random value between the lower bound and the current value. After generating a sufficient number of variants using the above method, all of them are saved.

[0043] 32) Test execution and feedback information collection; construct the required operating environment for the radar firmware, drive the radar firmware to run, and send the generated test cases to the radar firmware under test for execution. Monitor the program's running status, collect feedback on the test process, including coverage information and whether crashes have occurred. If the test has not reached the predetermined number of test rounds, add test cases that can improve coverage to the seed set, retrain the model, and start a new round of test execution; otherwise, output the crashes and corresponding test cases as a test report.

[0044] 33) Model retraining: In each round of test execution, test cases that can improve test coverage are added to the seed training set. When each round of test execution is completed, the neural network model is retrained using the new seed set. The new model obtained from the training is used to perform a new round of test execution. In this way, the high-quality test cases discovered during fuzz testing can improve the accuracy of the model in predicting the branch behavior of the tested radar firmware.

[0045] The above description is merely a preferred embodiment of the present invention, but the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention. Any simple modifications, equivalent changes, and alterations made by those skilled in the art to the above embodiments based on the technical essence of the present invention without departing from the principles and spirit of the present invention should be included within the protection scope of the present invention.

Claims

1. A machine learning-guided firmware fuzzy testing method for autonomous driving vehicle-mounted LiDAR, characterized in that, The method includes the following steps: Step 1) Construction of the firmware interface data specification model for autonomous driving LiDAR; static analysis is performed based on the source code to obtain the LiDAR firmware interface data specification model; the LiDAR firmware interface data specification model obtained from the static analysis is manually adjusted to obtain the final LiDAR firmware interface data specification model; Step 1) specifically includes the following steps: Step 11) Extraction of radar firmware interface data specifications based on static analysis; by performing static analysis on the source code of autonomous vehicle-mounted LiDAR firmware, a control flow graph is constructed, and the data structure definitions used to represent program input in the code are identified, and the data type and value range of each data member are automatically extracted. Step 12) Manual adjustment of data specification; providing a human-computer interaction interface so that users can freely adjust the input data specification obtained from static analysis in step 11) to obtain the final radar firmware input data specification model; Step 2) Model Training: Based on the final radar firmware input interface data reduction model obtained in Step 1), valid initialization seed test cases are generated. These test cases are used as training data to train the neural network model. The input of the neural network model is a byte sequence of a single test case, and the output is a bit vector representing the program edge coverage. Step 2) specifically includes the following steps: Step 21) Training set generation; Based on the final radar firmware input interface data reduction model obtained in step 12), generate valid initialization seed test cases, send the test cases to the radar firmware for execution, and record the edge coverage information of the radar system program space fed back by the execution of the test cases. Step 22) Model training: Based on the collected valid test cases and coverage feedback information, a machine learning algorithm is used for training and learning to generate a neural network model for predicting the relationship between test inputs and edge coverage. Step 3) Test execution; Based on the neural network model constructed in Step 2), and combined with the radar firmware interface data specification model constructed in Step 1), guide the generation and execution of test cases that conform to the radar firmware interface data specification and can fully explore the internal state space of the radar firmware; Step 3) specifically includes the following steps: Step 31) Gradient-guided mutation generation: Based on the seed set constructed in Step 2), for each test case, input it into the neural network model, calculate the gradient information of the model output layer with respect to the input layer, determine the bytes in a test input that may improve the coverage of the program under test and the mutation direction of the byte value according to the gradient information; combine the data reduction model output in Step 1) to generate test cases that conform to the radar firmware data specification, and send the test cases to the radar firmware for execution; Step 32) Test execution and feedback information collection; construct the required operating environment for the radar firmware, drive the radar firmware to run, and send the generated variants to the radar firmware under test for execution; collect the coverage information and whether a crash was triggered during the test process. If the test has not reached the predetermined number of test rounds, add test cases that can improve coverage to the seed set, retrain the model, and start a new round of test execution; if the test has reached the predetermined number of test rounds, output the crashes triggered and the corresponding test cases as a test report. Step 33) Model retraining: Retrain the model using the latest seed set, and then perform a new round of testing based on the retrained model.

Images