A verification method for the steering angle safety of an unmanned driving system based on convex optimization

Abstract

The present invention discloses a verification method for the steering angle safety of an unmanned driving system based on convex optimization, belonging to the field of verifying the safety of unmanned driving systems. The convex optimization technology in fault-safe trajectory planning is used to construct a safe steering angle interval, converting the problem of judging the predicted steering angle into a classifiable problem similar to image processing, and then expanding the deep neural network verification tool DLV to execute the verification algorithm for the safety of the steering angle. This technology is easy to integrate into existing unmanned driving systems; it is computationally efficient and can find adversarial counterexamples within a few seconds; compared with existing work, it has a higher success rate in finding adversarial counterexamples. In addition, when dealing with the steering angle classification problem, it is more accurate and reliable than the neuron coverage and relaxation relationship in the previous work SDLV.

Description

Technical Field

[0001] The present invention belongs to the field of verifying the safety of unmanned driving systems, and relates to a method for verifying the safety of the steering angle of an unmanned driving system based on convex optimization. Specifically, it involves using convex optimization techniques in fail-safe trajectories to judge the safety of the steering angle of an unmanned driving system, and expanding the verification algorithm for the safety of the steering angle by a deep neural network verification tool (DLV). Background Art

[0002] Advances in machine learning (ML) technology have promoted the development of driverless cars. Existing experimental results show that a well-trained driverless car system can travel millions of miles without any human intervention by using sensors such as cameras, radars, and lidar to obtain input information. Many well-known car manufacturers are manufacturing and actively testing driverless cars. The development of driverless cars is becoming increasingly prosperous, and this trend is likely to continue and intensify. However, when a system based on a deep neural network (DNN) is applied to a system with high safety requirements such as a driverless car, the predicted output is unstable when the input image changes slightly, and it may make incorrect responses. Accidents involving driverless cars have been reported many times, and some of them even caused fatal collisions. This will obviously bring potential safety problems to applications such as driverless driving and requires formal verification techniques to verify the correctness of its decisions. Therefore, there is an urgent need for a formal method to provide safety guarantees for driverless cars. However, due to the input-output space (i.e., all possible combinations of inputs and outputs) being too large to be thoroughly explored. On the other hand, using testing methods to provide guarantees for driverless cars, the required scenarios are difficult to simulate and the testing time is long. Existing research shows that driverless cars need to be tested for 440 million kilometers to prove that they have better performance than humans. This means that a fleet of 100 cars has been test-driven continuously for 24 hours a day for 12.5 years.

[0003] Existing methods usually use test cases to detect the safety of predicted steering angles in autonomous driving. For example, a system test tool called DeepTest is used to automatically detect incorrect behaviors of DNN-driven autonomous vehicles, which may lead to fatal collisions. This method uses nine different image transformations to automatically generate test cases and then discovers thousands of incorrect steering behaviors. Image transformations include changing brightness, changing contrast, translation, scaling, horizontal shearing, rotation, blurring, fog effect, and rain effect. However, despite significant progress, real-world driving conditions are far more than the above nine transformations. Another effective method is the simulation-based method. However, simulating a fully autonomous vehicle must travel hundreds of millions of miles, sometimes even hundreds of billions of miles, to prove its reliability in avoiding casualties, which takes decades to complete. Therefore, although case-based testing and simulation are often used to check the performance of autonomous systems, they do not provide sufficient assurance. This is especially true for safety-critical fields such as autonomous driving, where unsafe events are rare and difficult to describe. Summary of the Invention

[0004] The present invention aims to overcome the deficiencies of the prior art and provides a method for verifying the safety of the steering angle of an autonomous driving system based on convex optimization technology. The method combines the convex optimization technology in the fail-safe trajectory and the deep learning verification tool (DLV) (Huang X, Kwiatkowska M, Wang S, et al. Safety verification of deep neural networks [C] / / International conference on computer aided verification. Springer, Cham, 2017: 3-29.) to perform verification. DLV is a framework for automatically verifying the safety of image classification neural networks. The DLV is extended by convex optimization technology to solve the problem of judging the predicted steering angle, thereby realizing the verification of the safety of the steering angle of the autonomous driving system.

[0005] The specific technical solution of the present invention is as follows:

[0006] A method for verifying the safety of the steering angle of an autonomous driving system based on convex optimization includes the following steps:

[0007] 1) Use the convex optimization method in the fail-safe trajectory to solve the steering angle that can avoid collisions, that is, establish the collision-avoiding steering angle;

[0008] 2) Construct the interval of the safe steering angle: construct the interval of the safe steering angle with the collision-avoiding steering angle solved by convex optimization and the steering angle originally predicted by the autonomous driving system;

[0009] 3) Determine the neighborhood parameters and perturbation operation sets in the deep learning verification tool DLV, and search for adversarial counterexamples layer by layer;

[0010] In step 1), for the establishment of the collision avoidance steering angle, in the fail-safe trajectory planning of driverless vehicles, the vehicle motion planning is decoupled into longitudinal and lateral motions, and then the convex optimization method is used for solution. Let t and t h represent time and time interval respectively, and the specific steps are as follows:

[0011] 1-1) Plan the longitudinal trajectory, and obtain a comfortable longitudinal trajectory by weighting and to penalize high acceleration and jitter. The longitudinal trajectory is represented by the quadratic cost function J lon , and the formula is as follows:

[0012]

[0013] In the formula, the longitudinal motion of the vehicle is x lon (t) = (s, v, a, j) T , where s is the longitudinal position, v is the speed, a is the acceleration, and the jitter along the path is denoted as j, represents the i-th element of the function x lon (t);

[0014] Solve the longitudinal trajectory using the convex optimization method.

[0015] 1-2) Combine the longitudinal position s in the solution result of step 1-1), and obtain a comfortable lateral trajectory by weighting w d , w θ , w k and to penalize high curvature. The lateral trajectory is represented by the quadratic cost function J lat , and the formula is as follows:

[0016]

[0017] In the formula, the lateral motion of the vehicle is d is the lateral position, θ is the deviation, κ is the curvature, is the change in curvature, represents the i-th element of the function x lat (t);

[0018] Predict the maximum and minimum lateral position constraints through the obstacle reachable set, where the reachable set represents the set of all feasible states of the obstacle within a period of time; solve the lateral trajectory using the convex optimization method under the solved maximum and minimum lateral position constraints to obtain the steering angle that can avoid collisions.

[0019] In step 2), the construction of the interval of the safe steering angle specifically includes the following steps:

[0020] 2-1) For a trained DNN-based driverless system, input a given driving scenario image x, and the system can output a corresponding predicted steering angle, which is recorded as the original predicted steering angle.

[0021] 2-2) Use the steering angle in the lateral trajectory solved in step 1) to form an interval with the original predicted steering angle as the range of the safe steering angle.

[0022] In step 3), expand DLV to verify the safety of the steering angle of the DNN-based driverless system.

[0023] The verification of the present invention is based on the method of searching for adversarial counterexamples. For any DNN-based driverless system N, a given input driving scenario image x, a neighborhood, and a perturbation operation, the perturbation operation is implemented during the process of x propagating layer by layer in N. Once the predicted steering angle output by the system for the perturbed driving scenario image is not within the range of the safe steering angle, an adversarial counterexample is reported. This method ensures that if there is an adversarial misclassification (i.e., an incorrect steering decision), it can be successfully searched, so safety verification can be achieved. If no misclassification is found in all DNN layers of the driverless network, the network can be said to be stable or reliable; and the discovered adversarial counterexamples can be used to improve the network.

[0024] The layer-by-layer search for adversarial counterexamples specifically includes the following steps:

[0025] 3-1) Determine the neighborhood parameter η in the deep learning verification tool DLV k and the set of perturbation operations Δ;

[0026] Each layer L of the neural network k is associated with an n k dimensional vector space where each dimension corresponds to a neuron. For an input driving scenario image x, the activation of a DNN-based driverless system at layer k is denoted as α x，k , k ∈ {1,..., n}, and α x，0 = x. α x，k (p) represents the activation value of the input x at the neuron p ∈ P k at the k-th layer.

[0027] For a given input driving scenario image \(x\), it can be regarded as a point in a high-dimensional space. Suppose there is one or an infinite number of neighborhoods \(\eta\) around this point, and the predicted steering angles of all points within the neighborhood must have the same class as the predicted steering angle of \(x\). The neighborhood mentioned above is specified by the user and can be represented by a small diameter or by a set of all points with certain identical features.

[0028] Starting from a certain layer \(k\) of the neural network, the neighborhood \(\eta\) k (\(\alpha\) x，k ) is a subset of the dimensions \(dims\) pre-selected from the neurons corresponding to the activation values with the largest difference from the activation average of the current layer \(L\) k . Denote the activation average of the current layer \(L\) k by \(avg\) k . \(dims\) k (\(\eta\) k (\(\alpha\) k )) are the first batch of dimensions where \(|\alpha\) x，k (p) - avg| takes the maximum value among all dimensions. x，k

[0029]

[0030] In the formula, \(s\) p represents a small span, and \(m\) p represents the number of such spans.

[0031] Next, assume there is a set of perturbation operations \(\Delta\), which specify the modifications to the driving scenario image, that is, implementing perturbation operations on the driving scenario image. Under these modifications, the classification of the predicted steering angle within the \(\eta\) region should remain unchanged. The perturbation operations mentioned above can represent camera inaccuracy, changes in camera angles, or replacement of a certain feature.

[0032] Let \(d\) be a function that maps from \(dims\) k (\(\eta\) k (\(\alpha\) x，k )) to \(\{-1, 0, 1\}\). Each perturbation operation changes a subset of dimensions by adding or subtracting the width \(s\) p according to the direction given in \(d\), that is,

[0033]

[0034] In the formula, the set \(\Delta\) k is the set of all such perturbation operations at the \(k\)th layer .

[0035] 3 - 2) Search for adversarial counterexamples

[0036] The safety definition of the steering angle for a DNN-based driverless system is as follows:

[0037] If applying a perturbation operation to the input driving scene image x does not cause a classification change in the steering angle within the neighborhood η, the DNN network is considered safe with respect to x and η regarding the set of perturbation operations Δ.

[0038] Among them, the classification judgment method is: if the predicted steering angle after perturbation falls within the interval of the safe steering angle solved in step 2), the steering angle after perturbation and the original predicted steering angle are considered to be of the same classification; otherwise, the two steering angles are considered not to be of the same classification.

[0039] Use discretization to achieve a finite exhaustive search for adversarial misclassification of the steering angle within the high-dimensional η region. The safety analysis propagates layer by layer, mapping the neighborhood and perturbation operations to deeper layers. This propagation is complete under the guarantee of operation minimization. If there is misclassification, the verification framework can guarantee to find the misclassification. Since the verification is simplified to searching for adversarial examples, safety verification can be achieved, that is, if no misclassification is found in all layers, the safety verification of the steering angle can be achieved; if an adversarial counterexample is searched, it can be used to improve the network. The specific steps are as follows:

[0040] For a given neural network N, an input x, and a set of perturbation operations Δ k , if a complete tree is generated starting from α x，k , each node of the tree has the same steering angle classification as α x，k , and the finite number of hyperrectangles formed by adjacent nodes can cover the region η k , then N is safe for the input x, the region η k and the perturbation operations Δ k , denoted as N, η k ,

[0041] If the search result of this layer satisfies N, η k , then report that N is safe regarding η k and Δ, and continue the search for the next layer; if not satisfied, report an adversarial counterexample and stop the search.

[0042] The beneficial effects of the present invention are:

[0043] The present invention combines the convex optimization technology in the fail-safe trajectory and the deep learning verification tool (DLV) to perform the safety verification of the steering angle of the driverless vehicle. By using the convex optimization technology to expand the DLV verification framework to solve the judgment problem of the predicted steering angle, the safety verification of the steering angle of the driverless vehicle is realized, and it has the following characteristics:

[0044] 1. The verification method is easy to integrate into existing unmanned driving systems;

[0045] 2. As long as there are adversarial counterexamples in the given area and set of operations, adversarial misclassifications, i.e., incorrect steering decisions, can be successfully found;

[0046] 3. Implement safety verification, i.e., if no misclassifications are found in all DNN layers, in this case, the network can be said to make stable or reliable steering decisions;

[0047] 4. The searched adversarial counterexamples can be used to improve the network.

[0048] The present invention uses convex optimization technology in fail-safe trajectories to solve the steering angle for collision avoidance, making the judgment of the safety of the predicted steering angle more accurate; the proposed method has good scalability and is easy to integrate into existing unmanned driving systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is the network structure of an end-to-end unmanned driving system, which is an unmanned driving system based on DNN, inputs driving scene images, and outputs steering angles.

[0050] Figure 2 is the implementation flowchart for verifying the safety of the steering angle of the unmanned driving system of the present invention.

[0051] Figure 3 are the adversarial counterexamples searched for single-path and multi-path. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] The present invention proposes a method for verifying the safety of the steering angle of an unmanned driving system based on convex optimization technology, which is described in detail as follows in combination with the drawings and embodiments:

[0053] The present invention takes the NVIDIA end-to-end unmanned driving system as an example to execute the method proposed by the present invention to verify the safety of the steering angle, and the system structure is as Figure 1 shown. The network consists of 9 layers, including a normalization layer, 5 convolutional layers, and 3 fully connected layers. The size of the network input image is 135×240 and has three channels. The process of verifying the safety of the steering angle of the unmanned driving system of the present invention is as Figure 2 shown, and the verification method includes the following steps;

[0054] 1) Training of the end-to-end unmanned driving system;

[0055] 1-1) The end-to-end unmanned driving system is trained on the driving set for more than six hours, and the neural network obtains more than 1 million parameters;

[0056] 1-2) Use the trained neural network and the test image as inputs to perform the safety verification of the following driverless system.

[0057] 2) Use the method of convex optimization to solve for the steering angle that can avoid collisions: Decouple the motion planning of the driverless vehicle into longitudinal and lateral motions, and then use the method of convex optimization to solve for the planned path to obtain a safe steering angle. Here, t and t h represent time and time interval.

[0058] 2-1) Plan the longitudinal trajectory, and obtain a comfortable longitudinal trajectory by weighting and penalizing high acceleration and jitter. The longitudinal trajectory is represented by the quadratic cost function J lon as follows:

[0059]

[0060] In the formula, the longitudinal motion of the vehicle is x lon (t) = (s, v, a, j) T , where s is the longitudinal position, v is the speed, a is the acceleration, and the jitter along the path is denoted as j, represents the i-th element of the function x lon (t).

[0061] Use the method of convex optimization to solve for the longitudinal trajectory.

[0062] 2-2) Combine the longitudinal position s in the longitudinal trajectory obtained in step 2-1), and obtain a comfortable lateral trajectory by weighting w d , w θ , w κ and penalizing high curvature. The lateral trajectory is represented by the quadratic cost function J lat as follows:

[0063]

[0064] In the formula, the lateral motion of the vehicle is where d is the lateral position, θ is the deviation, K is the curvature, is the change in curvature, represents the i-th element of the function x lat (t).

[0065] Predict the maximum and minimum lateral position constraints through the reachable set of obstacles, where the reachable set represents the set of all feasible states of the obstacles within a period of time; Solve for the lateral trajectory using the method of convex optimization under the maximum and minimum lateral position constraints obtained, and obtain the steering angle that can avoid collisions.

[0066] 3) Construct the interval of the safe steering angle: For the input test driving scenario image x, construct a corresponding safe steering angle interval with the safe steering angle obtained by convex optimization for collision avoidance and the steering angle predicted by the trained driverless system for the original image.

[0067] 3-1) For a trained driverless system, input a given driving scenario image x, and the system can output a corresponding predicted steering angle. Record this steering angle as the original predicted steering angle.

[0068] 3-2) Use the planning path solved in step 2) to form an interval with the steering angle calculated in the trajectory and the predicted steering angle as the safe steering angle range. If the perturbed predicted steering angle falls within this safe steering angle interval, the perturbed steering angle and the original predicted steering angle are considered to be of the same classification; otherwise, the two steering angles are considered to be of different classifications.

[0069] 4) Expand DLV and verify the safety of the steering angle of the driverless system using the method based on searching for adversarial examples. For the given neighborhood and operation, the technical solution of the present invention ensures that if there is an adversarial misclassification (i.e., an incorrect steering decision), it can be successfully searched, so safety verification can be achieved. If no misclassification is found in all DNN layers of the network, the network can be said to be stable or reliable; and the discovered adversarial examples can be used to improve the network.

[0070] 4-1) Determine the neighborhood parameter η in DLV k and the operation set Δ;

[0071] Each layer L of the neural network k is associated with an n k -dimensional vector space where each dimension corresponds to a neuron. For an input driving scenario image x, the activation of a DNN-based driverless system at layer k is denoted as α x，k , k ∈ {1,..., n}, and α x，0 = x. α x，k (p) represents the activation value of the input x at the neuron p ∈ P k at layer k.

[0072] For a given input driving scenario image x (which can be regarded as a point in a high-dimensional space), assume that there is one or an infinite number of regions η around this point, and the predicted steering angles of all points within this region must have the same class as the predicted steering angle of x. This region is specified by the user and can be represented by a small diameter or by the set of all points with certain identical features.

[0073] Starting from a certain layer k of the neural network, the neighborhood ηk (α x，k ) is a subset of the dimensions dims selected in advance from the neurons corresponding to the activation values with the largest difference from the activation average of the current layer L k . Denote the activation average of the current layer L by avg k . dims k (η k (α k )) is the first batch of dimensions where |α k (p) - avg| takes the maximum value among all dimensions. x，k )) is the first batch of dimensions where |α x，k (p) - avg| takes the maximum value among all dimensions.

[0074]

[0075] Here, s p represents a small span, and m p represents the number of such spans.

[0076] Next, assume there is a set of operations Δ that specify modifications to the image. Under such modifications, the classification of the predicted steering angle within the η region should remain unchanged. Such operations can represent camera imprecision, a change in the camera angle, or the replacement of a certain feature.

[0077] Let d be a function that maps from dims k (η k (α x，k )) to {-1, 0, 1}. Each operation changes a subset of the dimensions by adding or subtracting the width s p according to the direction given in d, that is

[0078]

[0079] the set Δ k is the set of all such operations.

[0080] 4 - 2) Search for adversarial counterexamples

[0081] We give the following definition for the safety of the steering angle of a DNN - based unmanned driving system:

[0082] If applying an operation to the input driving scene image x does not cause a change in the classification of the steering angle within the neighborhood η, the network is considered safe with respect to x and η for the set of operations Δ.

[0083] Use discretization to achieve a finite exhaustive search for adversarial misclassification of steering angles within a high-dimensional η region. The safety analysis propagates layer by layer, mapping the neighborhood and operations to deeper layers. This propagation is complete under the guarantee of operation minimization. If there is a misclassification, the verification framework can guarantee to find the misclassification. Since the verification is reduced to searching for adversarial examples, we can achieve safety verification, that is, if no misclassification is found in all layers, the safety of the steering angle can be verified; if an adversarial counterexample is found, it can be used to improve the network.

[0084] For a given neural network N, an input x, and a set of operations Δ k , if a complete tree is generated starting from α x，k , each node of the tree has the same steering angle classification as α x，k , and the finite number of hyperrectangles formed by adjacent nodes can cover the region η k , then we say that N is safe for the input x with respect to the region η k and the operations Δ k , denoted as N, η k .

[0085] If the search result of this layer satisfies N, η k , then report that N is safe with respect to η k and Δ, and continue the search for the next layer; if not satisfied, report an adversarial counterexample and stop the search.

[0086] In this embodiment, two methods, single-path search and multi-path search, are respectively used to search for adversarial counterexamples. If the points within the region divided according to the features are checked in a pre-specified order, this method is called single-path search. If the points within the region divided according to the features are checked by exhaustively searching all possible orders, this method is called multi-path search. As Figure 3As shown, in each group of figures in the experiment, the original image is on the left and the perturbed image reported when verifying the discovery of adversarial counterexamples is on the right. Among them, figure (a) is the adversarial counterexample searched by the single-path method, and figure (b) is the adversarial counterexample searched by the multi-path method. Randomly select 100 images from the test set, and compare the proposed verification method with two existing methods for searching adversarial counterexamples: the Deep Neural Network (DNN)-driven Automated Testing Method for Autonomous Vehicles (DeepTest) (Tian Y, Pei K, Jana S, et al. DeepTest: Automated testing of deep-neural-network-driven autonomous cars [C] / / Proceedings of the 40th international conference on software engineering. 2018: 303-314.) and the Safety Verification Method for Steering Angles of Autonomous Vehicles (SDLV) (Wu H, Lv D, Cui T, et al. SDLV: verification of steering angle safety for self-driving cars [J]. Formal Aspects of Computing, 2021, 33(3): 325-341.). DeepTest is a method for automatically detecting incorrect steering angles of DNN-driven autonomous vehicles. It applies image transformation to automatically generate test cases to detect incorrect steering behaviors, while SDLV uses neuron coverage and relaxation conditions as the criteria for steering angle classification and explores a certain proportion of dimensions in the feature space of the input or hidden layer in a formal way. The data comparison table obtained from the comparative experiment is shown in Table 1. There are three data in the table, including the L 1 and L 2 distances between the perturbed image and the original image, and the success rate of finding adversarial counterexamples.

[0087] A comparison between the present invention and the two existing methods for searching adversarial counterexamples, DeepTest and SDLV, is shown in Table 1. It can be seen from the success rate of finding adversarial counterexamples that the method of the present invention has a higher success rate than the existing methods DeepTest and SDLV. When the distance between the adversarial counterexample and the original image is smaller, the transfer rate may be lower, that is, on another model trained on the same data set, misclassification may be more difficult to detect. From the L 1 distance and the L 2 distance, it can be found that when the dimension dim of the perturbation operation is 300, the method of the present invention and SDLV are superior to DeepTest.

[0088] Table 1 Comparison with two existing methods for searching adversarial counterexamples, DeepTest and SDLV

[0089]

Claims

1. A verification method for the steering angle safety of an unmanned driving system based on convex optimization, characterized in that, Including the following steps: 1) Use the convex optimization method in the fail-safe trajectory to solve the steering angle that can avoid collisions, that is, establish the collision-avoidance steering angle; 2) Construct the interval of the safe steering angle: construct the interval of the safe steering angle with the collision-avoidance steering angle solved by convex optimization and the steering angle originally predicted by the unmanned driving system; 3) Determine the neighborhood parameters and perturbation operation set in the deep learning verification tool DLV, and search for adversarial counterexamples layer by layer; In step 1), for the establishment of the collision avoidance steering angle, in the fail-safe trajectory planning of driverless vehicles, the vehicle motion planning is decoupled into longitudinal and lateral motions, and then a convex optimization method is used for solution. Let t and t h represent time and time interval respectively, and the specific steps are as follows: 1-1) Plan the longitudinal trajectory, and obtain a comfortable longitudinal trajectory by weighting and penalizing high acceleration and jerk. The longitudinal trajectory is represented by the quadratic cost function J lon as follows: wherein, the longitudinal motion of the vehicle is x lon (t) = (s, v, a, j) T , where s is the longitudinal position, v is the speed, a is the acceleration, and the jerk along the path is denoted as j represents the i-th element of the function x lon (t); Use the convex optimization method to solve the longitudinal trajectory; (1-2) Combine the longitudinal position s in the solution result of step (1-1), and obtain a comfortable lateral trajectory by means of weights w d , w θ , w κ and penalize high curvatures, where the lateral trajectory is represented by the quadratic cost function J lat and is given by the formula: In the formula, the lateral movement of the vehicle is d is the lateral position, θ is the deviation, and κ is the curvature, is the change in curvature, represents the function x lat (t) of the i-th element; Predict the maximum and minimum lateral position constraints through the reachable set of obstacles, where the reachable set represents the set of all feasible states of the obstacle within a period of time; use the convex optimization method to solve the lateral trajectory under the solved maximum and minimum lateral position constraints to obtain the steering angle that can avoid collisions; In step 2), the construction of the interval of the safe steering angle specifically includes the following steps: 2-1) For a trained DNN-based unmanned driving system, input a given driving scenario image x, and the system can output a corresponding predicted steering angle, and record this steering angle as the original predicted steering angle; 2-2) Use the steering angle in the lateral trajectory solved in step 1) to form an interval with the original predicted steering angle as the range of the safe steering angle; In step 3), expand DLV to verify the safety of the steering angle of the DNN-based unmanned driving system: The verification is based on the method of searching for adversarial counterexamples. For any DNN-based unmanned driving system N, a given input driving scenario image x, a neighborhood, and perturbation operations, perturbation operations are implemented during the process of x propagating layer by layer in N. Once the predicted steering angle output by the system for the perturbed driving scenario image is not within the range of the safe steering angle, an adversarial counterexample is reported; The specific steps of the layer-by-layer search for adversarial counterexamples include the following: 3-1) Determine the neighborhood parameter η in the deep learning verification tool DLV k and the set of perturbation operations Δ; Each layer L of the neural network k is associated with an n k -dimensional vector space where each dimension corresponds to a neuron; for an input driving scene image x, the activation of a DNN-based autonomous driving system at layer k is denoted as α x,k , k ∈ {1,..., n}, and α x,0 = x; α x,k (p) represents the activation value of the input x at neuron p ∈ P k at the k-th layer; Regard a given input driving scenario image x as a point in a high-dimensional space. Assume that there is one or an infinite number of neighborhoods η around this point. The predicted steering angles of all points within the neighborhood must have the same class as the predicted steering angle of x; the neighborhood is specified by the user and is represented by a small diameter or by the set of all points with certain identical features; Starting from a certain layer k of the neural network, the neighborhood η k (α x,k ) is a subset of the dimensions dims k selected in advance from the neurons corresponding to the activation values with the largest difference from the average activation of the current layer L k ; let avg k represent the average activation of the current layer L k , and dims k (η k (α x,k )) are the first batch of dimensions where |α x,k (p) - avg| takes the maximum value among all dimensions; In the formula, s p represents the span, in m p and represents the number of such spans; Next, assume that there is a set of perturbation operations Δ. These perturbation operations specify the modifications to the driving scenario image, that is, perturbation operations are implemented on the driving scenario image. Under these modifications, the classification of the predicted steering angle within the η region should remain unchanged; the perturbation operations include camera inaccuracy, change of camera angle, or replacement of a certain feature; Let d be a function that maps from dims k (η k (α x,k )) to {-1, 0, 1}; each perturbation operation changes a subset of the dimensions by adding or subtracting the width s p in accordance with the direction given in d, i.e., where the set Δ k is the set of all such perturbation operations at the k-th layer; 3-2) Search for adversarial counterexamples The safety of the steering angle of the DNN-based unmanned driving system is defined as follows: If applying perturbation operations to the input driving scenario image x does not cause a change in the classification of the steering angle within the neighborhood η, then the DNN network is considered safe with respect to x and η with respect to the set of perturbation operations Δ; Among them, the judgment method of classification is: if the predicted steering angle after perturbation falls within the interval of the safe steering angle solved in step 2), then the steering angle after perturbation and the original predicted steering angle are considered to be of the same classification; otherwise, the two steering angles are considered to be of different classifications; Use discretization to implement a finite exhaustive search for adversarial misclassification of steering angles within a high-dimensional η region. The specific steps are as follows: For a given neural network N, an input x, and a set of perturbation operations Δ k , if a complete tree is generated starting from α x,k , each node of the tree has the same steering angle classification as α x,k , and the finite number of hyperrectangles formed by adjacent nodes can cover the region η k , then N is safe for the input x, the region η k , and the perturbation operations Δ k , denoted as If the search result of this layer satisfies then report N with respect to η k and Δ are safe and continue with the next layer of search; if not, report an adversarial counterexample and stop the search.

Images