A method and device for fuzz testing of an autonomous driving perception system and a storage medium

Abstract

The application discloses a kind of fuzzy testing method, device and storage medium of automatic driving perception system, method includes: utilizing initial seed to construct seed queue;Current seed is selected from seed queue, and current seed is generated test seed by mutation operation;Test seed is tested prediction, and the test seed of output prediction result is failure test to failure set;The test coverage of current seed is obtained by using guide index processing after joining test seed, and seed queue is updated according to test coverage improvement test seed;Wherein, guide index includes space coverage and semantic coverage;The number of iterations is added 1, and the step of selecting current seed from seed queue is returned to execution, until the number of iterations reaches the maximum iteration number of pre-set, and output failure set.The application can efficiently realize the fuzzy testing of automatic driving perception system, and can be widely applied in data processing technical field.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a fuzzy testing method, apparatus and storage medium for an autonomous driving perception system. Background Technology

[0002] With the rapid development of artificial intelligence and sensor technology, autonomous vehicles have achieved tremendous success. As a typical safety-critical intelligent software, autonomous driving systems utilize sensors such as LiDAR (Light Detection and Ranging), and cameras to capture information about the surrounding environment as input, automatically completing various driving tasks through various functional components. Several major automakers and organizations, including Tesla, Waymo, Uber, and Baidu, are manufacturing and actively testing these vehicles. Recent results indicate that autonomous vehicles have already driven millions of miles without any human intervention.

[0003] However, when such intelligent systems are deployed in real-world environments, they often exhibit incorrect and unexpected extreme behaviors, potentially leading to fatal collisions. Many such real-world cases have been reported. For example, some autonomous vehicles collided directly with white trucks while driving. The cause of the accident was that the autonomous driving system failed to distinguish a white truck from the bright sky, primarily because the truck was too tall. Since these vehicles rely on various sensors (such as cameras and LiDAR) to perform driving tasks, the potential input space is very large, leading to numerous extreme situations. There is an urgent need to explore better methods for effectively testing autonomous driving software. In autonomous driving tasks, the LiDAR perception system is mainly responsible for environmental perception and obstacle detection. In adverse weather conditions (such as rain, fog, and snow), the detection accuracy of LiDAR is significantly reduced, and sparse point clouds can cause target detection to fail. Simultaneously, in certain driving scenarios, the combination of specific participants (i.e., the integration of specific semantic information in the scene) can also cause LiDAR recognition failure due to defects in the perception software. As one of the core components of autonomous vehicles, the LiDAR perception system is crucial for driving safety.

[0004] Based on the above, there is currently a lack of effective and comprehensive testing methods for LiDAR perception systems. There is an urgent need to explore a better way to effectively test autonomous driving systems, thereby improving their robustness. Summary of the Invention

[0005] The main objective of this invention is to provide a fuzzy testing method, apparatus, and storage medium for an autonomous driving perception system, in order to solve at least one problem in the prior art. This invention can efficiently realize fuzzy testing of an autonomous driving perception system.

[0006] To achieve the above objectives, one aspect of this invention proposes a fuzzy testing method for an autonomous driving perception system, the method comprising:

[0007] Obtain an initial seed and construct a seed queue using the initial seed; the initial seed is obtained based on a pre-defined dataset in the field of autonomous driving, and the seeds in the seed queue represent the 3D point cloud generated by the LiDAR system;

[0008] Select a preset number of current seeds from the seed queue, and perform mutation operations on the current seeds to generate test seeds;

[0009] The test seeds are input into the deep neural network of the preset autonomous driving perception system for test prediction, and the test seeds with the prediction result of failure are added to the failure set. The test coverage of the current seed and the test coverage after adding the test seed are obtained by using the guiding index. If the test coverage after adding the test seed is higher than the test coverage of the current seed, the test seed is added to the seed queue. The guiding index includes spatial coverage and semantic coverage.

[0010] Increment the iteration count by 1, return to the step of selecting a preset number of current seeds from the seed queue, until the iteration count reaches the preset maximum iteration count, and output the failure set; where the initial iteration count is 0.

[0011] In some embodiments, selecting a preset number of current seeds from the seed queue includes the following steps:

[0012] A seed selection strategy is used to randomly select a preset number of seeds from the seed queue as the current seeds.

[0013] In some embodiments, after updating the seed queue according to the test seed, the method further includes the following steps:

[0014] Update the selection probability of each seed in the updated seed queue;

[0015] The updated selection probability is expressed as follows:

[0016] P(s) = 1 / g(ori(s)) 2

[0017] In the formula, ori(s) represents the initial seed corresponding to the seed scene, and g(ori(s)) represents the number of test seeds generated by mutation of the initial seed.

[0018] In some embodiments, a mutation operation is performed on the current seed to generate a test seed, including the following steps:

[0019] For each current seed, a mutation operator is randomly selected and the mutation operation is executed sequentially until the number of mutations exceeds the preset maximum number of attempts. The test seed corresponding to each current seed is obtained based on the execution results of all mutation operations.

[0020] Among them, mutation operators include object transformation and the addition of severe weather conditions.

[0021] In some embodiments, after the step of generating a test seed by performing a mutation operation on the current seed, the method further includes the following steps:

[0022] Calculate the FRD score for each test seed, and discard test seeds whose FRD scores are less than a preset score threshold;

[0023] The expression for the FRD score is:

[0024] FRD(X,Y)=||μ X -μ Y || 2 +Tr(∑ X +∑ Y -2(∑ X ∑ Y ) 1 / 2 )

[0025] In the formula, X represents the initial seed, Y represents the test seed corresponding to X, FRD(X,Y) represents the FRD score corresponding to Y; μ represents the mean vector of data activation, ‖·‖ represents the L2 norm of the vector; ∑ represents the covariance matrix of data activation, and Tr represents the trace of the covariance matrix.

[0026] In some embodiments, when the test coverage rate is spatial coverage, the test coverage rate of the current seed and the test coverage rate after adding the test seed are obtained using the guiding metric, including the following steps:

[0027] The spatial coverage is obtained by summing and averaging the ratio of the sum of the areas of the objects to be detected in each 3D point cloud contained in the seed to the total area of ​​the space to be detected by the radar.

[0028] The expression for spatial coverage is:

[0029]

[0030] In the formula, SPC(T) represents spatial coverage; Avg represents the average treatment. The sum of the areas of the objects to be detected in the i-th 3D point cloud of the seed; m represents the number of initial seeds; This represents the total area of ​​the radar-sensed space in the i-th 3D point cloud of the seed.

[0031] In some embodiments, when the test coverage rate is semantic coverage, the test coverage rate of the current seed and the test coverage rate after adding the test seed are obtained using the guiding metric, including the following steps:

[0032] The semantic coverage is obtained by the ratio of the number of equivalence classes generated by the fuzzy test corresponding to the seed to the total number of all possible semantically perceptible equivalence classes. The steps for generating equivalence classes include: generating a corresponding scene graph for each seed, the scene graph being a directed graph; performing simplification operations on the scene graph to generate an abstract scene graph; clustering the abstract scene graph to generate equivalence classes, where an equivalence class is a set of abstract scene graphs with the same structure and connection relationship; and finally, calculating the number of equivalence classes.

[0033] The expression for semantic coverage is:

[0034]

[0035] In the formula, |ASG S (T)| represents the number of equivalence classes generated by the fuzz test. This represents the total number of equivalence classes of all possible perceptual semantics generated by the fuzz test.

[0036] To achieve the above objectives, another aspect of the present invention provides a fuzzy testing device for an autonomous driving perception system, the device comprising:

[0037] The first module is used to obtain the initial seed and construct a seed queue using the initial seed. The initial seed is obtained based on a pre-set dataset in the field of autonomous driving, and the seeds in the seed queue represent the 3D point cloud generated by the LiDAR system.

[0038] The second module is used to select a preset number of current seeds from the seed queue and perform mutation operations on the current seeds to generate test seeds.

[0039] The third module is used to input test seeds into the deep neural network of the preset autonomous driving perception system for test prediction, and output the test seeds that fail the test to the failure set. The test coverage of the current seed and the test coverage after adding the test seed are obtained by using the guidance index. If the test coverage after adding the test seed is higher than the test coverage of the current seed, the test seed is added to the seed queue. The guidance index includes spatial coverage and semantic coverage.

[0040] The fourth module increments the iteration count by 1, returns to the steps of the second module, and continues until the iteration count reaches the preset maximum iteration count, then outputs the failure set; the initial iteration count is 0.

[0041] In some embodiments, the apparatus further includes:

[0042] The fifth module is used to update the selection probability of each seed in the seed queue based on the updated seed queue;

[0043] The updated selection probability is expressed as follows:

[0044] P(s) = 1 / g(ori(s)) 2

[0045] In the formula, ori(s) represents the initial seed corresponding to the seed scene, and g(ori(s)) represents the number of test seeds generated by mutation of the initial seed.

[0046] In some embodiments, the apparatus further includes:

[0047] The sixth module is used to calculate the FRD score for each test seed and discard test seeds whose FRD scores are less than a preset score threshold.

[0048] The expression for the FRD score is:

[0049] FRD(X,Y)=||μ X -μ Y || 2 +Tr(∑ X +∑ Y -2(∑ X ∑ Y ) 1 / 2 )

[0050] In the formula, X represents the initial seed, Y represents the test seed corresponding to X, FRD(X,Y) represents the FRD score corresponding to Y; μ represents the mean vector of data activation, ‖·‖ represents the L2 norm of the vector; ∑ represents the covariance matrix of data activation, and Tr represents the trace of the covariance matrix.

[0051] To achieve the above objectives, another aspect of the present invention provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0052] To achieve the above objectives, another aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0053] The embodiments of this invention include at least the following beneficial effects: This invention provides a fuzzy testing method, apparatus, and storage medium for an autonomous driving perception system. This scheme obtains an initial seed and constructs a seed queue using the initial seed. The initial seed is obtained based on a preset dataset from the autonomous driving domain, and the seeds in the seed queue represent 3D point clouds generated by a LiDAR system. A preset number of current seeds are selected from the seed queue, and mutation operations are performed on the current seeds to generate test seeds. The test seeds are input into a preset deep neural network of the autonomous driving perception system for test prediction, and test seeds with failed predictions are output to a failure set. A guiding index is used to obtain the test coverage of the current seed and the test coverage after adding the test seeds. If the test coverage after adding the test seeds is higher than the test coverage of the current seed, the test seeds are added to the seed queue. The guiding index includes spatial coverage and semantic coverage. The iteration count is incremented by 1, and the process of selecting a preset number of current seeds from the seed queue is repeated until the iteration count reaches a preset maximum iteration count, at which point a failure set is output. The initial iteration count is 0. This invention designs a fuzzy testing framework for an autonomous driving perception system using LiDAR. This invention uses mutation operations to generate test data and perform automated testing. Furthermore, this invention introduces two metrics, spatial coverage and semantic coverage, to guide the testing process. The aim is to thoroughly search for potential defects in LiDAR-based perception models by considering scene semantics and the spatial distribution of obstacles. This invention enables efficient fuzzy testing of autonomous driving perception systems. Attached Figure Description

[0054] Figure 1 This is a flowchart of the fuzzy testing method for the autonomous driving perception system provided in this embodiment of the invention;

[0055] Figure 2 This is a general flowchart of the fuzzy testing method for the autonomous driving perception system provided in the embodiments of the present invention;

[0056] Figure 3 This is a flowchart illustrating the specific program execution of the fuzzy testing method for the autonomous driving perception system provided in this embodiment of the invention.

[0057] Figure 4 This is a flowchart for calculating the FRD score provided in an embodiment of the present invention;

[0058] Figure 5 This is a flowchart of equivalence class generation provided in an embodiment of the present invention;

[0059] Figure 6 This is a schematic diagram of the structure of the fuzzy testing device for the autonomous driving perception system provided in an embodiment of the present invention;

[0060] Figure 7This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0062] It is understood that the terms "first," "second," etc., used in this invention may be used to describe various concepts, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of embodiments of this invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" or "when" as used herein may be interpreted as "when," "in response to determination," or "in the event of a determination."

[0063] The terms “at least one,” “multiple,” “each,” “any,” etc., used in this invention, “at least one” includes one, two, or more than two; “multiple” includes two or more than two; “each” refers to each of the corresponding multiple; and “any” refers to any one of the multiple.

[0064] Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this invention is for descriptive purposes only and is not intended to limit the invention.

[0065] The fuzz testing method for an autonomous driving perception system provided in this invention relates to the field of data processing technology. This fuzz testing method for an autonomous driving perception system can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the fuzz testing method for the autonomous driving perception system, but is not limited to the above forms.

[0066] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0067] Figure 1 This is an optional flowchart of the fuzzy testing method for the autonomous driving perception system provided in this embodiment of the invention. Figure 1 The method may include, but is not limited to, steps S100 to S400.

[0068] S100. Obtain the initial seed and construct a seed queue using the initial seed;

[0069] The initial seed is obtained by organizing a pre-set dataset in the field of autonomous driving, and the seeds in the seed queue represent the three-dimensional point cloud generated by the lidar system.

[0070] For example, in some specific implementations, an initial seed is first obtained, and then a seed queue is generated using the initial seed. The seed refers to the 3D point cloud generated by the LiDAR system. Additionally, a pre-set LiDAR-based autonomous driving perception system DNN model can also be obtained.

[0071] In some specific application scenarios, the KITTI dataset can be used as input. KITTI (Karlsruhe Institute of Technology and Toyota Institute) is a well-known dataset in the field of autonomous driving research. This dataset includes a large amount of driving data obtained from various sensors (such as laser scanners and color cameras) in real driving scenarios. Specifically, KITTI utilizes the Velodyne HDL-64E rotating 3D laser scanner as part of its sensor setup. Each data point records its (x, y, z) coordinates and an additional reflectance value (r). The main use of this dataset is as a benchmark for 3D object detection, including 3712 training images, 3769 validation images, 7518 test images and their associated point clouds. S200: Select a preset number of current seeds from the seed queue, and perform a mutation operation on the current seeds to generate test seeds;

[0072] It should be noted that in some embodiments, selecting a preset number of current seeds from the seed queue may include the following steps: randomly selecting a preset number of seeds from the seed queue as current seeds using a seed selection strategy.

[0073] For example, in some specific implementations, a seed selection strategy may be used to randomly select k seeds from the seed queue.

[0074] In some embodiments, after updating the seed queue according to the test seed, the method may further include: updating the selection probability of each seed in the seed queue based on the updated seed queue; wherein the expression for the updated selection probability is:

[0075] P(s) = 1 / g(ori(s)) 2

[0076] In the formula, ori(s) represents the initial seed corresponding to the seed scene, and g(ori(s)) represents the number of test seeds generated by mutation of the initial seed.

[0077] For example, in some specific embodiments, the seed selection strategy may further include: recording the original seed corresponding to each mutated seed in a seed set, recording a variable for each original seed, the variable representing the number of variants of the seed in the seed set, selecting seeds according to probability, the probability of each seed being selected is calculated as: P(s)=1 / g(ori(s)) 2 , where ori(s) represents the initial seed corresponding to the seed scene, and g(ori(s)) represents the total number of seeds generated after transformation from the initial seed.

[0078] It should also be noted that in some embodiments, generating test seeds by performing mutation operations on the current seed may include the following steps: randomly selecting a mutation operator for each current seed and sequentially performing mutation operations until the number of mutations exceeds a preset maximum number of attempts; and then organizing the results of all mutation operations to obtain the test seed corresponding to each current seed. The mutation operators include object transformation and the addition of adverse weather conditions. Specifically, object transformation includes translation, rotation, insertion, and scaling; and adverse weather conditions include rain, snow, and fog.

[0079] For example, in some specific implementations, each seed can be mutated up to a maximum number of attempts (tryMax). Each mutation randomly selects a mutation operator to perform the mutation operation, and then a new set of test seeds is generated. Specifically, mutation operators include object transformations and adverse weather conditions. Object transformations include translation, rotation, insertion, and scaling, while adverse weather conditions include rain, snow, and fog.

[0080] In some embodiments, after the step of generating test seeds by performing mutation operations on the current seed, the method may further include the following steps: calculating the FRD score of each test seed, and discarding test seeds whose FRD scores are less than a preset score threshold; wherein, the expression for the FRD score is:

[0081] FRD(X,Y)=||μ X -μ Y || 2 +Tr(∑ X +∑ Y -2(∑ X ∑ Y ) 1 / 2 )

[0082] In the formula, X represents the initial seed, Y represents the test seed corresponding to X, FRD(X,Y) represents the FRD score corresponding to Y; μ represents the mean vector of data activation, ‖·‖ represents the L2 norm of the vector; ∑ represents the covariance matrix of data activation, and Tr represents the trace of the covariance matrix.

[0083] For example, in some specific implementations, the FRD score for each test seed can be calculated. If the FRD score corresponding to the test seed is less than a threshold t, it is retained; otherwise, it is discarded. The specific calculation formula is as follows:

[0084] FRD(X,Y)=||μ X -μ Y || 2 +Tr(∑ X +∑ Y -2(∑ X ∑ Y ) 1 / 2 )

[0085] Where X and Y represent the network activation distributions of real data and synthetic data, respectively. Real data refers to the initial seed, and synthetic data refers to the mutated seed generated after mutation of the initial seed. μ represents the mean vector of data activations, and ∑ is the covariance matrix of data activations. The trace of the matrix is ​​represented by Tr, and ‖·‖ represents the L2 norm of the vector.

[0086] S300: Input the test seed into the deep neural network of the preset autonomous driving perception system for test prediction, and output the test seed of the failed test to the failure set. Use the guidance index to process and obtain the test coverage of the current seed and the test coverage after adding the test seed. If the test coverage after adding the test seed is higher than the test coverage of the current seed, then add the test seed to the seed queue.

[0087] Among them, the guiding indicators include spatial coverage and semantic coverage;

[0088] It should be noted that in some embodiments, when the test coverage rate is the spatial coverage rate, obtaining the test coverage rate of the current seed and the test coverage rate after adding the test seed using the guiding index processing may include the following steps: summing and averaging the ratio of the sum of the areas corresponding to the objects to be detected in each 3D point cloud contained in the seed to the total area of ​​the radar-sensed test space, to obtain the spatial coverage rate; wherein, the expression for the spatial coverage rate is:

[0089]

[0090] In the formula, SPC(T) represents spatial coverage; Avg represents the average treatment. The sum of the areas of the objects to be detected in the i-th 3D point cloud of the seed; m represents the number of initial seeds; This represents the total area of ​​the radar-sensed space in the i-th 3D point cloud of the seed.

[0091] For example, in some specific embodiments, the formula for calculating spatial coverage is: Among them, for point cloud data c1...c n It is the data generated iteratively from the initial seed c0 during the fuzzing process. i Represents point cloud c i The object to be detected is PR, which is the area of ​​the 3D truth box projected onto the 2D plane containing the x and y axes. Let cn represent the area of ​​the nth object to be detected projected onto the 2D plane containing the x and y axes from the 3D truth box. Given an initial seed c0, the corresponding LiDAR sensing region can be represented as the sum of the areas of all objects to be detected, i.e.: For seed c i The corresponding LiDAR sensing area can be represented as the sum of the areas of all objects to be detected. To detect the total area of ​​the space to be measured by radar.

[0092] It should also be noted that in some embodiments, when the test coverage rate is the semantic coverage rate, obtaining the test coverage rate of the current seed and the test coverage rate after adding test seeds using guided index processing may include the following steps: obtaining the semantic coverage rate by the ratio of the number of equivalence classes generated by the fuzzy test corresponding to the seed to the total number of all possible semantically perceptible equivalence classes; the step of generating equivalence classes includes: generating a corresponding scene graph for each seed, the scene graph being a directed graph; performing simplification operations on the scene graph to generate an abstract scene graph; clustering the abstract scene graph to generate equivalence classes, where an equivalence class refers to a set of abstract scene graphs with the same structure and connection relationship; and finally calculating the number of equivalence classes; wherein, the expression for the semantic coverage rate is:

[0093]

[0094] In the formula, |ASG S (T)| represents the number of equivalence classes generated by the fuzz test. This represents the total number of equivalence classes of all possible perceptual semantics generated by the fuzz test.

[0095] For example, in some specific implementations, the formula for calculating semantic coverage is: |ASG S (T) represents the number of equivalence classes generated by the fuzz test. This represents the total number of equivalence classes of all possible perceptual semantics generated by the initial seed during fuzz testing.

[0096] S400, Increment the iteration count by 1, return to the step of selecting a preset number of current seeds from the seed queue, until the iteration count reaches the preset maximum iteration count, and output the failure set.

[0097] The initial number of iterations is 0.

[0098] For example, in some specific implementations, the operations of steps S200 and S300 are repeated until the termination condition is met, and the test ends. The termination condition is to set a maximum number of iterations (e.g., 1000 for the experiment). The iteration terminates when the number of iterations exceeds the number of iterations.

[0099] To explain in detail the principle of the technical solution of the present invention, the overall process of the present invention will be described below with reference to some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and should not be regarded as a limitation of the present invention.

[0100] First, it's important to note that recent testing techniques have demonstrated that autonomous driving systems lack robustness to synthetic images of driving scenarios. DeepTest employs a systematic approach to automatically generate test cases. This involves transforming labeled driving scenario images by applying various effect filters (such as fog / rain and simple affine transformations) to the original images and checking whether the system performs consistently in both the original and transformed scenes. DeepRoad proposes an unsupervised deep neural network (DNN) framework for automatically testing DNN-based autonomous driving systems. By applying generative adversarial networks (GANs) and corresponding real-world weather scenarios, more realistic driving scenarios can be generated under various weather conditions. Furthermore, DeepBillBoard is a systematic physical world testing method focused on generating a sufficiently robust and resilient printable physical world adversarial billboard for testing.

[0101] While the methods described above have successfully detected various erroneous behaviors in autonomous driving systems, the impact of point clouds collected by LiDAR in harsh environmental conditions has received little attention. LiDAR is one of the most critical sensors in autonomous driving systems. It calculates the distance to a target by emitting lasers and processing their echoes—specifically, by emitting laser pulses and measuring the time required for these pulses to reflect from and return to the target. LiDAR sensors are widely used in 3D object detection, acquiring 3D scene information in the form of irregular and sparse point clouds, providing an important solution for 3D scene perception and understanding. Furthermore, it offers highly accurate distance measurement and more reliable detection performance.

[0102] Furthermore, unlike cameras, it is unaffected by light and works well at night. However, these sensors are known to be sensitive to adverse conditions. For these reasons, designing an automated testing method for autonomous driving software based on LiDAR is crucial. First, in the operating environment of an autonomous driving system, the point cloud perceived by LiDAR may be affected by various environmental factors, including severe weather (such as fog, rain, snow, etc.). This may cause varying degrees of attenuation to the perceived point cloud or various types of noise information, leading to erroneous behavior of the autonomous driving system during operation. Furthermore, the integration of specific semantic information in the scene may cause the LiDAR-based perception system to fail, potentially leading to traffic accidents. Second, the testing process of modern autonomous driving systems largely depends on the collected datasets. In actual road tests, point clouds are typically acquired using LiDAR. Due to the diversity of environmental conditions, acquiring point clouds under different environmental factors requires significant resource consumption. Simultaneously, the sparse distribution of scanned points in three-dimensional space complicates the extraction of features from large-area targets.

[0103] Finally, manually labeling point clouds is a laborious and time-consuming task, requiring a large amount of manpower to manually check the visualization results of the point clouds.

[0104] In view of this, and to address the shortcomings of existing technologies, this invention provides a fuzzy testing method for an autonomous driving perception system based on LiDAR. In some specific application scenarios, the technical solution of this invention can be implemented as follows:

[0105] This invention enables fuzzy testing of a LiDAR-based autonomous driving perception system using Python 3.7 and PyTorch 1.13.1. All tests of the LiDAR-based perception system were performed on an Ubuntu 20.04.6 LTS server equipped with an NVIDIA GeForce RTX 4070 Ti graphics card and 12GB of video memory.

[0106] This invention can use the KITTI dataset as input. KITTI (Karlsruhe Institute of Technology and Toyota Institute) is a well-known dataset in the field of autonomous driving research. This dataset includes a large amount of driving data obtained from various sensors (such as laser scanners and color cameras) in real-world driving scenarios. Specifically, KITTI utilizes the Velodyne HDL-64E rotating 3D laser scanner as part of its sensor setup. Each data point records its (x, y, z) coordinates and an additional reflectivity value (r). The primary use of this dataset is as a benchmark for 3D object detection, comprising 3712 training images, 3769 validation images, 7518 test images, and their associated point clouds.

[0107] In order to effectively evaluate the performance of the test method of the present invention, the present invention can select four state-of-the-art lidar sensing systems as experimental subjects.

[0108] Here is a brief introduction to each system: PointPillars is a 3D object detection encoder system that uses PointNets to learn representations of point clouds organized in vertical columns. This model delivers superior detection performance at a faster speed, surpassing existing solutions. Notably, it is widely used in the open-source autonomous driving systems Apollo and Autoware.

[0109] The SECOND system incorporates enhanced sparse convolutions, significantly accelerating training and inference. Furthermore, a novel angle loss regression method is introduced to improve orientation estimation performance, and a new data augmentation method is employed to expedite convergence and enhance overall performance.

[0110] The PointRCNN system consists of two stages: an initial subnetwork segments the entire point cloud of the scene into foreground and background points using a bottom-up approach, directly generating a limited number of high-quality 3D results. Subsequently, the next-level subnetwork refines the 3D results in normalized coordinates to obtain the final detection results.

[0111] PV-RCNN is a voxel-based object detection system that, through a unique voxel set abstraction module, compresses a 3D scene using a 3D voxel convolutional neural network (CNN) into a simplified keypoint set, reducing subsequent computation and capturing representative scene features. Furthermore, the voxel-to-keypoint scene encoding and keypoint-to-mesh RoI feature abstraction methods in this framework significantly improve 3D object detection performance compared to previous methods.

[0112] To evaluate the effectiveness of the fuzz testing method in testing a LiDAR-based autonomous driving perception system, this embodiment of the invention uses OpenPCDet as the evaluation scheme for the testing technology in this study. OpenPCDet is a user-friendly, self-contained open-source platform for LiDAR-based 3D object detection. This embodiment of the invention uses default parameter configurations and accesses pre-trained models from a model repository.

[0113] This invention establishes a fuzzy testing framework for an autonomous driving perception system based on LiDAR, such as... Figure 2 and Figure 3 The diagram shows a flowchart of the fuzzy testing method for an autonomous driving perception system based on LiDAR, including the following steps:

[0114] Step 1 (corresponding) Figure 2 S110): The input has an initial seed. This embodiment of the invention uses point cloud data from the KITTI dataset and also includes a DNN model for an autonomous driving perception system based on LiDAR. This embodiment of the invention uses PointPillars, SECOND, PointRCNN and PV-RCNN to generate a seed queue using the initial seed. The seed refers to the three-dimensional point cloud generated by the LiDAR system.

[0115] Step Two (corresponding) Figure 2 S120): A seed selection strategy is adopted to randomly select k seeds from the seed queue. For each seed, mutation is performed without exceeding the maximum number of attempts tryMax. Each mutation is performed by randomly selecting a mutation operator to perform the mutation operation, and then a new set of test seeds is generated. The FRD score of each test seed is calculated. If the FRD score of the test seed is less than the threshold t, it is retained; otherwise, it is discarded.

[0116] Specifically, mutation operators include object transformation and adverse weather conditions. Object transformation includes translation, rotation, insertion, and scaling, while adverse weather conditions include rain, snow, and fog.

[0117] Optionally, the seed selection strategy also includes: recording the original seed corresponding to each mutated seed in the seed set, recording a variable for each original seed, which represents the number of variants of that seed in the seed set, and selecting seeds according to probability. The probability of each seed being selected is calculated as: P(s)=1 / g(ori(s)). 2 , where ori(s) represents the initial seed s corresponding to the seed scene, and g(s0) represents the total number of seeds generated after transformation through the initial seed s0;

[0118] Among them, such as Figure 4 The diagram shown is a flowchart for calculating the FRD score:

[0119] The steps for calculating the FRD score include: 1) Feature extraction: Using a pre-trained LiDAR perception network, features are extracted from LiDAR point cloud samples, and a certain number of feature activations are randomly selected from the bottleneck layer of the perception network; 2) Gaussian distribution fitting: Gaussian distributions are fitted to these feature activations to calculate the statistical differences between them; 3) Calculation of mean and covariance: The mean and covariance of the feature activations for synthetic and real samples are calculated respectively; 4) Calculation of Wasserstein distance (also known as Earth Mover's Distance, EMD, a method to measure the difference between two probability distributions): The Wasserstein distance is used to quantify the distance between two Gaussian distributions. The Wasserstein distance between the mean and covariance of the feature activations of synthetic and real samples is calculated; 5) Calculation of FRD score: The square of the Wasserstein distance is the FRD score. The specific calculation formula is as follows:

[0120] FRD(X,Y)=||μ X -μ Y || 2 +Tr(∑ X +∑ Y -2(∑ X ∑ Y ) 1 / 2 )

[0121] Where X and Y represent the network activation distributions of real data and synthetic data, respectively. Real data refers to the initial seed, and synthetic data refers to the mutated seed generated after mutation of the initial seed. μ represents the mean vector of data activations, and ∑ is the covariance matrix of data activations. The trace of the matrix is ​​represented by Tr, and ‖·‖ represents the L2 norm of the vector.

[0122] Step 3 (corresponding) Figure 2 S130): The test coverage is calculated using guiding indicators, and the test seeds are input into the DNN model of the autonomous driving perception system based on LiDAR for prediction. For each test seed, if it is a failed test, it is added to the failed set. If the test seed improves the test coverage, it is added to the seed queue. Seeds that do not meet the above two conditions will be discarded. Specifically, the guiding indicators include spatial coverage and semantic coverage.

[0123] The formula for calculating spatial coverage is: Among them, for point cloud data c1...c n It is the data generated iteratively from the initial seed c0 during the fuzzing process. i Represents point cloud c iThe object to be detected is PR, which is the area of ​​the 3D truth box projected onto the 2D plane containing the x and y axes. Let cn represent the area of ​​the nth object to be detected projected onto the 2D plane containing the x and y axes from the 3D truth box. Given an initial seed c0, the corresponding LiDAR sensing region can be represented as the sum of the areas of all objects to be detected, i.e.: For seed c i The corresponding LiDAR sensing area can be represented as the sum of the areas of all objects to be detected. To detect the total area of ​​the space to be measured by radar;

[0124] The formula for calculating semantic coverage is: |ASG S (t) represents the number of equivalence classes generated by the fuzz test. This represents the total number of equivalence classes of all possible perceptual semantics generated by the initial seed during fuzz testing; such as... Figure 5 The diagram shown is a flowchart for generating equivalence classes. The specific process is as follows:

[0125] The steps for generating equivalence classes include: generating a corresponding scene graph for each seed. The scene graph is a directed graph, where each node is a participant in the driving scene (e.g., roads, vehicles, pedestrians, weather, etc.), and nodes have connections (e.g., existence, connection, location, containment, weather effects, etc.). The scene graph is then simplified to generate an abstract scene graph. For example, if different identifiers are used, such as 'car_1' on the left road and 'car_2' on the right road, these identifiers are irrelevant to the positional relationship between the vehicles. If they are relabeled as "car", their semantic relationship remains unchanged. The abstract scene graphs are then clustered to generate equivalence classes. Abstract scene graphs with the same structure and connections will correspond to the same equivalence class; that is, if multiple abstract scene graphs are isomorphic, they are labeled as the same equivalence class. Finally, the number of equivalence classes is calculated.

[0126] Step Four (corresponding) Figure 2 S140): Repeat steps two and three until the termination condition is met, then end the test. The termination condition is to set a maximum number of iterations. When the number of iterations exceeds the number of iterations, the iteration terminates.

[0127] This invention was evaluated on four state-of-the-art LiDAR-based perception systems. Experimental results show that the variations applied to the testing method of this invention can detect a large number of erroneous behaviors in LiDAR-based autonomous driving perception systems. Furthermore, the proposed testing criteria can be used to improve testing efficiency and diversity. In the case study, we analyzed the error types in LiDAR-based perception systems and concluded that the fuzzy testing method of this invention provides a foundation for the testing and analysis development of autonomous driving systems (ADS systems).

[0128] In summary, this invention designs a fuzzy testing framework for an autonomous driving perception system based on LiDAR. It uses different mutation operators to generate a substantial amount of test data and execute automated testing. Furthermore, we introduce two metrics, spatial coverage and semantic coverage, to guide the testing process. The aim is to thoroughly search for potential defects in the LiDAR-based perception model by considering scene semantics and the spatial distribution of obstacles.

[0129] Compared with the prior art, the present invention has at least the following beneficial effects:

[0130] 1. The fuzzy testing framework for an autonomous driving perception system based on LiDAR of the present invention employs multiple mutation operators and seed selection strategies to automatically generate test cases and test the LiDAR perception system, effectively revealing...

[0131] Given sufficient testing time, the various defects inherent in lidar sensing systems can be discovered, revealing aspects that are difficult to detect with other testing methods.

[0132] 2. Currently available datasets contain relatively few test cases. The fuzzy testing framework for autonomous driving perception systems based on LiDAR of this invention can automatically generate more test cases with greater differences between them, providing diverse testing and avoiding the large amount of resources consumed by traditional LiDAR point cloud data acquisition.

[0133] 3. This invention also proposes two guiding indicators, namely spatial coverage and semantic coverage. These indicators can guide the fuzzy testing framework to develop in the direction of discovering more defects, thereby discovering more potential defects in LiDAR-based autonomous driving perception systems. At the same time, these indicators also indicate the adequacy of the testing process, allowing relevant personnel to intuitively understand the completeness of the test.

[0134] 4. This invention can automatically annotate point clouds, avoiding a lot of repetitive manual work in annotating point clouds and improving accuracy.

[0135] like Figure 6As shown, this embodiment of the invention also provides a fuzzy testing device 900 for an autonomous driving perception system, which may include:

[0136] The first module 901 is used to obtain an initial seed and construct a seed queue using the initial seed; wherein, the initial seed is obtained based on a preset dataset in the field of autonomous driving, and the seeds in the seed queue represent the three-dimensional point cloud generated by the lidar system;

[0137] The second module 902 is used to select a preset number of current seeds from the seed queue and perform mutation operations on the current seeds to generate test seeds.

[0138] The third module 903 is used to input test seeds into the deep neural network of the preset autonomous driving perception system for test prediction, and output the test seeds that fail the test to the failure set; it uses guiding indicators to process and obtain the test coverage of the current seed and the test coverage after adding the test seed. If the test coverage after adding the test seed is higher than the test coverage of the current seed, the test seed is added to the seed queue; the guiding indicators include spatial coverage and semantic coverage.

[0139] The fourth module 904 is used to increment the iteration count by 1, return to the steps of the second module, and continue until the iteration count reaches the preset maximum iteration count, and output the failure set; where the initial iteration count is 0.

[0140] In some embodiments, the apparatus may further include:

[0141] The fifth module is used to update the selection probability of each seed in the seed queue based on the updated seed queue;

[0142] The updated selection probability is expressed as follows:

[0143] P(s) = 1 / g(ori(s)) 2

[0144] In the formula, ori(s) represents the initial seed corresponding to the seed scene, and g(ori(s)) represents the number of test seeds generated by mutation of the initial seed.

[0145] In some embodiments, the apparatus may further include:

[0146] The sixth module is used to calculate the FRD score for each test seed and discard test seeds whose FRD scores are less than a preset score threshold.

[0147] The expression for the FRD score is:

[0148] FRD(X,Y)=||μ X -μY || 2 +Tr(∑ X +∑ Y -2(∑ X ∑ Y ) 1 / 2 )

[0149] In the formula, X represents the initial seed, Y represents the test seed corresponding to X, FRD(X,Y) represents the FRD score corresponding to Y; μ represents the mean vector of data activation, ‖·‖ represents the L2 norm of the vector; ∑ represents the covariance matrix of data activation, and Tr represents the trace of the covariance matrix.

[0150] The content of the method embodiments of the present invention is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0151] This invention also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the fuzzy testing method of the aforementioned autonomous driving perception system. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0152] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0153] Please see Figure 7 , Figure 7 The hardware structure of an electronic device 1000 according to another embodiment is illustrated. The electronic device 1000 includes:

[0154] The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0155] The memory 1002 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1002 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001 to execute the fuzzy testing method of the autonomous driving perception system of the embodiments of this invention.

[0156] Input / output interface 1003 is used to implement information input and output;

[0157] The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0158] Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004);

[0159] The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0160] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned fuzzy testing method for the autonomous driving perception system.

[0161] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0162] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0163] This invention provides a fuzzy testing method, device, electronic equipment, and storage medium for an autonomous driving perception system. The method involves acquiring an initial seed and constructing a seed queue. The initial seed is based on a pre-defined dataset from the autonomous driving domain, and the seeds in the seed queue represent 3D point clouds generated by a LiDAR system. A predetermined number of current seeds are selected from the seed queue, and mutation operations are performed on these seeds to generate test seeds. The test seeds are then input into a pre-defined deep neural network of the autonomous driving perception system for test prediction, and test seeds with failed predictions are output to a failure set. Guiding metrics are used to obtain the test coverage of the current seeds and the test coverage after adding test seeds. If the test coverage after adding test seeds is higher than the test coverage of the current seeds, the test seeds are added to the seed queue. Guiding metrics include spatial coverage and semantic coverage. The iteration count is incremented by 1, and the process of selecting a predetermined number of current seeds from the seed queue is repeated until the iteration count reaches a predetermined maximum. A failure set is then output. The initial iteration count is 0. This invention designs a fuzzy testing framework for an autonomous driving perception system using LiDAR, and it uses mutation operations to generate test data and perform automated testing. Furthermore, this invention introduces two metrics, spatial coverage and semantic coverage, to guide the testing process. The aim is to thoroughly search for potential defects in LiDAR-based perception models by considering scene semantics and the spatial distribution of obstacles. This invention enables efficient fuzzy testing of autonomous driving perception systems.

[0164] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0165] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0166] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0167] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0168] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0169] It should be understood that in this invention, "at least one (item)" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0170] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0171] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0172] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0173] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0174] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of the present invention should be within the scope of the claims of the present invention.

Claims

1. A fuzzy testing method for an autonomous driving perception system, characterized in that, The method includes the following steps: An initial seed is obtained, and a seed queue is constructed using the initial seed; wherein, the initial seed is obtained based on a preset dataset in the field of autonomous driving, and the seeds in the seed queue represent the three-dimensional point cloud generated by the lidar system; Select a preset number of current seeds from the seed queue, and perform a mutation operation on the current seeds to generate test seeds; The test seeds are input into a preset deep neural network of an autonomous driving perception system for test prediction. Test seeds with predicted failure results are added to the failure set. The test coverage of the current seed and the test coverage after adding the test seed are obtained using guiding indicators. If the test coverage after adding the test seed is higher than the test coverage of the current seed, the test seed is added to the seed queue. The guiding indicators include spatial coverage and semantic coverage. Wherein, when the test coverage rate is the spatial coverage rate, the process of obtaining the test coverage rate of the current seed and the test coverage rate after adding the test seed using the guiding metric includes the following steps: The spatial coverage rate is obtained by summing and averaging the ratio of the sum of the areas of the objects to be detected in each of the three-dimensional point clouds contained in the seed to the total area of ​​the space to be detected by the radar. The expression for the spatial coverage rate is as follows: In the formula, This indicates the spatial coverage rate; This indicates the average processing; This represents the sum of the areas corresponding to the object to be detected in the i-th point cloud of the seed; This indicates the number of initial seeds; This represents the total area of ​​the radar-sensed space in the i-th three-dimensional point cloud of the seed; Increment the iteration count by 1, return to the step of selecting a preset number of current seeds from the seed queue, until the iteration count reaches the preset maximum iteration count, and output the failure set; wherein, the initial iteration count is 0.

2. The fuzzy testing method for an autonomous driving perception system according to claim 1, characterized in that, Selecting a preset number of current seeds from the seed queue includes the following steps: A seed selection strategy is used to randomly select a preset number of seeds from the seed queue as the current seeds.

3. The fuzzy testing method for an autonomous driving perception system according to claim 1, characterized in that, After updating the seed queue according to the test seed, the method further includes the following steps: Update the selection probability of each seed in the seed queue based on the updated seed queue; The updated expression for the selection probability is: In the formula, where ori( ) represents the initial seed corresponding to the seed scene. (ori( )) represents the number of test seeds generated through mutation of the initial seed.

4. The fuzzy testing method for an autonomous driving perception system according to claim 1, characterized in that, The process of generating a test seed by performing a mutation operation on the current seed includes the following steps: For each current seed, a mutation operator is randomly selected and the mutation operation is executed sequentially until the number of mutations exceeds the preset maximum number of attempts. The test seed corresponding to each current seed is obtained based on the execution results of all mutation operations. The mutation operator includes object transformation and the addition of severe weather conditions.

5. The fuzzy testing method for an autonomous driving perception system according to claim 1, characterized in that, After the step of performing a mutation operation on the current seed to generate a test seed, the method further includes the following steps: Calculate the FRD score for each test seed, and discard test seeds whose FRD scores are less than a preset score threshold; The expression for the FRD score is as follows: In the formula, X This refers to the initial seed. Y express X The corresponding test seed, FRD ( X , Y )express Y The corresponding FRD score; Let represent the mean vector of the data activations, and let ‖•‖ represent the L2 norm of the vector; ∑ represent the covariance matrix of the data activations, and Tr represent the trace of the covariance matrix.

6. The fuzzy testing method for an autonomous driving perception system according to claim 1, characterized in that, When the test coverage rate is the semantic coverage rate, the process of obtaining the test coverage rate of the current seed and the test coverage rate after adding the test seed using the guiding metric includes the following steps: The semantic coverage is obtained by the ratio of the number of equivalence classes generated by the fuzzy test corresponding to the seed to the total number of all possible semantically perceptible equivalence classes. The steps of generating equivalence classes include: generating a corresponding scene graph for each seed, the scene graph being a directed graph; performing simplification operations on the scene graph to generate an abstract scene graph; clustering the abstract scene graph to generate equivalence classes, where an equivalence class refers to a set of abstract scene graphs with the same structure and connection relationship; and finally calculating the number of equivalence classes. The semantic coverage rate is expressed as follows: In the formula, This indicates the number of equivalence classes generated by the fuzz test. This represents the total number of equivalence classes of all possible perceptual semantics generated by the fuzz test.

7. A fuzzy testing device for an autonomous driving perception system, characterized in that, The device includes: The first module is used to obtain an initial seed and construct a seed queue using the initial seed; wherein, the initial seed is obtained based on a preset dataset in the field of autonomous driving, and the seeds in the seed queue represent the three-dimensional point cloud generated by the lidar system; The second module is used to select a preset number of current seeds from the seed queue and perform mutation operations on the current seeds to generate test seeds. The third module is used to input the test seed into a preset deep neural network of an autonomous driving perception system for test prediction, and output the test seed with the prediction result of failure to the failure set; using guiding indicators to process and obtain the test coverage of the current seed and the test coverage after adding the test seed; if the test coverage after adding the test seed is higher than the test coverage of the current seed, then the test seed is added to the seed queue; wherein, the guiding indicators include spatial coverage and semantic coverage; Wherein, when the test coverage rate is the spatial coverage rate, the process of obtaining the test coverage rate of the current seed and the test coverage rate after adding the test seed using the guiding metric includes the following steps: The spatial coverage rate is obtained by summing and averaging the ratio of the sum of the areas of the objects to be detected in each of the three-dimensional point clouds contained in the seed to the total area of ​​the space to be detected by the radar. The expression for the spatial coverage rate is as follows: In the formula, This indicates the spatial coverage rate; This indicates the average processing; This represents the sum of the areas corresponding to the object to be detected in the i-th point cloud of the seed; This indicates the number of initial seeds; This represents the total area of ​​the radar-sensed space in the i-th three-dimensional point cloud of the seed; The fourth module is used to increment the iteration count by 1, return to the steps of the second module, until the iteration count reaches the preset maximum iteration count, and output the failure set; wherein the iteration count is initially 0.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images