Fuzzy test method and device for automatic driving perception system and storage medium

Abstract

The invention discloses a fuzzy test method and device for an automatic driving perception system and a storage medium. The method comprises the steps of constructing a seed queue by using initial seeds; selecting a current seed from the seed queue, and performing mutation operation on the current seed to generate a test seed; performing test prediction on the test seeds, and outputting the test seeds of which the prediction results are failed tests to a failure set; processing by utilizing a guide index to obtain a test coverage rate of the current seed and a test coverage rate after the test seed is added, and updating a seed queue according to the test seed with the increased test coverage rate; wherein the guide index comprises a space coverage rate and a semantic coverage rate; and adding 1 to the number of iterations, returning to execute the step of selecting the current seed from the seed queue until the number of iterations reaches a preset maximum number of iterations, and outputting a failure set. The fuzzy test of the automatic driving perception system can be efficiently realized, and the method can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a fuzzy testing method, device and storage medium for an autonomous driving perception system. Background Art

[0002] With the rapid development of artificial intelligence and sensor technology, the development of self-driving cars has made great achievements. As a typical safety-critical intelligent software, the self-driving system uses sensors such as light detection and ranging (LiDAR) and cameras to capture surrounding environment information as input, and automatically completes different driving tasks through various functional components. Several major automakers and organizations, including Tesla, Waymo, Uber, and Baidu, are manufacturing and actively testing these cars. Recent results show that self-driving cars have traveled millions of miles without any human intervention.

[0003] However, when such intelligent systems are deployed in real environments, they often exhibit incorrect and unexpected extreme behaviors that may lead to fatal collisions. Many such real cases have been reported. For example, some autonomous vehicles directly hit white trucks while driving. The cause of the accident was that the autonomous driving system failed to distinguish a white truck from the bright sky, mainly because the truck was too high. Since these cars complete driving tasks based on the environment measured by different sensors (such as cameras, lidar, etc.), the possible input space is very large, resulting in a large number of extreme cases. There is an urgent need to explore a better way to effectively test autonomous driving software. In autonomous driving tasks, the lidar perception system is mainly responsible for environmental perception and obstacle detection. In bad weather (such as rain, fog, and snow), the detection accuracy of the lidar will be significantly reduced, and the sparse point cloud will make the target detection invalid; at the same time, in some driving scenarios, the combination of specific participants (i.e., the integration of specific semantic information in the scene) may also cause the lidar recognition to fail. This is due to defects in the perception software. As one of the core components of autonomous vehicles, the lidar perception system is crucial to driving safety.

[0004] Based on the above, there is currently a lack of effective, comprehensive testing methods for lidar perception systems, and there is an urgent need to explore a better method to effectively test autonomous driving systems to improve their robustness. Summary of the invention

[0005] The main purpose of the embodiments of the present invention is to propose a fuzzy testing method, device and storage medium for an autonomous driving perception system, in order to solve at least one problem of the prior art. The present invention can efficiently implement fuzzy testing of the autonomous driving perception system.

[0006] To achieve the above objective, an embodiment of the present invention provides a fuzzy testing method for an autonomous driving perception system, the method comprising:

[0007] Obtaining initial seeds, and using the initial seeds to construct a seed queue; wherein the initial seeds are obtained based on a preset data set in the field of autonomous driving, and the seeds in the seed queue represent a three-dimensional point cloud generated by a lidar system;

[0008] Select a preset number of current seeds from the seed queue, and perform mutation operations on the current seeds to generate test seeds;

[0009] Input the test seed into the preset deep neural network of the autonomous driving perception system for test prediction, and output the prediction result as the test seed of the failed test to the failure set; use the guidance indicator to process to obtain the test coverage of the current seed and the test coverage after adding the test seed. If the test coverage after adding the test seed is improved compared with the test coverage of the current seed, then add the test seed to the seed queue; where the guidance indicator includes spatial coverage and semantic coverage;

[0010] Add 1 to the number of iterations, return to the step of selecting a preset number of current seeds from the seed queue, until the number of iterations reaches the preset maximum number of iterations, and output the failure set; wherein the number of iterations is initially 0.

[0011] In some embodiments, selecting a preset number of current seeds from a seed queue comprises the following steps:

[0012] A seed selection strategy is used to randomly select a preset number of seeds from the seed queue as the current seeds.

[0013] In some embodiments, after the seed queue is updated according to the test seed, the method further comprises the following steps:

[0014] Update the selection probability of each seed in the seed queue based on the updated seed queue;

[0015] Among them, the expression of the updated selection probability is:

[0016] P(s)=1 / g(ori(s)) 2

[0017] Where ori(s) represents the initial seed corresponding to the seed scene, and g(ori(s)) represents the number of test seeds generated by mutation of the initial seed.

[0018] In some embodiments, performing a mutation operation on a current seed to generate a test seed includes the following steps:

[0019] A mutation operator is randomly selected for each current seed to perform mutation operations in sequence until the number of mutation operations exceeds the preset maximum number of attempts. The test seed corresponding to each current seed is obtained according to the execution results of all mutation operations.

[0020] Among them, mutation operators include the transformation of objects and the addition of severe weather conditions.

[0021] In some embodiments, after the step of performing a mutation operation on the current seed to generate a test seed, the method further includes the following steps:

[0022] Calculate the FRD score of each test seed, and discard the test seeds whose FRD score is less than the preset score threshold;

[0023] The expression of FRD score is:

[0024] FRD(X,Y)=||μ X -μ Y || 2 +Tr(∑ X +∑ Y -2(∑ X ∑ Y ) 1 / 2 )

[0025] Where X represents the initial seed, Y represents the test seed corresponding to X, FRD(X,Y) represents the FRD score corresponding to Y; μ represents the mean vector of data activations, ‖·‖ represents the L2 norm of the vector; ∑ represents the covariance matrix of data activations, and Tr represents the trace of the covariance matrix.

[0026] In some embodiments, when the test coverage is spatial coverage, obtaining the test coverage of the current seed and the test coverage after adding the test seed by using the guide indicator processing includes the following steps:

[0027] The ratio of the sum of the areas of the objects to be detected in each three-dimensional point cloud contained in the seed to the total area of ​​the space to be detected by the radar is accumulated and averaged to obtain the spatial coverage rate;

[0028] The expression of spatial coverage is:

[0029]

[0030] In the formula, SPC(T) represents the spatial coverage; Avg represents the average processing; represents the sum of the areas corresponding to the objects to be detected in the i-th 3D point cloud in the seed; m represents the number of initial seeds; Represents the total area of ​​the radar sensing space to be measured in the i-th 3D point cloud in the seed.

[0031] In some embodiments, when the test coverage is semantic coverage, obtaining the test coverage of the current seed and the test coverage after adding the test seed by using the guide indicator processing includes the following steps:

[0032] The semantic coverage is obtained by the ratio of the number of equivalence classes generated by the fuzzy test corresponding to the seed to the total number of equivalence classes of all possible perceptual semantics; the steps of generating equivalence classes include: generating a corresponding scene graph for each seed, the scene graph adopts a directed graph, simplifying the scene graph to generate an abstract scene graph, clustering the abstract scene graph to generate equivalence classes, the equivalence class refers to a collection of abstract scene graphs with the same structure and connection relationship, and finally calculating the number of equivalence classes;

[0033] The expression of semantic coverage is:

[0034]

[0035] In the formula, |ASG S (T)| represents the number of equivalence classes generated by fuzz testing, Represents the total number of equivalence classes for all possible semantically aware classes generated by the fuzz test.

[0036] To achieve the above object, another aspect of an embodiment of the present invention provides a fuzzy testing device for an autonomous driving perception system, the device comprising:

[0037] The first module is used to obtain initial seeds and construct a seed queue using the initial seeds; wherein the initial seeds are obtained based on a preset data set in the field of autonomous driving, and the seeds in the seed queue represent a three-dimensional point cloud generated by a lidar system;

[0038] The second module is used to select a preset number of current seeds from the seed queue and perform mutation operations on the current seeds to generate test seeds;

[0039] The third module is used to input the test seed into the deep neural network of the preset autonomous driving perception system for test prediction, and output the prediction result as the test seed of the failed test to the failure set; the test coverage of the current seed and the test coverage after adding the test seed are obtained by using the guidance indicator processing, and if the test coverage after adding the test seed is improved compared with the test coverage of the current seed, the test seed is added to the seed queue; wherein the guidance indicator includes spatial coverage and semantic coverage;

[0040] The fourth module is used to add 1 to the number of iterations, return to execute the steps of the second module, until the number of iterations reaches a preset maximum number of iterations, and output a failure set; wherein the number of iterations is initially 0.

[0041] In some embodiments, the apparatus further comprises:

[0042] A fifth module is used to update the selection probability of each seed in the seed queue based on the updated seed queue;

[0043] Among them, the expression of the updated selection probability is:

[0044] P(s)=1 / g(ori(s)) 2

[0045] Where ori(s) represents the initial seed corresponding to the seed scene, and g(ori(s)) represents the number of test seeds generated by mutation of the initial seed.

[0046] In some embodiments, the apparatus further comprises:

[0047] The sixth module is used to calculate the FRD score of each test seed and discard the test seeds whose FRD scores are less than a preset score threshold;

[0048] The expression of FRD score is:

[0049] FRD(X,Y)=||μ X -μ Y || 2 +Tr(∑ X +∑ Y -2(∑ X ∑ Y ) 1 / 2 )

[0050] Where X represents the initial seed, Y represents the test seed corresponding to X, FRD(X,Y) represents the FRD score corresponding to Y; μ represents the mean vector of data activations, ‖·‖ represents the L2 norm of the vector; ∑ represents the covariance matrix of data activations, and Tr represents the trace of the covariance matrix.

[0051] To achieve the above objective, another aspect of an embodiment of the present invention provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above method when executing the computer program.

[0052] To achieve the above objective, another aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0053] The embodiments of the present invention include at least the following beneficial effects: the present invention provides a fuzzy testing method, device and storage medium for an autonomous driving perception system, the scheme obtains an initial seed and constructs a seed queue using the initial seed; wherein the initial seed is obtained based on a preset data set in the autonomous driving field, and the seeds in the seed queue represent the three-dimensional point cloud generated by the laser radar system; a preset number of current seeds are selected from the seed queue, and a mutation operation is performed on the current seeds to generate test seeds; the test seeds are input into a preset deep neural network of the autonomous driving perception system for test prediction, and the test seeds with the prediction result of a failed test are output to a failure set; the test coverage of the current seed and the test coverage after adding the test seed are obtained by processing with a guiding indicator, and if the test coverage after adding the test seed is improved compared with the test coverage of the current seed, the test seed is added to the seed queue; wherein the guiding indicator includes a spatial coverage rate and a semantic coverage rate; the number of iterations is increased by 1, and the step of selecting a preset number of current seeds from the seed queue is returned to execute until the number of iterations reaches a preset maximum number of iterations, and a failure set is output; wherein the number of iterations is initially 0. The present invention designs a fuzzy testing framework for an autonomous driving perception system of a laser radar, and the present invention uses a mutation operation to generate test data and perform automated testing. In addition, the present invention introduces two indicators, spatial coverage and semantic coverage, to guide the testing process, with the aim of thoroughly searching for potential defects in the lidar-based perception model by considering the scene semantics and the spatial distribution of obstacles. The present invention can efficiently implement fuzzy testing of autonomous driving perception systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of a fuzzy testing method for an autonomous driving perception system provided by an embodiment of the present invention;

[0055] Figure 2 is an overall flow chart of a fuzzy testing method for an autonomous driving perception system provided by an embodiment of the present invention;

[0056] Figure 3 It is a flowchart of a specific program execution of a fuzzy testing method for an autonomous driving perception system provided by an embodiment of the present invention;

[0057] Figure 4 is a flow chart of calculating the FRD score provided by an embodiment of the present invention;

[0058] Figure 5 is a flow chart of equivalence class generation provided by an embodiment of the present invention;

[0059] Figure 6 is a schematic diagram of the structure of a fuzzy testing device for an autonomous driving perception system provided by an embodiment of the present invention;

[0060] Figure 7It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention. They are only examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the attached claims.

[0062] It is understood that the terms "first", "second", etc. used in the present invention can be used to describe various concepts in the present invention, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0063] The terms "at least one", "multiple", "each", "any", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0064] Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the present invention are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0065] The fuzzy testing method of the autonomous driving perception system provided by the embodiment of the present invention relates to the field of data processing technology. The fuzzy testing method of the autonomous driving perception system provided by the embodiment of the present invention can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a vehicle-mounted terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application of the fuzzy testing method for the autonomous driving perception system, etc., but is not limited to the above forms.

[0066] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0067] Figure 1 is an optional flowchart of a fuzzy testing method for an autonomous driving perception system provided by an embodiment of the present invention. Figure 1 The method may include but is not limited to steps S100 to S400.

[0068] S100, obtaining an initial seed, and using the initial seed to construct a seed queue;

[0069] The initial seeds are obtained based on a preset data set in the field of autonomous driving, and the seeds in the seed queue represent the three-dimensional point cloud generated by the lidar system;

[0070] For example, in some specific implementations, an initial seed is first obtained, and then a seed queue is generated using the initial seed, where the seed refers to a three-dimensional point cloud generated by a laser radar system. In addition, a pre-set laser radar-based autonomous driving perception system DNN model can also be obtained.

[0071] In some specific application scenarios, the KITTI dataset can be used as input. KITTI (Karlsruhe Institute of Technology and Toyota Institute) is a well-known dataset in the field of autonomous driving research. The dataset includes a large amount of driving data obtained from various sensors (such as laser scanners and color cameras) in real driving scenarios. Specifically, KITTI uses the Velodyne HDL-64E rotating 3D laser scanner as part of its sensor setup. Each data point is recorded with its (x, y, z) coordinates and an additional reflectivity value (r). The main purpose of this dataset is the 3D object detection benchmark, including 3712 training images, 3769 verification images, 7518 test images and their associated point clouds. S200, select a preset number of current seeds from the seed queue, and perform a mutation operation on the current seeds to generate test seeds;

[0072] It should be noted that, in some embodiments, selecting a preset number of current seeds from the seed queue may include the following steps: randomly selecting a preset number of seeds from the seed queue as the current seeds using a seed selection strategy.

[0073] For example, in some specific implementations, a seed selection strategy may be used to randomly select k seeds from a seed queue.

[0074] In some embodiments, after the seed queue is updated according to the test seed, the method may further include: updating the selection probability of each seed in the seed queue based on the updated seed queue; wherein the expression of the updated selection probability is:

[0075] P(s)=1 / g(ori(s)) 2

[0076] Where ori(s) represents the initial seed corresponding to the seed scene, and g(ori(s)) represents the number of test seeds generated by mutation of the initial seed.

[0077] For example, in some specific implementations, the seed selection strategy may further include: recording the original seed corresponding to each mutated seed in a seed set, recording a variable for each original seed, and the variable represents the number of variants of the seed in the seed set, selecting the seed according to probability, and the probability calculation formula for each seed being selected is: P(s) = 1 / g(ori(s)) 2 , where ori(s) represents the initial seed corresponding to the seed scene, and g(ori(s)) represents the total number of seeds generated after the initial seed conversion.

[0078] It should also be noted that, in some embodiments, performing a mutation operation on the current seed to generate a test seed may include the following steps: randomly selecting a mutation operator for each current seed and performing the mutation operation in sequence until the number of mutation operations exceeds the preset maximum number of attempts, and obtaining the test seed corresponding to each current seed according to the execution results of all mutation operations; wherein the mutation operator includes the conversion of the object and the addition of severe weather conditions. Specifically, the conversion of the object includes translation processing, rotation processing, insertion processing and scaling processing, and the severe weather conditions include rain, snow and fog.

[0079] For example, in some specific implementations, each seed may be mutated for no more than a maximum number of attempts tryMax, and a mutation operator is randomly selected for each mutation to perform the mutation operation, and then a new set of test seeds is generated. Specifically, the mutation operator includes object transformation and adverse weather conditions, the object transformation includes translation, rotation, insertion and scaling, and the adverse weather conditions include rain, snow and fog.

[0080] In some embodiments, after the step of performing a mutation operation on the current seed to generate a test seed, the method may further include the following steps: calculating the FRD score of each test seed, and discarding the test seed whose FRD score is less than a preset score threshold; wherein the expression of the FRD score is:

[0081] FRD(X,Y)=||μ X -μ Y || 2 +Tr(∑ X +∑ Y -2(∑ X ∑ Y ) 1 / 2 )

[0082] Where X represents the initial seed, Y represents the test seed corresponding to X, FRD(X,Y) represents the FRD score corresponding to Y; μ represents the mean vector of data activations, ‖·‖ represents the L2 norm of the vector; ∑ represents the covariance matrix of data activations, and Tr represents the trace of the covariance matrix.

[0083] For example, in some specific implementations, the FRD score of each test seed may be calculated. If the FRD score corresponding to the test seed is less than a threshold value t, the test seed is retained; otherwise, the test seed is discarded. The specific calculation formula is as follows:

[0084] FRD(X,Y)=||μ X -μ Y || 2 +Tr(∑ X +∑ Y -2(∑ X ∑ Y ) 1 / 2 )

[0085] Where X and Y represent the network activation distribution of real data and synthetic data respectively, real data refers to the initial seed, synthetic data refers to the mutated seed generated by the mutation of the initial seed, μ represents the mean vector of data activation, and ∑ is the covariance matrix of data activation. The trace of the matrix is ​​represented by Tr, and ‖·‖ represents the L2 norm of the vector.

[0086] S300, inputting the test seed into the preset deep neural network of the autonomous driving perception system for test prediction, and outputting the prediction result as the test seed of the failed test to the failure set; using the guiding indicator to process to obtain the test coverage of the current seed and the test coverage after adding the test seed, if the test coverage after adding the test seed is improved compared with the test coverage of the current seed, then adding the test seed to the seed queue;

[0087] Among them, the guiding indicators include spatial coverage and semantic coverage;

[0088] It should be noted that, in some embodiments, when the test coverage is the spatial coverage, the test coverage of the current seed and the test coverage after adding the test seed are obtained by using the guiding indicator processing, which may include the following steps: the ratio of the sum of the areas corresponding to the objects to be detected in each three-dimensional point cloud contained in the seed to the total area of ​​the radar-sensed space to be tested is cumulatively summed and averaged to obtain the spatial coverage; wherein the expression of the spatial coverage is:

[0089]

[0090] In the formula, SPC(T) represents the spatial coverage; Avg represents the average processing; represents the sum of the areas corresponding to the objects to be detected in the i-th 3D point cloud in the seed; m represents the number of initial seeds; Represents the total area of ​​the radar sensing space to be measured in the i-th 3D point cloud in the seed.

[0091] For example, in some specific implementations, the spatial coverage calculation formula is: Among them, for the point cloud data c1···c n , which is the data iteratively generated by the initial seed c0 during the fuzzification process. i Represents point cloud c i The object to be detected is inside the box, and PR is the area of ​​the three-dimensional truth box projected onto the two-dimensional plane where the x-axis and y-axis are located; It represents the area of ​​the nth object to be detected projected from the three-dimensional truth box onto the two-dimensional plane where the x-axis and y-axis are located. Given the initial seed c0, the corresponding LiDAR perception area can be expressed as the sum of the areas of all objects to be detected, that is: For seeds c i The corresponding LiDAR sensing area can be expressed as the sum of the areas of all objects to be detected. It is the total area of ​​the space to be measured by radar perception.

[0092] It should also be noted that in some embodiments, when the test coverage is semantic coverage, the test coverage of the current seed and the test coverage after adding the test seed are obtained by using the guiding indicator processing, which may include the following steps: obtaining the semantic coverage according to the ratio of the number of equivalence classes generated by the fuzzy test corresponding to the seed to the total number of equivalence classes of all possible perceptual semantics; the step of generating the equivalence class includes: generating a corresponding scene graph for each seed, the scene graph adopts a directed graph, simplifying the scene graph to generate an abstract scene graph, clustering the abstract scene graph to generate an equivalence class, the equivalence class refers to a set of abstract scene graphs with the same structure and connection relationship, and finally calculating the number of equivalence classes; wherein the expression of the semantic coverage is:

[0093]

[0094] In the formula, |ASG S (T)| represents the number of equivalence classes generated by fuzz testing, Represents the total number of equivalence classes for all possible semantically aware classes generated by the fuzz test.

[0095] For example, in some specific implementations, the calculation formula for semantic coverage is: |ASG S (T)| represents the number of equivalence classes generated by fuzz testing, Represents the total number of all possible semantically perceptual equivalence classes generated by the initial seed during the fuzz testing process.

[0096] S400, adding 1 to the number of iterations, returning to the step of selecting a preset number of current seeds from the seed queue, until the number of iterations reaches a preset maximum number of iterations, and outputting a failure set.

[0097] The number of iterations is initially 0.

[0098] Exemplarily, in some specific implementations, the operations of steps S200 and S300 are repeated until a termination condition is met to terminate the test. The termination condition is to set a maximum number of iterations (for example, the experiment is set to 1000). When the number of iterations exceeds iterations, the iteration is terminated.

[0099] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0100] First of all, it is important to note that state-of-the-art testing techniques have proven that autonomous driving systems are not robust to synthetic images of driving scenes. DeepTest takes a systematic approach to automatically generate test cases by transforming labeled driving scene images by applying various effect filters (such as fog / rain and simple affine transformations) to the original images and checking whether the system behaves consistently in the original and transformed scenes. DeepRoad proposes an unsupervised deep neural network (DNN)-based framework for automatically testing DNN-based autonomous driving systems. By applying generative adversarial networks (GANs) along with corresponding real-world weather scenarios, more realistic driving scenarios can be generated in various weather conditions. In addition, DeepBillBoard is a systematic physical world testing approach that focuses on generating a sufficiently robust and resilient printable physical world adversarial billboard test.

[0101] Although the above methods have successfully discovered various erroneous behaviors in autonomous driving systems, few people have paid attention to the impact of point clouds collected by lidar under harsh environmental conditions. Lidar is one of the most critical sensors in autonomous driving systems. It calculates the distance to the target by emitting lasers and processing laser echoes. Specifically, it calculates the distance to the target by emitting laser pulses and measuring the time required for these pulses to reflect and return from the target. Lidar sensors are widely used in 3D object detection, acquiring 3D scene information in the form of irregular and sparse point clouds, and providing an important solution for 3D scene perception and understanding. In addition, it provides highly accurate distance measurement and more reliable detection performance.

[0102] In addition, unlike cameras, it is not affected by light and works well at night. However, it is well known that these sensors are sensitive to adverse conditions. For the above reasons, it is crucial to design an automated testing method for autonomous driving software based on LiDAR. First, in the operating environment of the autonomous driving system, the point cloud perceived by LiDAR may be affected by various environmental factors, including bad weather (such as fog, rain, snow, etc.). The perceived point cloud or various types of noise information may be attenuated to varying degrees, resulting in erroneous behavior of the autonomous driving system during operation. And the integration of specific semantic information in the scene may cause the failure of the LiDAR-based perception system, which may lead to traffic accidents. Secondly, the testing process of modern autonomous driving systems relies heavily on the collected datasets. In actual road tests, point clouds are usually collected by LiDAR. Due to the diversity of environmental conditions, collecting point clouds under different environmental factors requires a lot of resource consumption. At the same time, the sparse distribution of scanning points in three-dimensional space complicates the extraction of target features over a large area.

[0103] Finally, manual annotation of point clouds is a laborious and time-consuming task, requiring a large amount of manpower to manually check the point cloud visualization results.

[0104] In view of this, and in view of the shortcomings of the prior art, the present invention provides a fuzzy testing method for an autonomous driving perception system based on a laser radar. In some specific application scenarios, the technical solution of the present invention can be implemented as follows:

[0105] The embodiment of the present invention can implement fuzzy testing of the lidar-based autonomous driving perception system on Python 3.7 and Pytorch 1.13.1. All tests of the lidar-based perception system are built on an Ubuntu 20.04.6 LTS server, which uses NVIDIA Geforce RTX 4070Ti and 12GB video memory.

[0106] The embodiments of the present invention can use the KITTI dataset as input. KITTI (Karlsruhe Institute of Technology and Toyota Institute) is a well-known dataset in the field of autonomous driving research. The dataset includes a large amount of driving data obtained from various sensors (such as laser scanners and color cameras) in real driving scenarios. Specifically, KITTI uses the Velodyne HDL-64E rotating 3D laser scanner as part of its sensor setup. Each data point is recorded with its (x, y, z) coordinates and an additional reflectivity value (r). The main purpose of this dataset is the 3D object detection benchmark, including 3712 training images, 3769 verification images, 7518 test images and their associated point clouds.

[0107] In order to effectively evaluate the performance of the test method of the embodiment of the present invention, the present invention can select four most advanced lidar perception systems as experimental objects.

[0108] Here is a brief introduction to each system:PointPillars is a 3D object detection encoder system that uses PointNets to learn representations of point clouds organized in vertical columns. The model provides superior detection performance at a faster speed, surpassing existing solutions. Notably, it is widely used in the open source autonomous driving systems Apollo and Autoware.

[0109] The SECOND system combines enhanced sparse convolutions to greatly speed up training and inference. In addition, a new angle loss regression method is introduced to improve direction estimation performance, and a new data augmentation method is introduced to speed up convergence and improve overall performance.

[0110] The PointRCNN system consists of two stages: the initial subnet segments the entire point cloud of the scene into foreground and background points through a bottom-up approach, directly producing a limited number of high-quality 3D results. Subsequently, the next level of subnet refines the 3D results under canonical coordinates to obtain the final detection results.

[0111] PV-RCNN is a point-voxel based object detection system that compresses 3D scenes using 3D voxel convolutional neural networks (CNNs) into a simplified set of key points through a unique voxel set abstraction module, reducing the amount of subsequent computation and capturing representative scene features. In addition, the voxel-to-keypoint scene encoding and keypoint-to-grid RoI feature abstraction methods in this framework significantly improve 3D object detection performance compared to previous methods.

[0112] In order to evaluate the effectiveness of the fuzzy testing method in the embodiment of the present invention in testing the lidar-based autonomous driving perception system, the embodiment of the present invention adopts OpenPCDet as the solution for evaluating the testing technology in this study. OpenPCDet is a user-friendly, self-contained open source platform for lidar-based 3D object detection. The default parameter configuration is used in the embodiment of the present invention, and the pre-trained model is accessed from the model repository.

[0113] The embodiment of the present invention builds a fuzzy testing framework for the autonomous driving perception system based on LiDAR. Figure 2 and Figure 3 As shown, it is a flowchart of the execution of the fuzzy testing method of the laser radar-based autonomous driving perception system, which includes the following steps:

[0114] Step 1 (corresponding to Figure 2 S110): an initial seed is input, the embodiment of the present invention uses point cloud data in the KITTI data set, and also includes a DNN model of an autonomous driving perception system based on a laser radar. The embodiment of the present invention uses PointPillars, SECOND, PointRCNN and PV-RCNN, and uses the initial seed to generate a seed queue, where the seed refers to a three-dimensional point cloud generated by the laser radar system;

[0115] Step 2 (corresponding to Figure 2 S120): k seeds are randomly selected from the seed queue using a seed selection strategy, and mutation is performed on each seed for no more than the maximum number of attempts tryMax. A mutation operator is randomly selected for each mutation to perform the mutation operation, and then a new set of test seeds is generated; and the FRD score of each test seed is calculated. If the FRD score corresponding to the test seed is less than the threshold t, it is retained, otherwise it is discarded;

[0116] Specifically, the mutation operators include object transformation and adverse weather conditions. Object transformation includes translation, rotation, insertion and scaling. Adverse weather conditions include rain, snow and fog.

[0117] Optionally, the seed selection strategy further includes: recording the original seed corresponding to each mutated seed in a seed set, recording a variable for each original seed, and the variable represents the number of variants of the seed in the seed set, selecting the seed according to probability, and the probability calculation formula of each seed being selected is: P(s) = 1 / g(ori(s)) 2 , where ori(s) represents the initial seed s corresponding to the seed scene, and g(s0) represents the total number of seeds generated after conversion from the initial seed s0;

[0118] Among them, Figure 4 As shown, this is a flowchart for calculating the FRD score:

[0119] The steps to calculate the FRD score include: 1) Feature extraction: Use the pre-trained lidar perception network to extract features from the lidar point cloud samples, and randomly select a certain number of feature activations from the bottleneck layer of the perception network; 2) Gaussian distribution fitting: Fit the feature activations with a Gaussian distribution so that the statistical differences between them can be calculated; 3) Calculate the mean and covariance: For the feature activations of synthetic samples and real samples, calculate their means and covariances respectively; 4) Calculate the Wasserstein distance (also known as the Earth Mover's Distance (EMD), which is a method to measure the difference between two probability distributions): Use the Wasserstein distance to quantify the distance between two Gaussian distributions, and calculate the Wasserstein distance between the mean and covariance of the feature activations of synthetic samples and real samples. 5) Calculate the FRD score: Square the Wasserstein distance, and this value is the FRD score. The specific calculation formula is as follows:

[0120] FRD(X,Y)=||μ X -μ Y || 2 +Tr(∑ X +∑ Y -2(∑ X ∑ Y ) 1 / 2 )

[0121] Where X and Y represent the network activation distribution of real data and synthetic data respectively, real data refers to the initial seed, synthetic data refers to the mutated seed generated by the mutation of the initial seed, μ represents the mean vector of data activation, and ∑ is the covariance matrix of data activation. The trace of the matrix is ​​represented by Tr, and ‖·‖ represents the L2 norm of the vector.

[0122] Step 3 (corresponding to Figure 2 S130): Calculate the test coverage using the guidance indicator, and input the test seed into the DNN model of the lidar-based autonomous driving perception system for prediction. For each test seed, if it is a failed test, it is added to the failed set. If the test seed improves the test coverage, it is added to the seed queue. Seeds that do not meet the above two conditions will be discarded. Specifically, the guidance indicator includes spatial coverage and semantic coverage.

[0123] The spatial coverage calculation formula is: Among them, for the point cloud data c1···c n , which is the data iteratively generated by the initial seed c0 during the fuzzification process. i Represents point cloud c iThe object to be detected is inside the box, and PR is the area of ​​the three-dimensional truth box projected onto the two-dimensional plane where the x-axis and y-axis are located; It represents the area of ​​the nth object to be detected projected from the three-dimensional truth box onto the two-dimensional plane where the x-axis and y-axis are located. Given the initial seed c0, the corresponding LiDAR perception area can be expressed as the sum of the areas of all objects to be detected, that is: For seeds c i The corresponding LiDAR sensing area can be expressed as the sum of the areas of all objects to be detected. The total area of ​​the space to be measured by radar perception;

[0124] The calculation formula of semantic coverage is: |ASG S (t)| represents the number of equivalence classes generated by fuzz testing, Represents the total number of all possible semantically perceptual equivalence classes generated by the initial seed during the fuzz testing process; Figure 5 As shown in the figure, it is a flowchart generated by equivalence class. The specific process is as follows:

[0125] The steps of generating equivalence classes include: generating a corresponding scene graph for each seed, the scene graph uses a directed graph, each node is a participant in the driving scene (for example: road, vehicle, pedestrian, weather, etc.), and there is a connection relationship between nodes (for example: existence, connection, orientation, inclusion, weather effect, etc.), simplifying the scene graph to generate an abstract scene graph, for example, the scene graph uses different identifiers, the car 'car_1' on the left road and the car 'car_2' on the right road, these identifiers have nothing to do with the positional relationship between the vehicles, if re-labeled as "car", their semantic relationship remains unchanged. Clustering the abstract scene graph to generate equivalence classes, abstract scene graphs with the same structure and connection relationship will correspond to the same equivalence class, that is: if multiple abstract scene graphs are isomorphic graphs, they are marked as the same equivalence class, and finally the number of equivalence classes is calculated.

[0126] Step 4 (corresponding to Figure 2 In S140), the operation of step 2 and step 3 is repeated until the termination condition is met and the test is terminated. The termination condition is to set a maximum number of iterations. When the number of iterations exceeds iterations, the iteration is terminated.

[0127] The embodiments of the present invention are evaluated on four state-of-the-art LiDAR-based perception systems. The experimental results show that the transformations applied by the test method of the embodiments of the present invention can detect a large number of erroneous behaviors in LiDAR-based autonomous driving perception systems. In addition, the proposed test criteria can be used to improve test efficiency and diversity. In the case study, we analyzed the types of errors in the LiDAR-based perception system and believed that the fuzzy testing method of the embodiments of the present invention provides a basis for the testing and analysis development of autonomous driving systems (ADS systems).

[0128] In summary, this paper designs a fuzzy testing framework for LiDAR-based autonomous driving perception systems, which uses different mutation operators to generate a considerable amount of test data and perform automated testing. In addition, we introduce two metrics, spatial coverage and semantic coverage, to guide the testing process, with the goal of thoroughly searching for potential defects in LiDAR-based perception models by considering scene semantics and the spatial distribution of obstacles.

[0129] Compared with the prior art, the present invention has at least the following beneficial effects:

[0130] 1. The fuzzy testing framework of the laser radar-based autonomous driving perception system of the present invention adopts a variety of mutation operators and seed selection strategies to automatically generate test cases and test the laser radar perception system, which can effectively reveal

[0131] Given sufficient testing time, various defects hidden in the LiDAR perception system can be discovered that are difficult to detect with other testing schemes.

[0132] 2. Currently, the existing data sets contain fewer test cases. The fuzzy testing framework of the lidar-based autonomous driving perception system of the present invention can automatically generate more test cases, and the differences between the test cases are large, providing diversified tests and avoiding the large amount of resources consumed by traditional lidar to collect point cloud data.

[0133] 3. The present invention also proposes two guiding indicators, including spatial coverage and semantic coverage. These indicators can guide the fuzzy testing framework to develop in the direction of discovering more defects, thereby discovering more potential defects in the lidar-based autonomous driving perception system. At the same time, these indicators also indicate the adequacy of the testing process, allowing relevant personnel to intuitively understand the completeness of the test.

[0134] 4. The present invention can realize automatic annotation of point clouds, avoid a lot of repetitive work of manual annotation of point clouds, and improve accuracy.

[0135] like Figure 6As shown, the embodiment of the present invention further provides a fuzzy testing device 900 for an autonomous driving perception system, which may include:

[0136] The first module 901 is used to obtain initial seeds and construct a seed queue using the initial seeds; wherein the initial seeds are obtained based on a preset data set in the field of autonomous driving, and the seeds in the seed queue represent a three-dimensional point cloud generated by a laser radar system;

[0137] The second module 902 is used to select a preset number of current seeds from the seed queue and perform mutation operations on the current seeds to generate test seeds;

[0138] The third module 903 is used to input the test seed into the preset deep neural network of the autonomous driving perception system for test prediction, and output the prediction result as the test seed of the failed test to the failure set; use the guidance indicator to process to obtain the test coverage of the current seed and the test coverage after adding the test seed, if the test coverage after adding the test seed is improved compared with the test coverage of the current seed, then add the test seed to the seed queue; wherein the guidance indicator includes spatial coverage and semantic coverage;

[0139] The fourth module 904 is used to add 1 to the number of iterations, return to execute the steps of the second module, until the number of iterations reaches a preset maximum number of iterations, and output a failure set; wherein the number of iterations is initially 0.

[0140] In some embodiments, the apparatus may further include:

[0141] A fifth module is used to update the selection probability of each seed in the seed queue based on the updated seed queue;

[0142] Among them, the expression of the updated selection probability is:

[0143] P(s)=1 / g(ori(s)) 2

[0144] Where ori(s) represents the initial seed corresponding to the seed scene, and g(ori(s)) represents the number of test seeds generated by mutation of the initial seed.

[0145] In some embodiments, the apparatus may further include:

[0146] The sixth module is used to calculate the FRD score of each test seed and discard the test seeds whose FRD scores are less than a preset score threshold;

[0147] The expression of FRD score is:

[0148] FRD(X,Y)=||μ X -μY || 2 +Tr(∑ X +∑ Y -2(∑ X ∑ Y ) 1 / 2 )

[0149] Where X represents the initial seed, Y represents the test seed corresponding to X, FRD(X,Y) represents the FRD score corresponding to Y; μ represents the mean vector of data activations, ‖·‖ represents the L2 norm of the vector; ∑ represents the covariance matrix of data activations, and Tr represents the trace of the covariance matrix.

[0150] The contents of the method embodiments of the present invention are all applicable to the device embodiments. The functions specifically implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0151] The embodiment of the present invention further provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned fuzzy testing method of the autonomous driving perception system when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0152] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0153] See also Figure 7 , Figure 7 The hardware structure of an electronic device 1000 of another embodiment is illustrated. The electronic device 1000 includes:

[0154] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0155] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the fuzzy testing method of the autonomous driving perception system of the embodiment of the present invention;

[0156] Input / output interface 1003, used to implement information input and output;

[0157] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0158] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0159] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0160] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the fuzzy testing method of the above-mentioned autonomous driving perception system.

[0161] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0162] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0163] The fuzzy testing method, fuzzy testing device, electronic device and storage medium of the autonomous driving perception system provided by the embodiment of the present invention obtains the initial seed and constructs the seed queue by using the initial seed; wherein the initial seed is obtained based on the preset data set of the autonomous driving field, and the seed in the seed queue represents the three-dimensional point cloud generated by the laser radar system; a preset number of current seeds are selected from the seed queue, and the current seeds are mutated to generate test seeds; the test seeds are input into the preset deep neural network of the autonomous driving perception system for test prediction, and the test seeds with the prediction result of the failed test are output to the failure set; the test coverage of the current seed and the test coverage after adding the test seed are obtained by processing with the guiding indicator, and if the test coverage after adding the test seed is improved compared with the test coverage of the current seed, the test seed is added to the seed queue; wherein the guiding indicator includes the spatial coverage and the semantic coverage; the number of iterations is increased by 1, and the step of selecting the preset number of current seeds from the seed queue is returned to execute until the number of iterations reaches the preset maximum number of iterations, and the failure set is output; wherein the number of iterations is initially 0. The present invention designs a fuzzy testing framework for the autonomous driving perception system of the laser radar, and the present invention uses mutation operations to generate test data and perform automated testing. In addition, the present invention introduces two indicators, spatial coverage and semantic coverage, to guide the testing process, with the aim of thoroughly searching for potential defects in the lidar-based perception model by considering the scene semantics and the spatial distribution of obstacles. The present invention can efficiently implement fuzzy testing of autonomous driving perception systems.

[0164] The embodiments described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art can appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0165] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present invention and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0166] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0167] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0168] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0169] It should be understood that in the present invention, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0170] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0171] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0172] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0173] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store programs.

[0174] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the embodiments of the present invention is not limited thereby. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present invention shall be within the scope of the rights of the embodiments of the present invention.

Claims

1. A fuzzy testing method for an autonomous driving perception system, characterized in that: The method comprises the following steps: Obtaining initial seeds, and using the initial seeds to construct a seed queue; wherein the initial seeds are obtained based on a preset data set in the field of autonomous driving, and the seeds in the seed queue represent a three-dimensional point cloud generated by a lidar system; Selecting a preset number of current seeds from the seed queue, and performing mutation operations on the current seeds to generate test seeds; Input the test seed into a preset deep neural network of an autonomous driving perception system for test prediction, and output the test seed with a prediction result of a failed test to a failure set; use the guidance indicator to process to obtain the test coverage of the current seed and the test coverage after adding the test seed, and if the test coverage after adding the test seed is improved compared to the test coverage of the current seed, then add the test seed to the seed queue; wherein the guidance indicator includes spatial coverage and semantic coverage; Add 1 to the number of iterations, return to execute the step of selecting a preset number of current seeds from the seed queue, until the number of iterations reaches a preset maximum number of iterations, and output the failure set; wherein the number of iterations is initially 0.

2. The fuzzy testing method for the autonomous driving perception system according to claim 1, characterized in that: The step of selecting a preset number of current seeds from the seed queue comprises the following steps: A seed selection strategy is adopted to randomly select the preset number of seeds from the seed queue as current seeds.

3. The fuzzy testing method of the autonomous driving perception system according to claim 1, characterized in that: After the seed queue is updated according to the test seed, the method further comprises the following steps: Update the selection probability of each seed in the seed queue based on the updated seed queue; Among them, the expression of the updated selection probability is: P(s)=1 / g(ori(s)) 2 In the formula, ori(s) represents the initial seed corresponding to the seed scene, and g(ori(s)) represents the number of test seeds generated by mutation of the initial seed.

4. The fuzzy testing method of the autonomous driving perception system according to claim 1, characterized in that: The step of performing a mutation operation on the current seed to generate a test seed comprises the following steps: Randomly selecting a mutation operator for each current seed to perform the mutation operation in sequence until the number of mutations of the mutation operation exceeds a preset maximum number of attempts, and obtaining the test seed corresponding to each current seed according to the execution results of all the mutation operations; The mutation operator includes the transformation of objects and the addition of severe weather conditions.

5. The fuzzy testing method of the autonomous driving perception system according to claim 1, characterized in that: After the step of performing a mutation operation on the current seed to generate a test seed, the method further comprises the following steps: Calculate the FRD score of each of the test seeds, and discard the test seeds whose FRD scores are less than a preset score threshold; Wherein, the expression of the FRD score is: FRD(X,Y)=||μ X -μ Y || 2 +Tr(∑ X +∑ Y -2(∑ X Z Y ) 1 / 2 ) Wherein, X represents the initial seed, Y represents the test seed corresponding to X, FRD(X,Y) represents the FRD score corresponding to Y; μ represents the mean value vector of data activation, ‖·‖ represents the L2 norm of the vector; ∑ represents the covariance matrix of data activation, and Tr represents the trace of the covariance matrix.

6. The fuzzy testing method of the autonomous driving perception system according to claim 1, characterized in that: When the test coverage is the spatial coverage, the process of using the guide indicator to obtain the test coverage of the current seed and the test coverage after adding the test seed includes the following steps: The ratio of the sum of the areas corresponding to the objects to be detected in each of the three-dimensional point clouds contained in the seed to the total area of ​​the space to be detected sensed by the radar is accumulated, summed and averaged to obtain the space coverage rate; Wherein, the expression of the space coverage is: Wherein, SPC(T) represents the spatial coverage; Avg represents the average processing; represents the sum of the areas corresponding to the object to be detected in the i-th three-dimensional point cloud in the seed; m represents the number of the initial seeds; Represents the total area of ​​the radar-sensed space to be measured in the i-th three-dimensional point cloud in the seed.

7. The fuzzy testing method for the autonomous driving perception system according to claim 1, characterized in that: When the test coverage is the semantic coverage, the process of using the guiding indicator to obtain the test coverage of the current seed and the test coverage after adding the test seed includes the following steps: The semantic coverage is obtained according to the ratio of the number of equivalence classes generated by the fuzzy test corresponding to the seed to the total number of equivalence classes of all possible perceptual semantics; the step of generating the equivalence classes comprises: generating a corresponding scene graph for each seed, the scene graph adopts a directed graph, simplifying the scene graph to generate an abstract scene graph, clustering the abstract scene graph to generate equivalence classes, the equivalence class refers to a collection of abstract scene graphs with the same structure and connection relationship, and finally calculating the number of equivalence classes; Wherein, the expression of the semantic coverage is: In the formula, |ASG S (T)| represents the number of equivalence classes generated by fuzz testing, Represents the total number of equivalence classes for all possible semantically aware classes generated by the fuzz test.

8. A fuzzy testing device for an autonomous driving perception system, characterized in that: The device comprises: The first module is used to obtain initial seeds and construct a seed queue using the initial seeds; wherein the initial seeds are obtained based on a preset data set in the field of autonomous driving, and the seeds in the seed queue represent a three-dimensional point cloud generated by a laser radar system; The second module is used to select a preset number of current seeds from the seed queue and perform mutation operations on the current seeds to generate test seeds; The third module is used to input the test seed into the deep neural network of the preset autonomous driving perception system for test prediction, and output the test seed with the prediction result of failed test to the failure set; use the guidance indicator to process to obtain the test coverage of the current seed and the test coverage after adding the test seed, if the test coverage after adding the test seed is improved compared with the test coverage of the current seed, then add the test seed to the seed queue; wherein the guidance indicator includes spatial coverage and semantic coverage; The fourth module is used to add 1 to the number of iterations, return to execute the steps of the second module, until the number of iterations reaches a preset maximum number of iterations, and output the failure set; wherein the number of iterations is initially 0.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images