Reliability testing method for autonomous driving perception system based on fault injection

Abstract

A fault injection-based reliability testing method for autonomous driving perception systems is provided, applicable to the field of autonomous vehicle technology. The method includes: constructing a key scenario library; matching test scenarios from the key scenario library based on the manufacturer's stated reliability testing requirements for perception system functions; analyzing the failure modes corresponding to the perception system functions in the test scenarios to determine the key sensors and fault signals to be injected with faults; injecting the fault signals into the key sensor signals to obtain faulty perception signals; testing the perception system based on the faulty perception signals to determine the reliability of the perception system functions; and finally, determining the overall reliability of the perception system and its reliability level. By injecting faults into key sensors in the test scenarios, the performance of the perception system under abnormal conditions is studied, effectively assessing its reliability.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving vehicle technology, and in particular to a reliability testing method, device, and electronic equipment for an autonomous driving perception system based on fault injection. Background Art

[0002] Autonomous driving perception systems use sensors such as cameras, LiDAR, millimeter-wave radar (radar), ultrasonic radar, real-time kinematic tracking (RTK), and inertial measurement units (IMUs) to perceive the environment and position information, providing critical support for autonomous vehicle decision-making and control. The ability of the perception system to reliably perform its perception functions directly impacts the proper operation of autonomous vehicles. Autonomous vehicles operate in complex and changing environments, requiring sensors to maintain their intended functionality despite environmental factors such as rain, fog, and dust, as well as sensor failures. To this end, manufacturers typically develop autonomous driving perception systems using multi-sensor information fusion technology. When some sensors experience problems (due to environmental influences or malfunctions), the perception system integrates data from other sensors using fusion algorithms to maintain normal perception and ensure system reliability. This multi-sensor information fusion mechanism is crucial for ensuring the reliability of perception systems, leading to an urgent need for industry-wide reliability assessment methods for perception systems in the face of partial sensor failures.

[0003] Fault injection technology is an effective method for evaluating the reliability of perception systems under sensor failures or environmental interference. By injecting faults, we can assess the performance of perception systems under potential failure conditions and environmental interference, which is crucial for assessing system reliability and identifying potential system vulnerabilities.

[0004] Current autonomous driving reliability testing is primarily conducted at the vehicle level. These methods do not specifically target the perception system, nor do they address reliability in the face of partial failures and environmental interference. Quantitative evaluation of the perception system makes it difficult to promptly identify potential flaws. Furthermore, vehicle-level testing requires autonomous vehicles, hindering the reliability evaluation of perception systems by perception system manufacturers. Furthermore, existing autonomous driving testing methods primarily rely on natural driving data. However, natural driving data contains a large number of non-critical scenarios, making it difficult to identify reliability issues in the perception system, resulting in inefficient testing.

[0005] Therefore, it is urgent to propose a reliability testing method for autonomous driving perception systems to efficiently and conveniently evaluate the reliability of the perception system under conditions of partial sensor failure or environmental interference, and to discover potential defects in the perception system. Summary of the Invention

[0006] (1) Technical issues to be resolved

[0007] To address at least one of the aforementioned technical issues encountered in prior art reliability testing of autonomous driving perception systems, embodiments of the present invention provide a method, apparatus, and electronic device for reliability testing of autonomous driving perception systems based on fault injection. By constructing a library of key scenarios and combining them with the manufacturer's stated reliability testing requirements for perception system functions, test scenarios are generated. Within these matching test scenarios, faults are injected into sensors associated with the claimed functions (critical sensors). The performance of the perception system under partial sensor failures and environmental interference conditions is tested, effectively assessing its reliability.

[0008] (2) Technical solution

[0009] In response to the above technical problems, embodiments of the present invention provide a reliability testing method, device, and electronic device for an autonomous driving perception system based on fault injection.

[0010] According to a first aspect of the present invention, a reliability testing method for an autonomous driving perception system based on fault injection is provided, comprising: constructing a key scenario library, wherein the key scenario library includes multiple key scenarios; matching corresponding key scenarios from the key scenario library as test scenarios and determining test pass criteria based on reliability test requirements of the perception system functions claimed by the manufacturer; analyzing the failure modes corresponding to the perception functions claimed by the manufacturer in the test scenarios, and determining key sensors and fault signals to be injected with faults; injecting the fault signals into corresponding fault-free key sensor signals to obtain sensor signals with faults; in the test scenarios, testing the perception system functions claimed by the manufacturer based on the fault-free key sensor signals and the sensor signals with faults, and obtaining the reliability of the perception system functions in the test scenarios based on the test pass criteria; and calculating the total reliability of the perception system and its reliability level based on the reliability of the perception system functions in each test scenario.

[0011] In some exemplary embodiments, constructing a key scenario library includes: obtaining historical accident data related to the autonomous driving perception system; cleaning and standardizing the historical accident data to obtain standardized accident data to ensure data consistency and integrity; extracting influencing factors of accidents based on the standardized accident data; combining the influencing factors to generate scenarios; evaluating the probability of occurrence of scenarios based on historical accident data; and selecting scenarios with a probability of occurrence higher than a preset probability to construct a key scenario library, wherein the historical accident data includes the environmental conditions, traffic conditions, human factors, and accident sequences of the accidents; and the key scenario library includes the environmental conditions, traffic conditions, human factors, accident sequences, and probability of occurrence of the scenarios.

[0012] In some exemplary embodiments, the method further includes: performing specific coding representation on the accident sequence to extract the process characteristics of the accident; using distance algorithm and clustering algorithm to analyze the similarity of the accident sequences, and grouping the accident sequences with similar process characteristics into one category to enhance the representativeness of key scenarios.

[0013] In some exemplary embodiments, the failure modes corresponding to the perception functions claimed by the manufacturer in the test scenarios are analyzed, and the key sensors and fault signals to be injected are determined, specifically including: identifying the causes of the functional failure of the perception system claimed by the manufacturer in the test scenarios, and determining the key sensors that cause potential functional failures; determining the failure modes of the key sensors that cause functional failures in each test scenario as the failure modes to be injected; and modeling the failure modes to generate corresponding fault signals.

[0014] In some exemplary embodiments, a fault signal is injected into a corresponding key sensor signal to obtain a sensor signal with a fault, specifically including: in a test scenario, real-time acquisition of multimodal sensor data of the autonomous driving perception system as initial fault-free sensor data; and superimposition of the fault signal with the initial fault-free sensor data to obtain a sensor signal with a fault, wherein parameters of the fault signal include signal type, intensity, duration, and triggering conditions; and the sensor signal with a fault can be output to the autonomous driving perception system in real time.

[0015] In some exemplary embodiments, the perception system function claimed by the manufacturer is tested based on fault-free key sensor signals and faulty sensor signals, and the reliability of the perception system function in the test scenario is obtained according to the test passing criteria, specifically including: in the test scenario, the perception system function claimed by the manufacturer is tested based on fault-free sensor signals to ensure that the perception system function claimed by the manufacturer in the initial state is normal; in the test scenario, the perception system function claimed by the manufacturer is tested based on faulty sensor signals, and according to the test passing criteria, the perception function test result after fault injection is obtained, wherein the test result is passed or failed; and in each test scenario, the ratio of the number of test results with a pass to the total number of tests on the perception system function claimed by the manufacturer based on the faulty sensor signals is calculated as the reliability of the perception system function claimed by the manufacturer in the test scenario.

[0016] In some exemplary embodiments, the failure modes include camera failure mode, lidar failure mode, ultrasonic radar failure mode, IMU failure mode, RTK failure mode, and millimeter wave radar failure mode.

[0017] In some exemplary embodiments, camera failure modes include: Gaussian image noise failure, salt and pepper image noise failure, gamma image noise failure, Poisson image noise failure, uniform image noise failure, Ruili image noise failure, rain and fog noise failure, intermittent image noise failure and disconnection failure; lidar failure modes include: high and low density noise failure in sandstorm weather, multiple density noise failure in rainy weather, disconnection failure and intermittent failure; ultrasonic radar failure modes include: random error distance failure, distance attenuation failure, disconnection failure and intermittent noise failure; IMU failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure; RTK failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure; and millimeter wave radar failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure.

[0018] According to a second aspect of the present invention, a reliability testing device for an autonomous driving perception system based on fault injection is provided, comprising: a construction module for constructing a key scenario library, wherein the key scenario library includes multiple key scenarios; a first acquisition module for matching corresponding key scenarios from the key scenario library as test scenarios and determining the test pass criteria based on the reliability test requirements of the perception system functions claimed by the manufacturer; a second acquisition module for analyzing the fault modes corresponding to the perception functions claimed by the manufacturer in the test scenarios, and determining the key sensors and fault signals to be injected with faults; a third acquisition module for injecting the fault signals into the corresponding fault-free key sensor signals to obtain sensor signals with faults; a testing module for testing the perception system functions claimed by the manufacturer based on the fault-free key sensor signals and the faulty sensor signals in the test scenarios, and obtaining the reliability of the perception system functions in the test scenarios according to the test pass criteria; and a calculation module for calculating the total reliability of the perception system and its reliability level based on the reliability of the perception system functions in each test scenario.

[0019] According to a third aspect of the present invention, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.

[0020] According to a fourth aspect of the present invention, a storage medium is provided, characterized in that: computer-readable instructions are stored in the storage medium, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the above method.

[0021] (3) Beneficial effects

[0022] As can be seen from the above technical solutions, the fault injection-based reliability testing method, device, and electronic device for an autonomous driving perception system provided by the embodiments of the present invention have at least the following beneficial effects:

[0023] (1) By collecting accident data of the perception system and analyzing the influencing factors, a key scenario library is constructed. Compared with the existing technology that uses a large number of non-critical scenario tests based on natural driving data or vehicle-level simulation tests, this application focuses on key scenarios that are prone to exposing unreliable problems of the perception system, ensuring that test resources are concentrated on discovering and solving potential hidden dangers of the perception system, shortening the test cycle, and improving test efficiency.

[0024] (2) Fault injection technology is used to simulate perception system failures and environmental interference factors, comprehensively covering the fault signals and environmental interference of multiple sensors, and quantitatively evaluating the reliability of the perception system functions claimed by manufacturers. Targeted improvement suggestions can be provided to improve the reliability level of the autonomous driving perception system.

[0025] (3) This application overcomes the limitations of traditional test environments by simulating environmental interference through an automated fault injection device. Furthermore, compared to actual hardware failures, software-level fault injection can avoid component damage.

[0026] (4) Independent reliability testing can be performed directly on the perception system without relying on the autonomous driving vehicle. Therefore, this application can significantly reduce testing costs and improve testing efficiency, providing an economical and convenient solution for reliability testing of autonomous driving perception systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0028] Figure 1 The following schematically illustrates a flow chart of a reliability testing method for an autonomous driving perception system based on fault injection according to an embodiment of the present invention;

[0029] Figure 2 A schematic diagram of a process for building a key scenario library according to an embodiment of the present invention is shown;

[0030] Figure 3 A schematic diagram of another process of building a key scenario library according to an embodiment of the present invention is shown;

[0031] Figure 4 The following schematically illustrates a flow chart of injecting a fault signal into a corresponding key sensor signal to obtain a sensor signal with a fault according to an embodiment of the present invention;

[0032] Figure 5A schematic diagram shows a structural block diagram of a device for implementing a fault injection mode of an autonomous driving perception system according to an embodiment of the present invention;

[0033] Figure 6 A block diagram schematically illustrates a reliability test device for an autonomous driving perception system based on fault injection according to an embodiment of the present invention; and

[0034] Figure 7 A block diagram of an electronic device for a reliability testing method of an autonomous driving perception system based on fault injection according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0035] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to specific embodiments and the accompanying drawings. It is apparent that the embodiments described are only a portion of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0036] Fault injection technology is an effective method for evaluating the reliability of perception systems under sensor failures or environmental interference. Using a device capable of generating different failure modes and simulating environmental interference factors such as rain, fog, and white balance imbalance, faults can be injected into key sensors to simulate abnormal conditions for testing. Introducing fault injection technology to evaluate the performance of perception systems under potential failure conditions and actual environmental interference factors is crucial for identifying system vulnerabilities and evaluating system reliability.

[0037] An embodiment of the present invention provides a reliability testing method for an autonomous driving perception system based on fault injection. The method first determines the key scenarios that affect the reliability of the perception system through data collection and influencing factor analysis. Then, using a fault injection device, a variety of fault injection modes are designed and implemented for multimodal sensors to simulate various faults and environmental interference factors that may be encountered in actual operation. In the test scenario, fault injection testing is performed on key sensors involved in the claimed function implementation to evaluate their performance under abnormal conditions, and reliability is calculated based on the test results. Compared with the existing technology based on a large number of non-critical scenario tests in natural driving data or vehicle-level simulation tests, this application does not require the provision of an autonomous driving vehicle, and the test is convenient, does not require large-scale testing, has a short test cycle, and is more efficient. At the same time, by using fault injection technology to simulate perception system failure problems and environmental interference factors, the reliability of the perception system is comprehensively evaluated, which can provide targeted improvement suggestions and improve the reliability level of the autonomous driving perception system.

[0038] Figure 1The present invention schematically illustrates a flow chart of a method for reliability testing of an autonomous driving perception system based on fault injection according to an embodiment of the present invention.

[0039] like Figure 1 As shown, a reliability testing method for an autonomous driving perception system based on fault injection according to an embodiment of the present invention includes steps S110-S160.

[0040] In step S110, a key scenario library is constructed, where the key scenario library includes multiple key scenarios. To effectively evaluate the reliability of the autonomous driving perception system in actual driving environments, it is first necessary to identify a set of key scenarios. These scenarios should be able to more effectively expose potential problems in the perception system.

[0041] In some exemplary embodiments, step S110 includes steps S111-S116, see Figure 2 .

[0042] In step S111, historical accident data related to the autonomous driving perception system is obtained, wherein the historical accident data includes environmental conditions, traffic conditions, human factors, and accident sequences where the accident occurred.

[0043] For example, historical accident data related to the autonomous driving perception system can be collected from authoritative traffic accident databases and accident reports of autonomous vehicles.

[0044] In step S112, the historical accident data is cleaned and standardized to obtain standardized accident data to ensure the consistency and integrity of the data.

[0045] For example, the collected data is cleaned to remove incomplete or inaccurate records, and the data is standardized to ensure that the data format of each factor is consistent to facilitate subsequent analysis.

[0046] In step S113 , factors influencing the occurrence of the accident are extracted based on the standardized accident data.

[0047] For example, identifying potential influencing factors: extracting factors that may affect the occurrence of accidents from the data, including environmental factors (such as weather conditions, lighting conditions, road types), traffic-related factors (such as traffic flow, vehicle speed range, road signs and signal complexity, pedestrian and non-motor vehicle behavior patterns), and human factors (such as running red lights, illegal lane changes, driving in the wrong direction, speeding, etc.).

[0048] In step S114, the influencing factors are combined to generate a scenario.

[0049] For example, Bayesian networks are used to generate feasible key scenarios.

[0050] In step S115 , the occurrence probability of the scenario is evaluated based on historical accident data.

[0051] For example, based on historical accident data, the Bayesian network inference method is used to evaluate the probability of occurrence of each scenario.

[0052] Specifically, the scenario prioritization method includes sorting the scenarios according to the probability of occurrence of each scenario, giving priority to scenarios with high probability of occurrence to form a key scenario library, which can more efficiently expose potential problems of the perception system.

[0053] In step S116, scenarios with a probability of occurrence higher than a preset probability are selected to construct a key scenario library, wherein the key scenario library includes the environmental conditions, traffic conditions, human factors, accident sequences, and probability of occurrence of the scenarios.

[0054] For example, a scenario library can be constructed based on the type and characteristics of each scenario, including factors such as environmental conditions, traffic conditions, human factors, and accident sequences, as well as assessment results of the scenario's probability of occurrence. Preferably, new accident data related to the perception system should be added periodically to maintain the timeliness of the key scenario library.

[0055] In some exemplary embodiments, step S110 further includes steps S117-S118, see Figure 3 .

[0056] In step S117, the accident sequence is specifically coded and represented to extract the process characteristics of the accident.

[0057] In step S118 , the distance algorithm and clustering algorithm are used to analyze the similarity of the accident sequences, and the accident sequences with similar process characteristics are grouped into one category to enhance the representativeness of the key scenarios.

[0058] For example, by encoding accident sequences in specific ways, we can extract the process characteristics of the accidents. Using distance and clustering algorithms to analyze the similarities of accident sequences, we can group sequences with similar accident progression patterns together. This helps identify common influencing factors and accident patterns, and enhances the representativeness of key scenarios.

[0059] The embodiment of the present invention identifies the main factors that affect the occurrence of accidents, such as environmental conditions, traffic conditions, human factors, and accident sequences, by collecting and analyzing historical accident data related to the perception system. Using methods such as Bayesian network generation and inference, the scenarios composed of these factor combinations are analyzed for comprehensive probability of occurrence, and the key scenarios that are most likely to expose potential problems in the perception system are screened out. This avoids inefficient testing of a large number of non-critical scenarios and can significantly improve testing efficiency. By focusing on these key scenarios that are prone to exposing unreliable problems in the perception system, it ensures that test resources are concentrated on discovering and resolving potential hidden dangers in the system, thereby improving the effectiveness and efficiency of the reliability testing of the perception system.

[0060] In step S120, according to the reliability test requirements of the perception system function claimed by the manufacturer, corresponding key scenarios are matched from the key scenario library as test scenarios and the test pass criteria are determined.

[0061] Example 1:

[0062] We collect and extract accident data related to autonomous vehicle perception systems from the California Department of Motor Vehicles' autonomous vehicle accident data reports. This data includes environmental conditions, traffic conditions, human factors, accident reports, and accident consequences (such as vehicle damage level and number of injuries). We preprocess the collected data to ensure its consistency and completeness, specifically manually annotating and correcting the accident sequences in the accident reports.

[0063] Accident sequences are represented using specific encodings. For example, the accident description "An autonomous vehicle stopped at an intersection for a red light and was rear-ended by an oncoming vehicle" is simplified to the sequence "V1-S1-X21," where "V1" indicates the vehicle stopped, "S1" indicates stopping at a traffic light, and "X21" indicates being rear-ended by another vehicle. Optionally, Levenshtein distance is used to calculate the differences between accident sequences, and the k-medoids clustering algorithm is used to perform cluster analysis on the accident sequences.

[0064] Based on collected accident data, key variables (environmental conditions, traffic conditions, human factors, accident sequence patterns, and accident consequences) are represented as nodes in a Bayesian network. Dependencies between nodes are determined by learning from the accident data. The Akaike Information Criterion is used to evaluate the model's goodness of fit and determine the optimal Bayesian network structure. A series of scenarios are generated by combining the states of the Bayesian network nodes. The probability of each scenario occurring is calculated using joint probabilistic inference within the Bayesian network.

[0065] Scenarios are sorted according to their probability of occurrence, and scenarios with high probability of occurrence are prioritized to form a key scenario library.

[0066] Test scenario matching: According to the reliability test requirements of the perception system functions claimed by the manufacturer, the corresponding key scenarios are matched from the key scenario library as test scenarios.

[0067] In one embodiment, the specific matching process is as follows:

[0068] Collect and analyze the functional descriptions and technical specifications provided by perception system manufacturers to clarify the capabilities and operating conditions of the perception system. For example, manufacturers claim that the perception system can accurately detect vehicles, pedestrians, and road signs in adverse conditions such as nighttime, rain, and fog.

[0069] Compare the manufacturer's test requirements with the scenario elements in the key scenario library. Each scenario in the key scenario library contains detailed information on environmental conditions, traffic conditions, human factors, and accident sequences. By matching the functions to be verified in the perception system with corresponding scenarios in the scenario library, select the corresponding key scenarios as test scenarios. For example, if you need to test the perception system's ability to identify pedestrians in the rain, select a scenario that includes rain and pedestrian-related factors.

[0070] Evaluate the match between candidate scenarios and test requirements, and give priority to those that can fully verify the functionality.

[0071] The passing criteria for the test in each scenario are determined based on whether the perception system functions claimed by the manufacturer can be achieved.

[0072] Through the above steps, the reliability test scenarios and passing criteria of the perception system are determined.

[0073] In step S130 , the failure modes corresponding to the perception functions claimed by the manufacturer in the test scenario are analyzed to determine the key sensors and fault signals to be injected into the faults.

[0074] In some exemplary embodiments, step S130 specifically includes identifying the causes of the manufacturer's claimed perception system functional failure in the test scenario, determining the key sensors that cause potential functional failure, optionally, the sensors include cameras, lidars, ultrasonic radars, IMUs, RTKs, and millimeter-wave radars; determining the failure modes of the key sensors that cause functional failure in each test scenario as the failure modes to be injected; and modeling the failure modes to generate corresponding fault signals, the parameters of the fault signals including the signal type, intensity, duration, and trigger conditions.

[0075] In some exemplary embodiments, the failure modes include camera failure mode, lidar failure mode, ultrasonic radar failure mode, IMU failure mode, RTK failure mode, and millimeter wave radar failure mode.

[0076] Furthermore, camera failure modes include: Gaussian image noise failure, salt and pepper image noise failure, gamma image noise failure, Poisson image noise failure, uniform image noise failure, Ruili image noise failure, rain and fog noise failure, intermittent image noise failure and disconnection failure; lidar failure modes include: high and low density noise failure in sandstorm weather, multiple density noise failure in rainy weather, disconnection failure and intermittent failure; ultrasonic radar failure modes include: random error distance failure, distance attenuation failure, disconnection failure and intermittent noise failure; IMU failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure; RTK failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure; and millimeter wave radar failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure.

[0077] In step S140 , the fault signal is injected into the corresponding key sensor signal without fault to obtain a sensor signal with fault.

[0078] In some exemplary embodiments, step S140 includes steps S141-S142, see Figure 4 .

[0079] In step S141, in the test scenario, multimodal sensor data of the autonomous driving perception system is collected in real time as initial fault-free sensor data.

[0080] In step S142, the fault signal is superimposed on the initial fault-free sensor data to obtain a sensor signal with a fault, wherein the parameters of the fault signal include the type, intensity, duration and triggering conditions of the signal; and the sensor signal with a fault can be output to the autonomous driving perception system in real time.

[0081] By injecting faults into sensor data in various formats, we can comprehensively evaluate the perception system's ability to reliably function under partial faults and environmental influences, and guide the system to improve and optimize redundant fault-tolerant design, thereby enhancing the reliability of the perception system.

[0082] In some exemplary embodiments, steps S130 and S140 can be completed by relying on the automatic driving perception system fault injection mode implementation device 700, the structural block diagram of which is shown in FIG. Figure 5 , including: a perception data collection module 710, a fault injection implementation module 720 and a fault injection execution module 730.

[0083] Perception data acquisition module 710 is used to collect multimodal sensor data from the autonomous driving perception system in real time during test scenarios. Specifically, perception data acquisition module 710 is responsible for acquiring raw perception data from sensors such as cameras, lidar, ultrasonic radar, inertial measurement unit (IMU), real-time differential positioning system (RTK), and millimeter-wave radar.

[0084] In real-world applications, autonomous vehicles are equipped with a variety of sensors that output data at varying frequencies and formats. The perception data acquisition module synchronizes and integrates data from each sensor through a unified interface, providing basic data support for fault injection.

[0085] Fault Injection Implementation Module 720, based on an analysis of potential failure modes and environmental interference factors in test scenarios, employs mathematical, physical, and computer science techniques to design and implement various fault injection patterns for multimodal sensors. This module develops and integrates a variety of independently functioning fault modes for sensors such as cameras, lidar, ultrasonic radar, IMU, RTK, and millimeter-wave radar. Once these fault modes are implemented, they can be injected into test scenarios to test the response of perception system algorithms under realistic environmental and fault conditions. For example, rain and fog noise can be injected to simulate adverse weather conditions to evaluate the system's target detection and tracking capabilities.

[0086] Taking camera fault injection as an example, the fault injection implementation module 720 simulates the following typical fault types for cameras in autonomous driving systems: Gaussian image noise injection: This injects random noise following a Gaussian distribution into the image to simulate the effects of factors such as illumination variations and electromagnetic interference on image quality. Other image noise types include salt and pepper noise, gamma noise, Poisson noise, uniform noise, Rayleigh noise, rain and fog noise, intermittent noise, and disconnection noise.

[0087] For other sensors, the fault injection implementation module 720 also designs corresponding failure modes: LiDAR failure modes: high and low density noise in dusty weather, multiple density noise in rainy weather, disconnection failure, and intermittent failure. Ultrasonic radar failure modes: random error distance failure, distance attenuation failure, disconnection failure, and intermittent failure. IMU failure modes: deviation noise failure, random noise failure, disconnection failure, and intermittent noise failure. RTK failure modes: deviation noise failure, random noise failure, disconnection failure, and intermittent noise failure. Millimeter-wave radar failure modes: deviation noise failure, random noise failure, disconnection failure, and intermittent noise failure.

[0088] The fault injection implementation module 720 generates corresponding fault signals by modeling typical faults and environmental interference factors that may occur in various types of sensors, providing data support for the fault injection execution module.

[0089] The fault injection execution module 730 is responsible for injecting the signal generated by the fault injection implementation module 720 into the sensor data, simulating the sensor's operating state under actual fault conditions. This module can input real-time fault signals into the data of key sensors according to the preset fault injection strategy, ensuring the controllability and accuracy of the fault injection process. Specifically, the fault injection execution module operates according to the following steps:

[0090] (1) Fault injection strategy setting: Set parameters such as the type, intensity, duration, and trigger conditions of the fault injection.

[0091] (2) Fault signal superposition: The signal generated by the fault injection implementation module is superimposed with the original sensor data obtained by the perception system data acquisition module to obtain the sensor signal with fault.

[0092] (3) Real-time data output: The faulty sensor signal is output to the perception system in real time, simulating the output of the sensor under abnormal conditions and providing data support for the reliability test of the perception system.

[0093] Through the collaborative work of the above three modules, faults can be injected into sensor data in various formats of the autonomous driving perception system in the test scenario, and the reliability of the perception system under fault and environmental interference conditions can be comprehensively tested.

[0094] In step S150, in the test scenario, the perception system function claimed by the manufacturer is tested based on the key sensor signals without faults and the sensor signals with faults, and the reliability of the perception system function in the test scenario is obtained according to the test passing criteria.

[0095] In some exemplary embodiments, step S150 specifically includes: in a test scenario, testing the perception system function claimed by the manufacturer based on a fault-free sensor signal to ensure that the perception system function claimed by the manufacturer is normal in the initial state; in a test scenario, testing the perception system function claimed by the manufacturer based on a faulty sensor signal, and obtaining a test result of the perception function after fault injection based on the test pass standard, wherein the test result is passed or failed; and in each test scenario, calculating the ratio of the number of test results with a pass to the total number of tests on the perception system function claimed by the manufacturer based on the faulty sensor signal, as the reliability of the perception system function claimed by the manufacturer in the test scenario.

[0096] For example, according to the predetermined test process, the perception system is run in a test scenario, undergoing fault injection testing to monitor the system's performance under sensor failures and environmental interference. Assuming 22 fault injection patterns are identified and each fault is injected and tested once, the total number of tests performed on the manufacturer's claimed perception system functionality based on faulty sensor signals is 22. The perception system's output under the fault injection conditions is recorded, and the test is determined to have passed or failed based on the test pass criteria.

[0097] Test scenario reliability calculation: For test scenario i, the calculation result is the ratio of the number of passed tests to the total number of tests based on the sensor signals with faults to test the manufacturer's claimed perception system functions, which is used as the reliability measure of scenario i.

[0098] Test scenario reliability M i = Number of tests with a passing result / Total number of tests based on sensor signals with faults to test the manufacturer's claimed perception system functionality × 100%.

[0099] In step S160, based on the reliability of the perception system function in each test scenario, the overall reliability of the perception system and its reliability level are calculated.

[0100] For example, the reliability metrics of each test scenario are combined to obtain the total reliability of the perception system. Optionally, the average method is used to calculate the total reliability M T :M T =(M1+M2+M3) / 3, where M1, M2, and M3 are the reliability metrics of each test scenario.

[0101] According to the calculated total reliability M T ,The reliability rating of the manufacturer’s claimed perception system functions is set as follows: ,High: M T ≥90%; Medium: 75%≤M T <90%; and low: M T <75%.

[0102] This rating standard can intuitively evaluate the reliability level of the perception system functions and provide a reference for subsequent improvements.

[0103] During fault injection testing, the system's perception parameter states under fault injection are monitored and compared against the test pass criteria. The result is calculated as the ratio of the number of passed tests to the total number of tests performed on the perception system's claimed functionality based on faulty sensor signals. This quantifies the system's reliability in achieving the claimed functionality in the test scenario. For test scenarios with low reliability and failure modes that failed the test, the causes are thoroughly investigated to identify system weaknesses. This quantitative assessment method provides objective reliability metrics, facilitating comparison and tracking of system reliability. Based on the assessment results, targeted improvement suggestions can be made, such as optimizing algorithms, adding redundant designs, and strengthening fault diagnosis, to enhance the overall reliability of the system.

[0104] Example 2:

[0105] Assume that a perception system manufacturer claims that its system can "identify traffic lights within a range of 150 meters without a map." This function needs to be tested for reliability. The perception system consists of cameras, lidar, millimeter-wave radar, IMU, and RTK. Based on the matching of the key scenario library, three key scenarios were selected as test scenarios, including "a long straight road with no traffic flow and a traffic light in front," "a traffic light at an intersection," and "a long straight road with traffic flow at night and a traffic light in front." Taking "a long straight road with no traffic flow and a traffic light in front" as an example, the specific implementation steps are as follows:

[0106] Step a: Fault injection pattern analysis.

[0107] (1) Whether traffic lights are detected when entering the 150-meter range.

[0108] Failure criterion: When entering the 150-meter range, the perception system does not generate a traffic light recognition detection frame.

[0109] Possible sensor failure: The camera cannot detect the traffic light, and the lidar cannot detect the traffic light.

[0110] (2) Whether the traffic light color is detected when entering the 150-meter range.

[0111] Failure criterion: When entering the 150-meter range, the perception system does not generate traffic light classification data.

[0112] Possible sensor failure: The camera cannot distinguish colors.

[0113] (3) When entering the 150-meter range, is the traffic light color classified correctly?

[0114] Failure criterion: When entering the 150-meter range, the perception system generates traffic light classification data, but the classification data is incorrect.

[0115] Possible sensor failure: The camera perceives colors as distorted.

[0116] Through the above analysis, the passing standard of the perception function test is the ability to correctly detect traffic lights within a range of 150 meters and the accurate color classification results. The key sensors are determined to be cameras and lidar.

[0117] Its fault injection modes include:

[0118] Image failure: Gaussian, salt and pepper, gamma, uniform, Rayleigh, Poisson, rain and fog, disconnection, and intermittent failure, a total of 9 types.

[0119] LiDAR faults: There are four types of faults, including noise faults simulating dust and rain, intermittent faults, and disconnection faults.

[0120] Therefore, the total number of fault injection tests in this test scenario is 13.

[0121] Step b: Perform fault injection testing.

[0122] (1) Fault injection: Use the fault injection device to inject faults into the camera and lidar.

[0123] (2) Test execution: The perception system is run in a simulated test scenario of a long straight road with no traffic flow and a traffic light in front of it, and the perception system parameters are monitored under different sensor failure modes.

[0124] (3) Data collection: record the output results of the perception system.

[0125] Step c: Test scenario test result evaluation.

[0126] Test scenario reliability calculation:

[0127] The perception system functions claimed by the manufacturer are tested based on faulty sensor signals. Assuming that 11 out of 13 tests meet the test pass criteria, the reliability metric M1 of this test scenario is: M1=11 / 13×100%=84.62%.

[0128] Overall reliability calculation and rating:

[0129] Assume that in the three test scenarios, the reliability evaluation results M1, M2 and M3 of the perception system for the three test scenarios are 84.62%, 92.31% and 84.62% respectively. Optionally, the total reliability metric M of the perception system can be calculated by the average method. T For: M T =(M1+M2+M3) / 3=87.18%

[0130] According to the above rating standards, the total reliability M is calculatedT is 87.18%, which is consistent with 75%≤M T <90% of the conditions, so the manufacturer claims that the reliability rating of the perception system function is “medium”.

[0131] Through reliability ratings, perception system manufacturers can intuitively understand the reliability level of their perception system functions. For systems rated "low" and "medium", it is recommended to continue optimizing performance in test scenarios and strive to reach the "high" level. Based on the overall reliability assessment results, analyze the system's functional implementation weaknesses and potential risks after fault injection. For test scenarios and failure modes with low pass rates, it is necessary to focus on analyzing the causes, deeply explore the system's shortcomings, and propose corresponding improvement measures, including:

[0132] Algorithm optimization: Improve perception algorithms, enhance fault detection and handling capabilities, and enhance the robustness of the system in complex environments.

[0133] Redundant design: Improve the redundant configuration of sensors, increase multi-sensor data fusion, improve the fault tolerance of the system, and ensure that it can still work normally when some sensors fail.

[0134] Fault diagnosis: Strengthen real-time monitoring and diagnosis of sensor faults, establish a comprehensive fault detection and recovery mechanism, and take timely measures to prevent faults from affecting system performance.

[0135] Through the above-mentioned improvement measures, manufacturers can improve the reliability of the perception system and ensure the reliable operation of the autonomous driving perception system.

[0136] The embodiment of the present invention adopts a parameterized key scenario library and an automated fault injection device, and performs scenario matching according to the reliability test requirements of the perception system functions claimed by the manufacturer, providing a convenient reliability evaluation method. Fault injection technology is used to simulate multiple types of sensor failures, comprehensively cover the types of failures, and quantitatively evaluate the reliability of the perception system functions claimed by the manufacturer, which can provide targeted improvement suggestions and improve the reliability level of the autonomous driving perception system. Environmental interference is simulated by an automated fault injection device to overcome the limitations of the traditional test environment. At the same time, compared with actual hardware failures, software-level fault injection can avoid component damage. In addition, independent reliability testing is performed directly on the perception system without relying on vehicle testing. Therefore, the present application can significantly reduce testing costs and improve testing efficiency.

[0137] In summary, the present invention addresses the existing challenges of a lack of reliability assessment for perception systems under fault conditions, insufficient fault type coverage, and low testing efficiency. By proposing a fault injection-based reliability testing method for autonomous driving perception systems, this method, through focused testing scenarios, fault injection, and quantitative reliability assessment, provides perception system manufacturers with efficient and low-cost reliability testing, providing strong support for the application of autonomous driving technology.

[0138] Figure 6 The present invention schematically shows a structural block diagram of a reliability testing device for an autonomous driving perception system based on fault injection according to an embodiment of the present invention.

[0139] like Figure 6 As shown, according to an embodiment of the present invention, a reliability testing device 800 for an autonomous driving perception system based on fault injection includes a construction module 810, a first acquisition module 820, a second acquisition module 830, a third acquisition module 840, a test module 850 and a calculation module 860.

[0140] The construction module 810 is configured to construct a key scene library, wherein the key scene library includes a plurality of key scenes.

[0141] The first acquisition module 820 is used to match corresponding key scenarios from the key scenario library as test scenarios and determine the test pass criteria based on the reliability test requirements of the perception system functions claimed by the manufacturer.

[0142] The second acquisition module 830 is used to analyze the failure mode corresponding to the perception function claimed by the manufacturer in the test scenario, and determine the key sensors and failure signals to be injected into the failure.

[0143] The third acquisition module 840 is configured to inject the fault signal into the corresponding key sensor signal without fault to obtain the sensor signal with fault.

[0144] The test module 850 is used to test the perception system functions claimed by the manufacturer based on the fault-free key sensor signals and the faulty sensor signals in the test scenario, and obtain the reliability of the perception system functions in the test scenario according to the test passing standards.

[0145] The calculation module 860 is used to calculate the total reliability of the perception system and its reliability level based on the reliability of the perception system function in each test scenario.

[0146] In some specific embodiments, any multiple modules among the construction module 810, the first acquisition module 820, the second acquisition module 830, the third acquisition module 840, the testing module 850, and the calculation module 860 can be combined into a single module for implementation, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in a single module.

[0147] In some specific embodiments, at least one of the construction module 810, the first acquisition module 820, the second acquisition module 830, the third acquisition module 840, the test module 850, and the calculation module 860 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented in hardware or firmware by any other reasonable means of integrating or packaging circuits, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the construction module 810, the first acquisition module 820, the second acquisition module 830, the third acquisition module 840, the test module 850, and the calculation module 860 may be at least partially implemented as a computer program module, which, when executed, may perform the corresponding function.

[0148] Figure 7 A block diagram of an electronic device for a reliability testing method of an autonomous driving perception system based on fault injection according to an embodiment of the present invention is schematically shown.

[0149] like Figure 7 As shown, an electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 602 or programs loaded from a storage unit 608 into a random access memory (RAM) 603. Processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or related chipsets, and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)). Processor 601 may also include onboard memory for caching purposes. Processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0150] Various programs and data required for the operation of the electronic device 600 are stored in the RAM 603. The processor 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. The processor 601 executes the programs in the ROM 602 and / or the RAM 603 to perform various operations according to the method flow of the embodiment of the present invention. It should be noted that the program may also be stored in one or more memories other than the ROM 602 and the RAM 603. The processor 601 may also execute the programs stored in one or more memories to perform various operations according to the method flow of the embodiment of the present invention.

[0151] In some specific embodiments, electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to bus 604. Electronic device 600 may also include one or more of the following components connected to I / O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage portion 608 including a hard disk; and a communication portion 609 including a network interface card such as a LAN card or modem. Communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. Removable media 611, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 610 as needed, so that computer programs read from the removable media can be installed into storage portion 608 as needed.

[0152] One embodiment of the present invention provides a computer-readable storage medium. This computer-readable storage medium may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not incorporated into the device / apparatus / system. This computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiment of the present invention.

[0153] In some specific embodiments, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0154] An embodiment of the present invention provides a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is executed in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0155] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when the computer program is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0156] In some specific embodiments, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0157] In some specific embodiments, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from a removable medium 611. When the computer program is executed by the processor 601, the above-described functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the systems, devices, apparatuses, modules, units, etc. described above can be implemented by computer program modules.

[0158] In some specific embodiments, the program code for executing the computer program provided by the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, Python, "C" language, or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0159] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0160] The above specific embodiments further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A reliability testing method for an autonomous driving perception system based on fault injection, characterized in that: include: Building a key scene library, wherein the key scene library includes multiple key scenes; According to the reliability test requirements of the perception system function claimed by the manufacturer, corresponding key scenarios are matched from the key scenario library as test scenarios and the test pass criteria are determined; Analyze the failure modes corresponding to the manufacturer's claimed perception functions in the test scenario and identify the key sensors and fault signals to be injected into the faults. Injecting the fault signal into a corresponding key sensor signal without fault to obtain a sensor signal with fault; In the test scenario, the perception system function claimed by the manufacturer is tested based on the fault-free key sensor signals and the faulty sensor signals, and the reliability of the perception system function in the test scenario is obtained according to the test passing standard; Based on the reliability of the perception system function in each test scenario, the total reliability of the perception system and its reliability level are calculated. The step of constructing a key scenario library includes: Obtain historical accident data related to autonomous driving perception systems; Cleaning and standardizing the historical accident data to obtain standardized accident data to ensure data consistency and integrity; Extracting factors influencing the occurrence of accidents based on the standardized accident data; Combining the influencing factors to generate a scenario; Based on the historical accident data, evaluating the probability of occurrence of the scenario; and Select scenarios with a higher probability of occurrence than the preset probability to build a key scenario library, The historical accident data includes the environmental conditions, traffic conditions, human factors and accident sequence of the accident; and The key scenario library includes the environmental conditions, traffic conditions, human factors, accident sequences and occurrence probabilities of the scenarios; The building of the key scene library further includes: Specific coding is performed on the accident sequence to extract the process characteristics of the accident; Distance and clustering algorithms are used to analyze the similarity of accident sequences, and accident sequences with similar process characteristics are grouped together to enhance the representativeness of key scenarios. The injecting the fault signal into the corresponding key sensor signal to obtain the sensor signal with the fault includes: In the test scenario, real-time acquisition of multimodal sensor data from the autonomous driving perception system as initial fault-free sensor data; and Superimposing the fault signal with the initial fault-free sensor data to obtain a sensor signal with a fault, wherein the parameters of the fault signal include signal type, intensity, duration, and triggering conditions; and The faulty sensor signal can be output to the autonomous driving perception system in real time.

2. The method according to claim 1, characterized in that The analysis of the failure modes corresponding to the manufacturer's claimed perception functions in the test scenario determines the key sensors and fault signals to be injected, specifically including: Identify the causes of manufacturer-claimed perception system failures in test scenarios and identify the key sensors that cause potential failures; determining a failure mode of the key sensor that causes functional failure in each test scenario as a failure mode to be injected; and The fault mode is modeled and a corresponding fault signal is generated.

3. The method according to claim 1, characterized in that The manufacturer's claimed perception system function is tested based on the fault-free key sensor signals and the faulty sensor signals, and the reliability of the perception system function in the test scenario is obtained according to the test passing criteria, specifically including: In the test scenario, the perception system functions claimed by the manufacturer are tested based on the sensor signals without faults to ensure that the perception system functions normally as claimed by the manufacturer in the initial state; In the test scenario, the perception system function claimed by the manufacturer is tested based on the sensor signal with the fault, and according to the test passing standard, a perception function test result after the fault injection is obtained, wherein the test result is passed or failed; and In each test scenario, the test result is calculated as the ratio of the number of passes to the total number of tests based on the sensor signal with faults to the perception system function claimed by the manufacturer, and the ratio is used as the reliability of the perception system function claimed by the manufacturer in the test scenario.

4. The method according to claim 1, wherein The failure modes include camera failure mode, lidar failure mode, ultrasonic radar failure mode, IMU failure mode, RTK failure mode and millimeter wave radar failure mode.

5. The method according to claim 4, characterized in that The camera failure modes include: Gaussian image noise failure, salt and pepper image noise failure, gamma image noise failure, Poisson image noise failure, uniform image noise failure, Ruili image noise failure, rain and fog noise failure, intermittent image noise failure and disconnection failure; The laser radar failure modes include: high and low density noise failure in sandstorm weather, multiple density noise failure in rainy weather, disconnection failure and intermittent failure; The ultrasonic radar failure modes include: random error distance failure, distance attenuation failure, disconnection failure and intermittent noise failure; The IMU failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure; The RTK failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure; and The millimeter wave radar failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure.

6. A reliability testing device for an autonomous driving perception system based on fault injection, characterized in that: The device comprises: A construction module, configured to construct a key scene library, wherein the key scene library includes a plurality of key scenes; A first acquisition module is configured to match corresponding key scenarios from the key scenario library as test scenarios and determine a test pass criterion based on the reliability test requirements of the perception system function claimed by the manufacturer; The second acquisition module is used to analyze the failure modes corresponding to the perception functions claimed by the manufacturer in the test scenario and determine the key sensors and fault signals to be injected into the fault; A third acquisition module is used to inject the fault signal into the corresponding key sensor signal without fault to obtain the sensor signal with fault; a testing module configured to test the perception system functionality claimed by the manufacturer based on the fault-free key sensor signals and the faulty sensor signals in the test scenario, and obtain the reliability of the perception system functionality in the test scenario based on the test passing criteria; and The calculation module is used to calculate the total reliability of the perception system and its reliability level based on the reliability of the perception system function in each test scenario. The step of constructing a key scenario library includes: Obtain historical accident data related to autonomous driving perception systems; Cleaning and standardizing the historical accident data to obtain standardized accident data to ensure data consistency and integrity; Extracting factors influencing the occurrence of accidents based on the standardized accident data; Combining the influencing factors to generate a scenario; Based on the historical accident data, evaluating the probability of occurrence of the scenario; and Select scenarios with a higher probability of occurrence than the preset probability to build a key scenario library, The historical accident data includes the environmental conditions, traffic conditions, human factors and accident sequence of the accident; and The key scenario library includes the environmental conditions, traffic conditions, human factors, accident sequences and occurrence probabilities of the scenarios; The building of the key scene library further includes: Specific coding is performed on the accident sequence to extract the process characteristics of the accident; Distance and clustering algorithms are used to analyze the similarity of accident sequences, and accident sequences with similar process characteristics are grouped together to enhance the representativeness of key scenarios. The injecting the fault signal into the corresponding key sensor signal to obtain the sensor signal with the fault includes: In the test scenario, real-time acquisition of multimodal sensor data from the autonomous driving perception system as initial fault-free sensor data; and Superimposing the fault signal with the initial fault-free sensor data to obtain a sensor signal with a fault, wherein the parameters of the fault signal include signal type, intensity, duration, and triggering conditions; and The faulty sensor signal can be output to the autonomous driving perception system in real time.

7. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 5.

Images