Automatic driving perception system reliability test method based on fault injection

Abstract

The invention provides an automatic driving perception system reliability test method based on fault injection, which can be applied to the technical field of automatic driving vehicles, and comprises the following steps: constructing a key scene library; matching a test scene from the key scene library according to a reliability test requirement of a sensing system function claimed by a manufacturer; analyzing a fault mode corresponding to the sensing system function in the test scene, and determining a key sensor to be subjected to fault injection and a fault signal; injecting the fault signal into the key sensor signal to obtain a sensing signal with a fault; testing the sensing system based on the sensing signal with the fault to obtain the reliability of the function of the sensing system; and the total reliability and the reliability level of the sensing system are obtained. By injecting a fault into a key sensor in a test scene, the performance of a sensing system under an abnormal condition is studied, and the reliability of the sensing system is effectively evaluated.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving vehicles, and in particular to a reliability testing method, device and electronic equipment for an autonomous driving perception system based on fault injection. Background Art

[0002] The autonomous driving perception system perceives the environment and posture information through sensors such as cameras, LiDAR, millimeter-wave radar (Radar), ultrasonic radar, differential positioning signal (RTK) and inertial measurement unit (IMU), providing key support for the decision-making and control of autonomous driving vehicles. Whether the perception system can reliably realize the perception function directly affects the normal operation of autonomous driving vehicles. The operating environment of autonomous driving vehicles is complex and changeable, requiring the perception system level to still complete the predetermined functions when the sensors are affected by environmental factors such as rain, fog, dust, and when some sensors fail. For this reason, the autonomous driving perception system developed by manufacturers usually adopts multi-sensor information fusion technology. When some sensors have problems (affected by the environment or failure), the perception system integrates the data of other sensors through fusion algorithms to maintain normal perception functions and ensure system reliability. This multi-sensor information fusion mechanism is crucial in ensuring the reliability of the perception system, so the industry urgently needs to propose a reliability evaluation method for the perception system when some sensors have problems.

[0003] Fault injection technology is an effective means to evaluate the reliability of perception systems under sensor failures or environmental interference factors. By injecting faults, the performance of perception systems under potential fault conditions and environmental interference factors can be evaluated, which is of great significance for evaluating system reliability and discovering system hidden dangers.

[0004] At present, the reliability test of autonomous driving is mainly carried out at the whole vehicle level. Such methods do not target the perception system, nor do they focus on the reliability in the face of partial failures and environmental interference. It is difficult to timely discover the potential defects of the perception system based on the quantitative evaluation of the perception system, and the whole vehicle test requires the use of autonomous driving vehicles, which poses a certain obstacle to the perception system manufacturers to evaluate the reliability of the perception system. In addition, the existing autonomous driving test methods mainly rely on natural driving data. However, natural driving data contains a large number of non-critical scenes, and the perception system is not easy to expose reliability-related problems, resulting in low test efficiency.

[0005] Therefore, there is an urgent need to propose a reliability testing method for autonomous driving perception systems to efficiently and conveniently evaluate the reliability of perception systems under conditions of partial sensor failure or environmental interference, and to discover potential defects in the perception system. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] In order to solve at least one of the above technical problems in the reliability test of the autonomous driving perception system in the prior art, the embodiments of the present invention provide a reliability test method, device and electronic device for the autonomous driving perception system based on fault injection, and obtain the test scenario by constructing a key scenario library and combining the reliability test requirements of the perception system functions claimed by the manufacturer. In the matching test scenario, the sensor (key sensor) related to the claimed function is injected with a fault, and the performance of the perception system under partial sensor failure and environmental interference conditions is tested to effectively evaluate its reliability.

[0008] (II) Technical solution

[0009] In response to the above technical problems, an embodiment of the present invention proposes a reliability testing method, device and electronic device for an autonomous driving perception system based on fault injection.

[0010] According to a first aspect of the present invention, a reliability testing method for an autonomous driving perception system based on fault injection is provided, comprising: constructing a key scenario library, wherein the key scenario library includes multiple key scenarios; matching corresponding key scenarios from the key scenario library as test scenarios and determining test pass criteria based on reliability testing requirements for perception system functions claimed by manufacturers; analyzing the fault modes corresponding to the perception functions claimed by manufacturers in the test scenarios, and determining key sensors and fault signals to be injected with faults; injecting the fault signals into corresponding fault-free key sensor signals to obtain sensor signals with faults; in the test scenarios, testing the perception system functions claimed by manufacturers based on fault-free key sensor signals and faulty sensor signals, and obtaining the reliability of the perception system functions in the test scenarios based on the test pass criteria; and calculating the total reliability of the perception system and its reliability level based on the reliability of the perception system functions in each test scenario.

[0011] In some exemplary embodiments, building a key scenario library includes: obtaining historical accident data related to the autonomous driving perception system; cleaning and standardizing the historical accident data to obtain standardized accident data to ensure data consistency and integrity; extracting influencing factors of accidents based on the standardized accident data; combining the influencing factors to generate scenarios; evaluating the probability of occurrence of scenarios based on historical accident data; and selecting scenarios with a probability of occurrence higher than a preset probability to build a key scenario library, wherein the historical accident data includes environmental conditions, traffic conditions, human factors, and accident sequences in which the accidents occurred; and the key scenario library includes environmental conditions, traffic conditions, human factors, accident sequences, and probability of occurrence of the scenarios.

[0012] In some exemplary embodiments, the method further includes: performing specific coding representation on the accident sequence to extract the process characteristics of the accident; using a distance algorithm and a clustering algorithm to analyze the similarity of the accident sequences, and grouping the accident sequences with similar process characteristics into one category to enhance the representativeness of key scenarios.

[0013] In some exemplary embodiments, the failure modes corresponding to the perception functions claimed by the manufacturer in the test scenarios are analyzed to determine the key sensors and fault signals to be injected, specifically including: identifying the causes of the failure of the perception system functions claimed by the manufacturer in the test scenarios, and determining the key sensors that cause potential functional failures; determining the failure modes of the key sensors that cause functional failures in each test scenario as the failure modes to be injected; and modeling the failure modes to generate corresponding fault signals.

[0014] In some exemplary embodiments, a fault signal is injected into a corresponding key sensor signal to obtain a sensor signal with a fault, specifically including: in a test scenario, multimodal sensor data of an autonomous driving perception system is collected in real time as initial fault-free sensor data; and the fault signal is superimposed with the initial fault-free sensor data to obtain a sensor signal with a fault, wherein parameters of the fault signal include signal type, intensity, duration, and trigger conditions; and the sensor signal with a fault can be output to the autonomous driving perception system in real time.

[0015] In some exemplary embodiments, the perception system functions claimed by the manufacturer are tested based on fault-free key sensor signals and faulty sensor signals, and the reliability of the perception system functions in the test scenario is obtained based on the test passing criteria, specifically including: in the test scenario, the perception system functions claimed by the manufacturer are tested based on fault-free sensor signals to ensure that the perception system functions claimed by the manufacturer in the initial state are normal; in the test scenario, the perception system functions claimed by the manufacturer are tested based on faulty sensor signals, and the perception function test results after fault injection are obtained based on the test passing criteria, wherein the test results are passed or failed; and in each test scenario, the ratio of the number of test results with a pass to the total number of tests on the perception system functions claimed by the manufacturer based on the faulty sensor signals is calculated as the reliability of the perception system functions claimed by the manufacturer in the test scenario.

[0016] In some exemplary embodiments, the failure modes include camera failure mode, lidar failure mode, ultrasonic radar failure mode, IMU failure mode, RTK failure mode, and millimeter wave radar failure mode.

[0017] In some exemplary embodiments, camera failure modes include: Gaussian image noise failure, salt and pepper image noise failure, gamma image noise failure, Poisson image noise failure, uniform image noise failure, Ruili image noise failure, rain and fog noise failure, intermittent image noise failure and disconnection failure; lidar failure modes include: high and low density noise failure in sandstorm weather, multiple density noise failure in rainy weather, disconnection failure and intermittent failure; ultrasonic radar failure modes include: random error distance failure, distance attenuation failure, disconnection failure and intermittent noise failure; IMU failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure; RTK failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure; and millimeter wave radar failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure.

[0018] According to a second aspect of the present invention, there is provided an apparatus for reliability testing an autonomous driving perception system based on fault injection, comprising: a construction module for constructing a key scenario library, wherein the key scenario library comprises a plurality of key scenarios; a first acquisition module for matching corresponding key scenarios from the key scenario library as test scenarios and determining the criterion for passing the test according to the reliability test requirements of the perception system functions claimed by the manufacturer; a second acquisition module for analyzing the fault mode corresponding to the perception function claimed by the manufacturer in the test scenario, and determining the key sensors and fault signals to be injected with the fault; a third acquisition module for injecting the fault signal into the corresponding key sensor signal without fault to obtain the sensor signal with fault; a test module for testing the perception system function claimed by the manufacturer based on the key sensor signal without fault and the sensor signal with fault in the test scenario, and obtaining the reliability of the perception system function in the test scenario according to the criterion for passing the test; and a calculation module for calculating the total reliability of the perception system and its reliability level based on the reliability of the perception system function in each test scenario.

[0019] According to a third aspect of the present invention, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is executed by the processor to implement the above method.

[0020] According to a fourth aspect of the present invention, a storage medium is provided, characterized in that: computer-readable instructions are stored in the storage medium, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the above method.

[0021] (III) Beneficial effects

[0022] It can be seen from the above technical solutions that the reliability testing method, device and electronic device of an autonomous driving perception system based on fault injection provided by the embodiments of the present invention have at least the following beneficial effects:

[0023] (1) By collecting accident data of the perception system and analyzing the influencing factors, a key scenario library is constructed. Compared with the existing technology that uses a large number of non-critical scenario tests based on natural driving data or vehicle-level simulation tests, this application focuses on key scenarios that are prone to expose unreliable problems of the perception system, ensuring that test resources are concentrated on discovering and solving potential hidden dangers of the perception system, shortening the test cycle, and improving test efficiency.

[0024] (2) Fault injection technology is used to simulate perception system failures and environmental interference factors, comprehensively covering the fault signals and environmental interference of various sensors, and quantitatively evaluating the reliability of the perception system functions claimed by the manufacturer. Targeted improvement suggestions can be provided to improve the reliability level of the autonomous driving perception system.

[0025] (3) This application uses an automated fault injection device to simulate environmental interference, overcoming the limitations of traditional test environments. At the same time, compared to actual hardware failures, software-level fault injection can avoid component damage.

[0026] (4) Independent reliability testing is performed directly on the perception system without relying on the autonomous driving vehicle. Therefore, this application can significantly reduce testing costs and improve testing efficiency, providing an economical and convenient solution for reliability testing of autonomous driving perception systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0028] Figure 1 A schematic diagram of a process flow of a reliability testing method for an autonomous driving perception system based on fault injection according to an embodiment of the present invention is shown;

[0029] Figure 2 A schematic diagram of a process of building a key scene library according to an embodiment of the present invention is shown;

[0030] Figure 3 A schematic diagram of another process of building a key scene library according to an embodiment of the present invention is shown;

[0031] Figure 4 A schematic diagram of a process of injecting a fault signal into a corresponding key sensor signal to obtain a sensor signal with a fault according to an embodiment of the present invention is shown;

[0032] Figure 5A structural block diagram of a device for implementing a fault injection mode of an autonomous driving perception system according to an embodiment of the present invention is schematically shown;

[0033] Figure 6 A structural block diagram of a reliability testing device for an autonomous driving perception system based on fault injection according to an embodiment of the present invention is schematically shown; and

[0034] Figure 7 A block diagram of an electronic device for a reliability testing method of an autonomous driving perception system based on fault injection according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Fault injection technology is an effective method to evaluate the reliability of perception systems under sensor failures or environmental interference factors. Through devices that can generate different failure modes and simulate environmental interference factors such as rain, fog, and white balance imbalance, faults can be injected into key sensors to simulate abnormal conditions for testing. Introducing fault injection technology to evaluate the performance of perception systems under potential failure conditions and actual environmental interference factors is of great significance for discovering system risks and evaluating system reliability.

[0037] The embodiment of the present invention provides a reliability test method for an autonomous driving perception system based on fault injection. The method first determines the key scenarios that affect the reliability of the perception system through data collection and influencing factor analysis. Then, a fault injection device is used to design and implement multiple fault injection modes for multimodal sensors to simulate various faults and environmental interference factors that may be encountered in actual operation. In the test scenario, a fault injection test is performed on the key sensors involved in the claimed function implementation to evaluate their performance under abnormal conditions, and the reliability is calculated based on the test results. Compared with the prior art based on a large number of non-critical scenario tests or vehicle-level simulation tests in natural driving data, this application does not require the configuration of an autonomous driving vehicle, the test is convenient, no large-scale testing is required, the test cycle is short, and the test efficiency is higher. At the same time, the fault injection technology is used to simulate the perception system failure problems and environmental interference factors, and the reliability of the perception system is comprehensively evaluated, which can provide targeted improvement suggestions and improve the reliability level of the autonomous driving perception system.

[0038] Figure 1The present invention schematically shows a flow chart of a method for reliability testing of an autonomous driving perception system based on fault injection according to an embodiment of the present invention.

[0039] like Figure 1 As shown, a reliability testing method for an autonomous driving perception system based on fault injection according to an embodiment of the present invention includes steps S110-S160.

[0040] In step S110, a key scenario library is constructed, wherein the key scenario library includes multiple key scenarios. In order to efficiently evaluate the reliability of the autonomous driving perception system in the actual driving environment, a set of key scenarios needs to be determined first. These scenarios should be able to more efficiently expose potential problems of the perception system.

[0041] In some exemplary embodiments, step S110 includes steps S111-S116, see Figure 2 .

[0042] In step S111, historical accident data related to the autonomous driving perception system is obtained, wherein the historical accident data includes environmental conditions, traffic conditions, human factors, and accident sequences where the accidents occurred.

[0043] For example, historical accident data related to the autonomous driving perception system can be collected from authoritative traffic accident databases and accident reports of autonomous vehicles.

[0044] In step S112, the historical accident data is cleaned and standardized to obtain standardized accident data to ensure the consistency and integrity of the data.

[0045] For example, the collected data is cleaned to remove incomplete or inaccurate records. The data is standardized to ensure that the data format of each factor is consistent for subsequent analysis.

[0046] In step S113, influencing factors of the accident are extracted based on the standardized accident data.

[0047] For example, identifying potential influencing factors: extracting factors that may affect the occurrence of accidents from the data, including environmental factors (such as weather conditions, lighting conditions, road types), traffic-related factors (such as traffic flow, vehicle speed range, road signs and signal complexity, pedestrian and non-motor vehicle behavior patterns), and human factors (such as running red lights, illegal lane changes, driving in the wrong direction, speeding, etc.).

[0048] In step S114, the influencing factors are combined to generate a scenario.

[0049] For example, Bayesian networks are used to generate feasible critical scenarios.

[0050] In step S115 , based on historical accident data, the occurrence probability of the scenario is evaluated.

[0051] For example, based on historical accident data, the Bayesian network inference method is used to evaluate the probability of occurrence of each scenario.

[0052] Specifically, the scenario priority sorting method includes sorting the scenarios according to the probability of occurrence of each scenario, and giving priority to scenarios with high probability of occurrence to form a key scenario library, which can more efficiently expose potential problems of the perception system.

[0053] In step S116, scenarios with a probability of occurrence higher than a preset probability are selected to construct a key scenario library, wherein the key scenario library includes the environmental conditions, traffic conditions, human factors, accident sequences, and probability of occurrence of the scenarios.

[0054] For example, according to the type and characteristics of the scenario, the scenario library is constructed, including factors such as environmental conditions, traffic conditions, human factors and accident sequences, as well as the evaluation results of the probability of scenario occurrence. Preferably, new perception system-related accident data is added periodically to maintain the timeliness of the key scenario library.

[0055] In some exemplary embodiments, step S110 further includes steps S117-S118, see Figure 3 .

[0056] In step S117, the accident sequence is specifically coded and represented to extract the process characteristics of the accident.

[0057] In step S118, the distance algorithm and clustering algorithm are used to analyze the similarity of the accident sequences, and the accident sequences with similar process characteristics are classified into one category to enhance the representativeness of the key scenes.

[0058] For example, by encoding the accident sequence in a specific way, the process characteristics of the accident can be extracted. The distance algorithm and clustering algorithm are used to analyze the similarity of the accident sequence, and the sequences with similar accident progression patterns are classified into one category, which helps to identify common influencing factors and accident patterns and enhance the representativeness of key scenarios.

[0059] The embodiment of the present invention collects and analyzes historical accident data related to the perception system to identify the main factors that affect the occurrence of accidents, such as environmental conditions, traffic conditions, human factors, and accident sequences. Using methods such as Bayesian network generation and inference, the scenarios composed of these factors are analyzed for comprehensive probability of occurrence, and the key scenarios that are most likely to expose potential problems in the perception system are screened out. Inefficient testing of a large number of non-critical scenarios is avoided, and the testing efficiency can be significantly improved. By focusing on these key scenarios that are prone to exposing the unreliability of the perception system, it is ensured that test resources are concentrated on discovering and resolving potential hidden dangers in the system, thereby improving the effectiveness and efficiency of the reliability testing of the perception system.

[0060] In step S120, according to the reliability test requirements of the perception system function claimed by the manufacturer, the corresponding key scenarios are matched from the key scenario library as test scenarios and the criteria for passing the test are determined.

[0061] Embodiment 1:

[0062] Collect and extract accident data related to the autonomous vehicle perception system from the California Department of Motor Vehicles' autonomous vehicle accident data report. The data includes environmental conditions, traffic conditions, human factors, accident reports, and accident consequences (such as vehicle damage level and number of injured people). Preprocess the collected data to ensure data consistency and completeness, especially manually annotate and correct the accident sequence in the accident report.

[0063] The accident sequence is represented by a specific code. For example, the accident description "the autonomous driving vehicle stopped at the intersection and waited for the red light, and was rear-ended by the vehicle behind" is simplified to the sequence "V1-S1-X21", where "V1" means the vehicle stopped, "S1" means stopped at the traffic light, and "X21" means rear-ended by other vehicles. Optionally, the Levenshtein distance is used to calculate the difference between accident sequences, and the k-medoids clustering algorithm is used to perform cluster analysis on the accident sequences.

[0064] Based on the collected accident data, the main variables (environmental conditions, traffic conditions, human factors, accident sequence patterns and accident consequences) are represented as nodes of the Bayesian network. The dependencies between the nodes are determined by learning the accident data. The Akaike information criterion is used to evaluate the goodness of fit of the model and determine the optimal structure of the Bayesian network. A series of scenarios are generated by setting the state combination of the Bayesian network nodes. The probability of occurrence of each scenario is calculated by the joint probability reasoning of the Bayesian network.

[0065] Scenarios are sorted according to their probability of occurrence, with priority given to scenarios with high probability of occurrence to form a key scenario library.

[0066] Test scenario matching: According to the reliability test requirements of the perception system functions claimed by the manufacturer, the corresponding key scenarios are matched from the key scenario library as test scenarios.

[0067] In one embodiment, the specific matching process is as follows:

[0068] Collect and analyze the functional descriptions and technical specifications provided by perception system manufacturers to clarify the functions and working conditions of the perception system. For example, the manufacturer claims that the perception system can accurately detect vehicles, pedestrians, and road signs in adverse conditions such as nighttime, rainy and foggy weather.

[0069] Compare the manufacturer's test requirements with the scenario elements in the key scenario library. Each scenario in the key scenario library contains detailed information such as environmental conditions, traffic conditions, human factors, and accident sequences. By matching the functions that need to be verified in the perception system with the corresponding scenarios in the scenario library, select the corresponding key scenarios as test scenarios. For example, if you need to test the perception system's ability to recognize pedestrians on rainy days, select a scenario that includes rainy days and pedestrian-related factors.

[0070] Evaluate the match between candidate scenarios and test requirements, and give priority to those scenarios that can fully verify the functionality.

[0071] Determine the criteria for passing the test in each scenario based on whether the perception system functions claimed by the manufacturer can be achieved.

[0072] Through the above steps, the reliability test scenarios and passing criteria of the perception system are determined.

[0073] In step S130, the failure mode corresponding to the perception function claimed by the manufacturer in the test scenario is analyzed to determine the key sensors and fault signals to be injected into the fault.

[0074] In some exemplary embodiments, step S130 specifically includes identifying the causes of the functional failure of the perception system claimed by the manufacturer in the test scenario, determining the key sensors that cause potential functional failure, optionally, the sensors include cameras, lidars, ultrasonic radars, IMUs, RTKs, and millimeter-wave radars; determining the failure modes of the key sensors that cause functional failures in each test scenario as the failure modes to be injected; and modeling the failure modes to generate corresponding fault signals, the parameters of the fault signals including the type, intensity, duration, and trigger conditions of the signal.

[0075] In some exemplary embodiments, the failure modes include camera failure mode, lidar failure mode, ultrasonic radar failure mode, IMU failure mode, RTK failure mode, and millimeter wave radar failure mode.

[0076] Furthermore, camera failure modes include: Gaussian image noise failure, salt and pepper image noise failure, gamma image noise failure, Poisson image noise failure, uniform image noise failure, Ruili image noise failure, rain and fog noise failure, intermittent image noise failure and disconnection failure; lidar failure modes include: high and low density noise failure in sandstorm weather, multiple density noise failure in rainy weather, disconnection failure and intermittent failure; ultrasonic radar failure modes include: random error distance failure, distance attenuation failure, disconnection failure and intermittent noise failure; IMU failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure; RTK failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure; and millimeter wave radar failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure.

[0077] In step S140, the fault signal is injected into the corresponding key sensor signal without fault to obtain a sensor signal with fault.

[0078] In some exemplary embodiments, step S140 includes steps S141-S142, see Figure 4 .

[0079] In step S141, in the test scenario, multimodal sensor data of the autonomous driving perception system is collected in real time as initial fault-free sensor data.

[0080] In step S142, the fault signal is superimposed on the initial fault-free sensor data to obtain a sensor signal with a fault, wherein the parameters of the fault signal include the type, intensity, duration and triggering conditions of the signal; and the sensor signal with a fault can be output to the autonomous driving perception system in real time.

[0081] By injecting faults into sensor data in various formats, we can comprehensively evaluate the perception system's ability to reliably implement its functions under partial faults and environmental influences, and guide the system to improve and optimize redundant fault-tolerant design to enhance the reliability of the perception system.

[0082] In some exemplary embodiments, steps S130 and S140 can be completed by relying on the automatic driving perception system fault injection mode implementation device 700, the structural block diagram of which is shown in FIG. Figure 5 , including: a perception data collection module 710, a fault injection implementation module 720 and a fault injection execution module 730.

[0083] The perception data acquisition module 710 is used to collect multi-modal sensor data of the autonomous driving perception system in real time in the test scenario. Specifically, the perception data acquisition module 710 is responsible for obtaining raw perception data from sensors such as cameras, lidars, ultrasonic radars, inertial measurement units (IMUs), real-time differential positioning systems (RTKs), and millimeter-wave radars.

[0084] In actual applications, autonomous vehicles are equipped with a variety of sensors that output data in different frequencies and formats. The perception data acquisition module synchronizes and integrates the data of each sensor through a unified interface to provide basic data support for fault injection.

[0085] The fault injection implementation module 720 uses mathematics, physics and computer technology to design and implement multiple fault injection modes for multimodal sensors based on the analysis of potential fault modes and environmental interference factors in the test scenario. This module has developed and integrated a variety of independently working fault modes, covering sensors such as cameras, lidars, ultrasonic radars, IMUs, RTKs and millimeter-wave radars. After these fault modes are implemented, they can be injected into the test scenario to test the response of the perception system algorithm under actual environment and fault conditions. For example, injecting rain and fog noise to simulate severe weather conditions can evaluate the system's target detection and tracking capabilities.

[0086] Taking camera fault injection as an example, for typical faults that cameras in autonomous driving systems may encounter, the fault injection implementation module 720 simulates the following fault forms: Gaussian image noise injection: adding random noise that obeys Gaussian distribution to the image to simulate the impact of factors such as illumination changes and electromagnetic interference on image quality. Other image noises include salt and pepper image noise, gamma image noise, Poisson image noise, uniform image noise, Rayleigh image noise, rain and fog noise, intermittent image noise, and disconnection faults.

[0087] For other sensors, the fault injection implementation module 720 also designs corresponding fault modes: LiDAR fault mode: high and low density noise in dust weather, multiple density noise in rainy weather, disconnection fault, and intermittent fault. Ultrasonic radar fault mode: random error distance fault, distance attenuation fault, disconnection fault, and intermittent fault. IMU fault mode: deviation noise fault, random noise fault, disconnection fault, and intermittent noise fault. RTK fault mode: deviation noise fault, random noise fault, disconnection fault, and intermittent noise fault. Millimeter wave radar fault mode: deviation noise fault, random noise fault, disconnection fault, and intermittent noise fault.

[0088] The fault injection implementation module 720 generates corresponding fault signals by modeling typical faults and environmental interference factors that may occur in various types of sensors, thereby providing data support for the fault injection execution module.

[0089] The fault injection execution module 730 is responsible for injecting the signal generated by the fault injection implementation module 720 into the sensing data to simulate the working state of the sensor under actual fault conditions. This module can input real-time fault signals to the data of key sensors according to the preset fault injection strategy to ensure the controllability and accuracy of the fault injection process. Specifically, the fault injection execution module works according to the following steps:

[0090] (1) Fault injection strategy setting: Set parameters such as fault injection type, intensity, duration, and trigger conditions.

[0091] (2) Fault signal superposition: The signal generated by the fault injection implementation module is superimposed with the original sensor data obtained by the perception system data acquisition module to obtain the sensor signal with fault.

[0092] (3) Real-time data output: The faulty sensor signal is output to the perception system in real time, simulating the output of the sensor under abnormal conditions and providing data support for the reliability test of the perception system.

[0093] Through the collaborative work of the above three modules, faults can be injected into sensor data in various formats of the autonomous driving perception system in the test scenario, comprehensively testing the reliability of the perception system under fault and environmental interference conditions.

[0094] In step S150, in the test scenario, the perception system function claimed by the manufacturer is tested based on the key sensor signals without faults and the sensor signals with faults, and the reliability of the perception system function in the test scenario is obtained according to the test passing standard.

[0095] In some exemplary embodiments, step S150 specifically includes: in a test scenario, testing the perception system function claimed by the manufacturer based on a fault-free sensor signal to ensure that the perception system function claimed by the manufacturer in the initial state is normal; in a test scenario, testing the perception system function claimed by the manufacturer based on a sensor signal with a fault, and obtaining a test result of the perception function after fault injection based on a test pass standard, wherein the test result is a pass or a fail; and in each test scenario, calculating the ratio of the number of test results with a pass to the total number of tests of the perception system function claimed by the manufacturer based on the sensor signal with a fault, as the reliability of the perception system function claimed by the manufacturer in the test scenario.

[0096] For example, according to the predetermined test process, the perception system is run in the test scenario, and fault injection testing is performed to monitor the system's performance under sensor failures and environmental interference. Assuming that 22 fault injection modes are determined, and each fault is injected and tested once, the total number of tests on the perception system functions claimed by the manufacturer based on the faulty sensor signal is 22 times. Record the output results of the perception system under fault injection conditions, and determine whether the test passes according to the test pass criteria.

[0097] Test scenario reliability calculation: For test scenario i, the calculation result is the ratio of the number of passed tests to the total number of tests based on the sensor signals with faults to test the perception system functions claimed by the manufacturer, which is used as the reliability measure of scenario i.

[0098] Test scenario reliability M i =Number of tests with a passing result / Total number of tests based on faulty sensor signals to test the manufacturer's claimed perception system functionality × 100%.

[0099] In step S160, based on the reliability of the perception system function in each test scenario, the overall reliability of the perception system and its reliability level are calculated.

[0100] For example, the reliability metrics of each test scenario are combined to obtain the total reliability of the perception system. Optionally, the total reliability M is calculated using the average method. T :M T =(M1+M2+M3) / 3, where M1, M2, and M3 are the reliability metrics of each test scenario.

[0101] According to the calculated total reliability M T ,The reliability rating of the manufacturer’s claims of the perception system functions is set as follows: ,High: M T ≥90%; Medium: 75%≤M T <90%; and low: M T <75%.

[0102] This rating standard can intuitively evaluate the reliability level of the perception system functions and provide a reference for subsequent improvements.

[0103] In the fault injection test, the state of the perception parameters of the monitoring system in the fault injection mode is compared with the test pass standard, and the calculation result is the ratio of the number of passed tests to the total number of tests based on the sensor signal with faults to test the perception system functions claimed by the manufacturer. In this way, the reliability of the perception system in achieving the functions claimed by the manufacturer in the test scenario is quantified. For test scenarios with low reliability and failure modes that fail the test, the causes are deeply explored to identify the weak links of the system. This quantitative evaluation method provides objective reliability indicators, which is convenient for comparing and tracking system reliability. According to the evaluation results, targeted improvement suggestions can be put forward, such as optimizing algorithms, adding redundant designs, strengthening fault diagnosis, etc., to improve the overall reliability of the system.

[0104] Embodiment 2:

[0105] Assume that the perception system manufacturer claims that its system can "recognize traffic lights within a range of 150 meters without a map", and the reliability of this function needs to be tested. Its perception system consists of cameras, lidar, millimeter-wave radar, IMU and RTK. According to the matching of the key scenario library, three key scenarios were selected as test scenarios, including "traffic lights in front of a long straight road with no traffic flow", "traffic lights at the intersection" and "traffic lights in front of a long straight road with traffic flow at night". Taking "traffic lights in front of a long straight road with no traffic flow" as an example, the specific implementation steps are as follows:

[0106] Step a: Fault injection pattern analysis.

[0107] (1) Whether traffic lights are detected when entering the 150-meter range.

[0108] Failure criterion: When entering the 150-meter range, the perception system does not generate a traffic light recognition detection frame.

[0109] Possible sensor failure: The camera cannot detect the traffic light, and the lidar cannot detect the traffic light.

[0110] (2) Whether the color of the traffic light is detected when entering the 150-meter range.

[0111] Failure criterion: When entering the 150-meter range, the perception system does not generate traffic light classification data.

[0112] Possible sensor failure: The camera cannot distinguish between colors.

[0113] (3) When entering the 150-meter range, are the traffic light colors classified correctly?

[0114] Failure criterion: When entering the 150-meter range, the perception system generates traffic light classification data, but the classification data is wrong.

[0115] Possible sensor failure: The camera perceives colors distorted.

[0116] Through the above analysis, the passing standard of the perception function test is that the traffic lights can be correctly detected within a range of 150 meters, and the color classification results are accurate. The key sensors are determined to be cameras and lidar.

[0117] Its fault injection modes include:

[0118] Image failure: Gaussian, salt and pepper, gamma, uniform, Rayleigh, Poisson, rain and fog, disconnection, and intermittent failure, a total of 9 types.

[0119] LiDAR failures: There are 4 types in total, including noise failures simulating dust and rain, intermittent failures and disconnection failures.

[0120] Therefore, the total number of fault injection tests in this test scenario is 13.

[0121] Step b: Perform fault injection testing.

[0122] (1) Fault injection: Use a fault injection device to inject faults into the camera and lidar.

[0123] (2) Test execution: The perception system is run in a simulated test scenario of a long straight road with no traffic flow and a traffic light in front of it, and the perception system parameters are monitored under different sensor failure modes.

[0124] (3) Data collection: recording the output results of the perception system.

[0125] Step c: Test scenario test result evaluation.

[0126] Test scenario reliability calculation:

[0127] The perception system functions claimed by the manufacturer are tested based on faulty sensor signals. Assuming that 11 out of 13 tests meet the test pass criteria, the reliability metric M1 of the test scenario is: M1=11 / 13×100%=84.62%.

[0128] Overall reliability calculation and rating:

[0129] Assume that in the three test scenarios, the reliability evaluation results M1, M2 and M3 of the perception system for the three test scenarios are 84.62%, 92.31% and 84.62% respectively. Optionally, the total reliability metric M of the perception system can be calculated by the average method. T For: M T =(M1+M2+M3) / 3=87.18%

[0130] According to the above rating criteria, the total reliability M is calculatedT is 87.18%, which is consistent with 75%≤M T <90% of the conditions, so the manufacturer claims that the reliability rating of the perception system function is “medium”.

[0131] Through reliability rating, perception system manufacturers can intuitively understand the reliability level of their perception system functions. For systems rated "low" and "medium", it is recommended to continue to optimize the performance in the test scenario and strive to reach the "high" level. Based on the overall reliability evaluation results, analyze the weak links and hidden dangers of the system's functional implementation after fault injection. For test scenarios and failure modes with low pass rates, it is necessary to focus on analyzing the causes, deeply explore the deficiencies of the system, and propose corresponding improvement measures, including:

[0132] Algorithm optimization: Improve the perception algorithm, enhance the ability to detect and handle faults, and enhance the robustness of the system in complex environments.

[0133] Redundant design: Improve the redundant configuration of sensors, increase multi-sensor data fusion, improve the fault tolerance of the system, and ensure that it can still work normally when some sensors fail.

[0134] Fault diagnosis: Strengthen real-time monitoring and diagnosis of sensor faults, establish a complete fault detection and recovery mechanism, and take timely measures to avoid faults affecting system performance.

[0135] The above-mentioned improvement measures can help manufacturers improve the reliability of perception systems and ensure the reliable operation of autonomous driving perception systems.

[0136] The embodiment of the present invention adopts a parameterized key scenario library and an automated fault injection device, and performs scenario matching according to the reliability test requirements of the perception system function claimed by the manufacturer, to provide a convenient reliability evaluation method. Fault injection technology is used to simulate multiple types of sensor failures, comprehensively cover the types of failures, and quantitatively evaluate the reliability of the perception system functions claimed by the manufacturer, so as to provide targeted improvement suggestions and improve the reliability level of the autonomous driving perception system. Environmental interference is simulated by an automated fault injection device to overcome the limitations of the traditional test environment. At the same time, compared with actual hardware failures, software-level fault injection can avoid component damage. In addition, independent reliability testing is performed directly on the perception system without relying on vehicle testing. Therefore, the present application can significantly reduce testing costs and improve testing efficiency.

[0137] In summary, the embodiments of the present invention address the problems of the prior art lacking reliability assessment of the perception system under fault conditions, insufficient coverage of fault types, and low test efficiency, and propose a reliability testing method for an autonomous driving perception system based on fault injection. By focusing on test scenarios, fault injection, and quantitative reliability assessment, efficient and low-cost reliability testing is provided for perception system manufacturers, providing a strong guarantee for the application of autonomous driving technology.

[0138] Figure 6 The structural block diagram of a reliability testing device for an autonomous driving perception system based on fault injection according to an embodiment of the present invention is schematically shown.

[0139] like Figure 6 As shown, a reliability testing device 800 for an autonomous driving perception system based on fault injection according to an embodiment of the present invention includes a construction module 810, a first acquisition module 820, a second acquisition module 830, a third acquisition module 840, a test module 850 and a calculation module 860.

[0140] The construction module 810 is used to construct a key scene library, wherein the key scene library includes multiple key scenes.

[0141] The first acquisition module 820 is used to match corresponding key scenarios from the key scenario library as test scenarios and determine the criteria for passing the test according to the reliability test requirements of the perception system functions claimed by the manufacturer.

[0142] The second acquisition module 830 is used to analyze the failure mode corresponding to the perception function claimed by the manufacturer in the test scenario, and determine the key sensors and failure signals to be injected with the failure.

[0143] The third acquisition module 840 is used to inject the fault signal into the corresponding key sensor signal without fault to obtain the sensor signal with fault.

[0144] The test module 850 is used to test the perception system functions claimed by the manufacturer based on the fault-free key sensor signals and the faulty sensor signals in the test scenario, and obtain the reliability of the perception system functions in the test scenario according to the test passing standards.

[0145] The calculation module 860 is used to calculate the total reliability of the perception system and its reliability level based on the reliability of the perception system function in each test scenario.

[0146] In some specific embodiments, any multiple modules among the construction module 810, the first acquisition module 820, the second acquisition module 830, the third acquisition module 840, the test module 850, and the calculation module 860 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.

[0147] In some specific embodiments, at least one of the construction module 810, the first acquisition module 820, the second acquisition module 830, the third acquisition module 840, the test module 850 and the calculation module 860 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the construction module 810, the first acquisition module 820, the second acquisition module 830, the third acquisition module 840, the test module 850 and the calculation module 860 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0148] Figure 7 A block diagram of an electronic device for a reliability testing method of an autonomous driving perception system based on fault injection according to an embodiment of the present invention is schematically shown.

[0149] like Figure 7 As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an on-board memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0150] In RAM603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM602 and RAM603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to an embodiment of the present invention by executing the program in ROM602 and / or RAM603. It should be noted that the program can also be stored in one or more memories other than ROM602 and RAM603. Processor 601 can also perform various operations of the method flow according to an embodiment of the present invention by executing the program stored in one or more memories.

[0151] In some specific embodiments, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the I / O interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0152] An embodiment of the present invention shows a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0153] In some specific embodiments, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM602 and / or RAM603 described above and / or one or more memories other than ROM602 and RAM603.

[0154] An embodiment of the present invention shows a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present invention.

[0155] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0156] In some specific embodiments, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0157] In some specific embodiments, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.

[0158] In some specific embodiments, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages, specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, such as Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0159] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0160] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A reliability testing method for an autonomous driving perception system based on fault injection, characterized in that: include: Building a key scene library, wherein the key scene library includes a plurality of key scenes; According to the reliability test requirements of the perception system function claimed by the manufacturer, corresponding key scenarios are matched from the key scenario library as test scenarios and the test pass criteria are determined; Analyze the failure modes corresponding to the perception functions claimed by the manufacturer in the test scenario, and determine the key sensors and fault signals to be injected; Injecting the fault signal into a corresponding key sensor signal without fault to obtain a sensor signal with fault; In the test scenario, the perception system function claimed by the manufacturer is tested based on the key sensor signal without fault and the sensor signal with fault, and the reliability of the perception system function in the test scenario is obtained according to the standard of passing the test; And based on the reliability of the perception system function in each test scenario, the overall reliability of the perception system and its reliability level are calculated.

2. The method according to claim 1, characterized in that The construction of the key scene library includes: Obtain historical accident data related to the autonomous driving perception system; Cleaning and standardizing the historical accident data to obtain standardized accident data to ensure the consistency and integrity of the data; Extracting factors influencing the occurrence of accidents based on the standardized accident data; Combining the influencing factors to generate a scenario; Based on the historical accident data, assessing the probability of occurrence of the scenario; and Select scenarios with a higher probability of occurrence than the preset probability to build a key scenario library. The historical accident data includes the environmental conditions, traffic conditions, human factors and accident sequence where the accident occurred; and The key scenario library includes the environmental conditions, traffic conditions, human factors, accident sequences and occurrence probabilities of the scenarios.

3. The method according to claim 2, characterized in that The building of the key scene library also includes: Specific coding is performed on the accident sequence to extract the process characteristics of the accident; Distance algorithm and clustering algorithm are used to analyze the similarity of accident sequences, and accident sequences with similar process characteristics are grouped together to enhance the representativeness of key scenarios.

4. The method according to claim 1, characterized in that: The analysis is to determine the failure modes corresponding to the perception functions claimed by the manufacturer in the test scenario, and to determine the key sensors and failure signals to be injected with the failures, including: Identify the causes of the perception system failures claimed by the manufacturer in the test scenarios and identify the key sensors that cause potential failures; Determining the failure mode of the key sensor that causes functional failure in each test scenario as the failure mode to be injected; and The fault mode is modeled and a corresponding fault signal is generated.

5. The method according to claim 1, characterized in that The step of injecting the fault signal into the corresponding key sensor signal to obtain the sensor signal with the fault specifically includes: In the test scenario, real-time acquisition of multi-modal sensor data from the autonomous driving perception system as initial fault-free sensor data; and The fault signal is superimposed on the initial fault-free sensor data to obtain a sensor signal with a fault, The parameters of the fault signal include the type, intensity, duration and triggering conditions of the signal; and The faulty sensor signal can be output to the autonomous driving perception system in real time.

6. The method according to claim 1, characterized in that The testing of the perception system function claimed by the manufacturer based on the key sensor signal without fault and the sensor signal with fault, and obtaining the reliability of the perception system function in the test scenario according to the standard of passing the test, specifically includes: In the test scenario, the perception system function claimed by the manufacturer is tested based on the sensor signal without faults to ensure that the perception system function claimed by the manufacturer is normal in the initial state; In the test scenario, the perception system function claimed by the manufacturer is tested based on the sensor signal with the fault, and the perception function test result after the fault injection is obtained according to the test pass standard, wherein the test result is passed or failed; and In each test scenario, the test result is calculated as the ratio of the number of passes to the total number of tests based on faulty sensor signals to the perception system functions claimed by the manufacturer, and the ratio is used as the reliability of the perception system functions claimed by the manufacturer in the test scenario.

7. The method according to claim 1, characterized in that The failure modes include camera failure mode, lidar failure mode, ultrasonic radar failure mode, IMU failure mode, RTK failure mode and millimeter wave radar failure mode.

8. The method according to claim 7, characterized in that The camera failure modes include: Gaussian image noise failure, salt and pepper image noise failure, gamma image noise failure, Poisson image noise failure, uniform image noise failure, Ruili image noise failure, rain and fog noise failure, intermittent image noise failure and disconnection failure; The laser radar failure modes include: high and low density noise failures in sandstorm weather, multiple density noise failures in rainy weather, disconnection failures, and intermittent failures; The ultrasonic radar failure modes include: random error distance failure, distance attenuation failure, disconnection failure and intermittent noise failure; The IMU failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure; The RTK failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure; and The millimeter wave radar failure modes include: deviation noise failure, random noise failure, disconnection failure and intermittent noise failure.

9. A reliability test device for an autonomous driving perception system based on fault injection, characterized in that: The device comprises: A construction module, used to construct a key scene library, wherein the key scene library includes a plurality of key scenes; A first acquisition module is used to match corresponding key scenarios from the key scenario library as test scenarios and determine the test pass criteria according to the reliability test requirements of the perception system functions claimed by the manufacturer; The second acquisition module is used to analyze the failure mode corresponding to the perception function claimed by the manufacturer in the test scenario and determine the key sensors and failure signals to be injected into the failure; A third acquisition module is used to inject the fault signal into the corresponding key sensor signal without fault to obtain the sensor signal with fault; A testing module, configured to test the perception system function claimed by the manufacturer based on the key sensor signal without fault and the sensor signal with fault in the test scenario, and obtain the reliability of the perception system function in the test scenario according to the standard of passing the test; and The calculation module is used to calculate the total reliability of the perception system and its reliability level based on the reliability of the perception system function in each test scenario.

10. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 8.

Images