Intelligent perception system safety and reliability evaluation system and device

Abstract

The present invention provides a safety and reliability evaluation system for an intelligent perception system, which can be applied to the field of autonomous driving evaluation technology, and includes: a data acquisition module for acquiring initially fault-free sensor data and transmitting the initially fault-free sensor data to the intelligent perception system; a fault injection module for injecting fault data into the initially fault-free sensor data to obtain sensor fault data, and transmitting the sensor fault data to the intelligent perception system; a result analysis module for receiving and analyzing test data of the intelligent perception system to obtain test results; and an interactive module for controlling the operation of the data acquisition module, the fault injection module, and the result analysis module. Fault information in different formats is generated for data formats of different types of sensors, which can realize real-time sensor data reading, fault signal injection, and data recording and analysis. The present invention also provides a safety and reliability evaluation device for an intelligent perception system.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving evaluation, and in particular to a safety and reliability evaluation system and device for an intelligent perception system. Background Art

[0002] Autonomous vehicles rely on a variety of sensors to perceive and navigate their environment, including cameras, laser range finders (LiDAR), millimeter-wave radar (Radar), ultrasonic sensors, real-time tracking (RTK), and inertial measurement units (IMUs). LiDAR provides precise distance and shape information by emitting laser light and measuring its reflection time. Millimeter-wave radar uses electromagnetic waves in the millimeter-wave band to measure the distance, speed, and angle of objects, making it particularly suitable for use in adverse weather conditions. Cameras capture images to identify traffic signs, lane markings, vehicles, and other targets. Ultrasonic sensors detect close-range obstacles in low-speed environments. RTK data is used for navigation and path planning. IMUs monitor the vehicle's speed, direction, and inclination, assisting RTK in providing more precise positioning. These sensors work together, using multi-sensor data fusion technology, to ensure that autonomous driving systems can operate safely and efficiently, maintaining high functionality and accuracy even in complex driving environments.

[0003] However, despite the high level of integration and technological advantages these sensors offer, they can still experience various faults, such as sensor failure, data inconsistency, or signal interference. The perception system's fault tolerance and diagnostic capabilities for these faults are crucial factors influencing the safety and reliability of autonomous driving. Therefore, it is necessary to test and verify the perception system's performance under the conditions of some sensor failures.

[0004] Current testing systems primarily assume normal and undisturbed sensor data, or rely solely on limited environmental interference conditions such as rain and fog contained in the dataset. This approach makes it difficult to fully cover all possible fault types. Furthermore, the diverse sensor information types and the sheer volume of data make real-time data reception and transmission difficult. Consequently, it is imperative to inject different types of sensor faults in real time for each type of sensor information. Summary of the Invention

[0005] (1) Technical issues to be resolved

[0006] In order to solve at least one of the above-mentioned technical problems arising from the safety and reliability testing of autonomous driving perception systems in the prior art, an embodiment of the present invention provides an intelligent perception system safety and reliability evaluation system and device, which generates fault information in different formats according to the data formats of different types of sensors, and can realize real-time sensor data reading, fault signal injection, and data recording and analysis, further assisting in a more comprehensive fault tolerance and fault diagnosis capability evaluation at the perception system level, so as to test the design redundancy and adaptive recovery mechanism of the algorithm in response to possible faults, and further improve the safety and reliability of the system in the face of faults.

[0007] (2) Technical solution

[0008] In response to the above technical problems, embodiments of the present invention provide a security and reliability evaluation system and device for an intelligent perception system.

[0009] According to a first aspect of the present invention, a safety and reliability evaluation system for an intelligent perception system is provided, comprising: a data acquisition module for acquiring initially fault-free sensor data and transmitting the initially fault-free sensor data to the intelligent perception system; a fault injection module for injecting fault data into the initially fault-free sensor data to obtain sensor fault data, and transmitting the sensor fault data to the intelligent perception system; a result analysis module for receiving and analyzing test data of the intelligent perception system to obtain test results; and an interactive module for controlling the operation of the data acquisition module, the fault injection module, and the result analysis module.

[0010] In some exemplary embodiments, the sensor data collected by the data acquisition module includes lidar point cloud data, camera image data, IMU data, millimeter wave radar data, ultrasonic radar data, and RTK data.

[0011] In some exemplary embodiments, the fault data injected into the point cloud data includes random noise faults, disconnection faults, and intermittent faults; the fault data injected into the image includes Gaussian noise faults, salt and pepper noise faults, gamma noise faults, uniform noise faults, Rayleigh noise faults, Poisson noise faults, rain and fog noise faults, disconnection faults, and intermittent noise faults; the fault data injected into the IMU data includes deviation noise faults, random noise faults, disconnection faults, and intermittent faults; the fault data injected into the millimeter wave radar data includes random noise faults, disconnection faults, and intermittent faults; the fault data injected into the ultrasonic radar data includes random error distance faults, distance attenuation faults, disconnection faults, and intermittent faults; and the fault data injected into the RTK data includes deviation noise faults, random noise faults, disconnection faults, and intermittent faults.

[0012] In some exemplary embodiments, the interactive module includes a sensor program startup interface, a data display interface, and a data recording interface, wherein the sensor program startup interface is used to start the sensor driver and operate the sensor to publish the sensor data it collects; the data display interface is used to display the sensor data and fault injection status; and the data recording interface is used to record the collected sensor normal status data and sensor fault status data.

[0013] In some exemplary embodiments, the data acquisition module is capable of saving initial fault-free sensor data to enable offline storage and viewing of the initial fault-free sensor data; the fault injection module is capable of saving sensor fault data to enable offline storage and viewing of the sensor fault data; and the result analysis module is capable of saving test data and test results to enable offline storage and viewing of the test data and test results.

[0014] According to a second aspect of the present invention, a safety and reliability evaluation device for an intelligent perception system based on the above-mentioned evaluation system is provided, comprising: a sensor for generating initial fault-free sensor data; a controller for collecting the initial fault-free sensor data, injecting fault data into the initial fault-free sensor data to obtain sensor fault data, and transmitting the initial fault-free sensor data and the sensor fault data to the intelligent perception system, and receiving and analyzing the test data of the intelligent perception system to obtain test results; a display for displaying an operation interface to realize human-computer interaction; and a power supply for supplying power to the sensor, the controller and the display, wherein the operation interface includes a sensor program startup interface, a data display interface and a data recording interface.

[0015] In some exemplary embodiments, the sensors include: a lidar for collecting point cloud data; an IMU for collecting inertial measurement unit data containing inertial attitude; an RTK for collecting differential positioning signal data containing positioning information; a camera for collecting image data; a millimeter-wave radar for collecting millimeter-wave radar data containing position information; and an ultrasonic radar for collecting ultrasonic radar data containing position information.

[0016] In some exemplary embodiments, a quick-release mechanism for integrated sensors of laser radar, IMU, RTK, camera, millimeter-wave radar and ultrasonic radar is provided, and the housing of the sensor quick-release mechanism is provided with holes for the camera, IMU, millimeter-wave radar, ultrasonic radar, RTK, laser radar and ultrasonic radar harness.

[0017] In some exemplary embodiments, the device also includes a shell, which includes a three-layer structure from top to bottom, wherein the top of the first layer includes a liftable bracket, which is used to place a display, and the liftable bracket is equipped with a folding hinge mechanism to achieve storage of the display; a switch display area is set on the side of the first layer, and the switch display area includes an emergency stop button and a main power switch, and the main power switch includes a DC switch and an AC switch. Corresponding DC indicator lights and AC indicator lights are set between the switch and the emergency stop button, and the switch display area is used to control the start and stop of the device and judge the operating status of the device; the bottom of the second layer includes a pull-out tray, which is used to place a sensor quick-release mechanism; and the third layer is used to place a controller, and the third layer is connected to the second layer to facilitate the arrangement of sensor wiring harnesses.

[0018] In some exemplary embodiments, the housing further comprises: a power bus interface for connecting to a power source to power the sensor, controller, and display; an exhaust vent for dissipating heat from the sensor quick-release mechanism and controller; a storage layer disposed between the first and second layers; and a wheel assembly structure for moving the device. According to a fourth aspect of the present invention, a storage medium is provided, characterized in that the storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the aforementioned method.

[0019] (3) Beneficial effects

[0020] As can be seen from the above technical solutions, the intelligent perception system security and reliability evaluation system and device provided by the embodiments of the present invention have at least the following beneficial effects:

[0021] (1) It can realize the time synchronization of multi-sensor serial data transmission and reception, and can also realize the offline storage and viewing of six types of sensor data.

[0022] (2) Fault noise is simulated based on the physical model on the ROS platform, and faults are injected into the above sensor data, including hardware faults such as circuit disconnection and circuit jump, and environmental faults such as sandstorms, rain and fog. The fault tolerance and fault diagnosis capabilities of the perception system are fully tested and verified. For the simulation of circuit noise, circuit disconnection is indicated by interrupting the sensor data stream and setting the data to send empty data. The signal loss when the circuit is disconnected is simulated by combining the physical event trigger mechanism. Circuit jump is achieved by randomly or regularly injecting Gaussian or Poisson noise into the data stream, and causing abnormal values ​​or mutations in the sensor data to simulate signal instability. For the simulation of environmental noise, sandstorms can be achieved by injecting random noise into the LiDAR point cloud data or losing some point clouds. At the same time, blur or granular noise is added to the camera image stream to reduce the contrast and simulate visual blur. In rain and fog, the contrast and resolution of the image are reduced by superimposing a dynamic water droplet layer on the optical sensor data, and signal attenuation and point cloud loss are simulated in the LiDAR data to achieve the effect of reduced sensor perception ability.

[0023] (3) The operation interface can realize the start and stop of different sensors, real-time data collection and transmission, real-time visualization of normal data, injection of different fault types and real-time visualization of injected fault data. At the same time, it can realize offline storage of data for easy subsequent operation and viewing. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0025] Figure 1 The following schematically shows a structural block diagram of a security and reliability evaluation system for an intelligent perception system according to an embodiment of the present invention;

[0026] Figure 2 A schematic diagram schematically shows the fault types corresponding to sensors according to an embodiment of the present invention;

[0027] Figure 3 A schematic diagram schematically shows a sensor program startup interface according to an embodiment of the present invention;

[0028] Figure 4 A schematic diagram schematically illustrates a sensor data display interface according to an embodiment of the present invention;

[0029] Figure 5 A schematic diagram of a sensor data recording interface according to an embodiment of the present invention is schematically shown;

[0030] Figure 6The following schematically shows a structural diagram of a device for evaluating the safety and reliability of an intelligent perception system according to an embodiment of the present invention;

[0031] Figure 7 A schematic diagram of the structure of an integrated sensor quick-release mechanism according to an embodiment of the present invention is shown;

[0032] Figure 8 The following schematically shows a schematic diagram of the housing structure of a device for evaluating the safety and reliability of an intelligent sensing system according to an embodiment of the present invention;

[0033] Figure 9 Schematically shows a front view of a housing of a device for evaluating safety and reliability of an intelligent sensing system according to an embodiment of the present invention; and

[0034] Figure 10 A side view of a housing of a device for evaluating safety and reliability of an intelligent perception system according to an embodiment of the present invention is schematically shown.

[0035] Reference numerals:

[0036] 100-Sensor quick-release mechanism; 101-Camera hole; 102-IMU hole; 103-Millimeter-wave radar hole; 104-Ultrasonic radar harness hole; 105-Quick-release mechanism fixing hole; 106-Ultrasonic radar hole; 107-RTK hole; 108-LiDAR hole; 200-Casing; 210-First layer; 211-Liftable bracket; 212-Switch display area; 220-Second layer; 221-Pull-out tray; 222-Exhaust vents; 230-Third layer; 240-Storage layer; 250-Wheel structure; 260-Power bus interface. DETAILED DESCRIPTION

[0037] To make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to specific embodiments and the accompanying drawings. It is apparent that the embodiments described are only a portion of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0038] Figure 1 The structural block diagram of an intelligent perception system security and reliability evaluation system 800 according to an embodiment of the present invention is schematically shown.

[0039] like Figure 1As shown, according to an embodiment of the present invention, an intelligent perception system safety and reliability evaluation system 800 according to an embodiment of the present invention includes: a data acquisition module 810, used to acquire initial fault-free sensor data and transmit the initial fault-free sensor data to the intelligent perception system; a fault injection module 820, used to inject fault data into the initial fault-free sensor data to obtain sensor fault data, and transmit the sensor fault data to the intelligent perception system; a result analysis module 830, used to receive and analyze test data of the intelligent perception system to obtain test results; and an interactive module 840, used to control the operation of the data acquisition module 810, the fault injection module 820 and the result analysis module 830.

[0040] In some exemplary embodiments, the sensor data collected by the data acquisition module 810 includes lidar point cloud data, camera image data, IMU data, millimeter wave radar data, ultrasonic radar data, and RTK data.

[0041] In some exemplary embodiments, the fault data injected into the point cloud data includes random noise faults, disconnection faults, and intermittent faults; the fault data injected into the image includes Gaussian noise faults, salt and pepper noise faults, gamma noise faults, uniform noise faults, Rayleigh noise faults, Poisson noise faults, rain and fog noise faults, disconnection faults, and intermittent noise faults; the fault data injected into the IMU data includes deviation noise faults, random noise faults, disconnection faults, and intermittent faults; the fault data injected into the millimeter wave radar data includes random noise faults, disconnection faults, and intermittent faults; the fault data injected into the ultrasonic radar data includes random error distance faults, distance attenuation faults, disconnection faults, and intermittent faults; and the fault data injected into the RTK data includes deviation noise faults, random noise faults, disconnection faults, and intermittent faults.

[0042] In some exemplary embodiments, the data acquisition module 810 is capable of saving initial fault-free sensor data to enable offline storage and viewing of the initial fault-free sensor data; the fault injection module 820 is capable of saving sensor fault data to enable offline storage and viewing of the sensor fault data; the result analysis module 830 is capable of saving test data and test results to enable offline storage and viewing of the test data and test results.

[0043] In some exemplary embodiments, the interactive module 840 includes a sensor program startup interface, a data display interface, and a data recording interface, wherein the sensor program startup interface is used to start the sensor driver and operate the sensor to publish the sensor data it collects; the data display interface is used to display sensor data and fault injection status; and the data recording interface is used to record and collect sensor normal status data and sensor fault status data.

[0044] Figure 2 The diagram schematically shows the fault types corresponding to the sensors according to the embodiment of the present invention.

[0045] like Figure 2 As shown in the figure, fault noise is simulated according to the physical model under the Robot Operating System (ROS), and faults are injected into the above sensor data, including hardware faults such as circuit breaks and circuit jumps, and environmental faults such as sandstorms, rain and fog. 29 fault types can be generated. The designed operating system can realize the collection of different sensor data, the injection of different fault types, and the visualization of injected fault data. The above process realizes the output of normal non-injected fault data and randomly generated sensor fault data, the test data recording of the external perception system, and the offline storage and viewing of sensor data.

[0046] Specifically, the working principles of common sensors in autonomous driving systems and the various types of faults that may occur are classified and analyzed in detail, including:

[0047] Camera failure: Gaussian noise failure, salt and pepper noise failure, gamma noise failure, uniform noise failure, Ruili noise failure, Poisson noise failure, rain and fog noise failure, disconnection failure, and intermittent noise failure, a total of 9 types of failures.

[0048] LiDAR faults: There are five types of faults, including three types of random noise faults simulating dust and rain, disconnection faults, and intermittent faults.

[0049] Ultrasonic faults: random error distance fault, distance attenuation fault, disconnection fault, and intermittent fault, a total of 4 types of faults.

[0050] IMU faults: bias noise fault, random noise fault, disconnection fault, and intermittent fault, a total of 4 types of faults.

[0051] RTK faults: deviation noise fault, random noise fault, disconnection fault, and intermittent fault, a total of 4 types of faults.

[0052] Millimeter wave faults: random noise fault, disconnection fault, and intermittent fault, a total of 3 types of faults.

[0053] Achieve data injection of 29 types of fault forms, covering the above six types of sensor data.

[0054] Specifically, the fault generation method is as follows:

[0055] Detailed fault generation is performed for each sensor based on the noise mathematical model corresponding to the sensor fault type. First, the noise mathematical model is clarified as follows:

[0056] The mathematical representation of Gaussian noise is as follows:

[0057] Normal distribution: The noise value n of Gaussian noise obeys a mean of μ and a standard deviation of Normal distribution of:

[0058] (1)

[0059] in, represents the normal distribution, μ and are the mean and standard deviation respectively.

[0060] Gaussian noise is generally considered to be additive noise, that is, the noise value is directly added to the image pixel value:

[0061] (2)

[0062] in, is the original image pixel value, is the pixel value after adding noise, is Gaussian noise.

[0063] Mathematical representation of uniform noise:

[0064] Let the original image be , the image with uniform noise is , then the uniform noise can be expressed as:

[0065] (3)

[0066] in: is the pixel value of the original image, is uniform noise, obeying uniform distribution , where a and b are the lower and upper bounds of the uniform distribution, respectively.

[0067] The probability density function of the uniform distribution is:

[0068] (4)

[0069] Mathematical representation of deviation noise:

[0070] Bias noise is a common fixed deviation or drift in IMU. Its mathematical model is expressed as

[0071] (5)

[0072] (6)

[0073] in: and are the measured acceleration and angular velocity, and are the real acceleration and angular velocity, and is a constant bias, representing the inherent bias of the IMU sensor, and is random noise, usually assumed to be Gaussian noise with mean 0 and .

[0074] This model assumes that the IMU has fixed bias and is also affected by random noise. After clarifying the mathematical models for various types of noise, fault injection is performed based on project requirements and the operating principles of each sensor.

[0075] For the Gaussian noise of the above image, the image pixels are normalized to an independent value n, and a set of standard normal distributed random numbers are generated based on the random number generator. These standard normal distributed random numbers are converted into a random number with a specified mean through linear transformation. and standard deviation Gaussian noise.

[0076] The generation formula is as follows:

[0077] (7)

[0078] Here, z is a random number from a standard normal distribution.

[0079] For the uniform noise in the above image, in the project application, determine the noise range: select the range of uniform noise , generate uniformly distributed random numbers: use the random number generator to generate uniformly distributed random numbers The generated uniform noise value is superimposed on the pixel value of the original image to form an image with uniform noise.

[0080] To address the aforementioned IMU bias noise, random bias noise is added to the linear acceleration. The bias value for each axis is randomly generated within a specified range and added to the corresponding attitude axis of the fault data.

[0081] The recorded PCD point cloud files, png image files, IMU data, RTK latitude and longitude, ultrasonic radar data, and millimeter wave radar data can be generated as xlsx files based on the rosbag data package.

[0082] Interactive Module 840 provides sensor startup and shutdown functions for LiDAR, camera, IMU, RTK, ultrasonic radar, and millimeter-wave radar, as well as corresponding fault injection program startup and shutdown functions. Users can select which functions to enable as needed. Interactive Module 840 includes a sensor program startup interface, a data display interface, and a data recording interface.

[0083] Users can launch the corresponding sensor driver through the sensor program startup interface and operate the sensor to publish the data topics it collects. The data display interface can visualize the data of each sensor, showing the sensor's operating data and fault injection status. The data recording interface can record and collect sensor data in normal and faulty states. These functions allow users to flexibly select and manage the sensor data they want to record for subsequent analysis and application.

[0084] Figure 3 The figure schematically shows a sensor program startup interface according to an embodiment of the present invention.

[0085] like Figure 3 As shown, the sensor program startup interface according to an embodiment of the present invention provides sensor start and stop functions for lidar, camera, IMU, RTK, ultrasonic radar and millimeter wave radar, as well as corresponding fault injection program start and stop functions. Users can choose to turn on the functions as needed. After clicking the "Open" button of the specified sensor, the system will start the corresponding sensor driver and start publishing the data topic it collects, and at the same time feed back the sensor drive information to the sensor status bar. After clicking the "Open" button of the specified fault injection program, the system starts the fault injection program of the corresponding sensor, subscribes to the data collected by the sensor, and publishes the corresponding fault program after processing. In addition, when the "Open" button is clicked, it will enter a locked state to prevent the program from being started repeatedly and causing conflicts until the corresponding "Close" button is clicked to unlock it.

[0086] Figure 4 The figure schematically shows a sensor data display interface according to an embodiment of the present invention.

[0087] like Figure 4 As shown, the sensor data display interface according to an embodiment of the present invention includes an image data area, a point cloud data area, an IMU data area, an RTK data area, an ultrasonic radar data area, and a millimeter-wave radar data area. Click the "Status" button to enter the data display page, which displays raw sensor data, fault data, and corresponding fault types. Each area also has a "Refresh" button.

[0088] The image data area displays the original image on the left and the image after fault injection on the right. The fault type is displayed in real time above the fault image, including Gaussian noise, salt and pepper noise, gamma noise, uniform noise, Rayleigh noise, Poisson noise, disconnection, and discontinuity. During runtime, clicking "Refresh" resets the area to its original state and re-injects data. The point cloud data area displays the original point cloud in white, and the fault point cloud in red. The display angle can be controlled by panning, rotating, zooming in, and zooming out with the mouse. The fault type is displayed in real time in the upper left corner of the data display area, including random noise, simulated dust, simulated rain, zero cycle, disconnection, and discontinuity. The IMU data area displays quaternions, three-axis acceleration, and angular velocity. It can simultaneously display the original data, fault data, and the difference between the two, including deviation noise, random noise, zero cycle, and disconnection. The RTK data area displays the original longitude and latitude data, the faulted longitude and latitude data, and the difference between the two. The fault type is displayed in real time in the upper left corner of the data display area, including Gaussian noise, random noise, disconnection, and discontinuity. During runtime, click "Refresh" to reset the area to its original state and then receive data again. The data displayed in the ultrasonic data area includes the original distance data measured by the four ultrasonic sensors, the fault data, and the difference between the two, in mm, with a minimum measurement distance of 250. The fault type is displayed in real time in the upper left corner of the data display area, including deviation noise, random noise, zero cycle, and disconnection. The millimeter wave data area displays the first six target data detected by the millimeter wave radar, and "NULL" is displayed for the insufficient part. The displayed data includes the target ID, the original XY coordinates, the XY coordinates of the fault, and the corresponding difference. The fault type is displayed in real time in the upper left corner of the data display area, including deviation noise, random noise, zero cycle, and disconnection. During runtime, click "Refresh" to reset the area to its original state and then receive data again.

[0089] Figure 5 The figure schematically shows a sensor data recording interface according to an embodiment of the present invention.

[0090] like Figure 5 As shown, the sensor data recording interface according to an embodiment of the present invention is used to display and record sensor data. The "Topic List" on the page displays the topic data in the current ROS system, with the first column containing the topic name and the second column containing the topic format. Clicking the "Refresh" button refreshes the current data topic in the "Topic List." The "Recording List" displays the topic data to be recorded, using the same format as the "Topic List." The "Output Bar" displays system information generated during the recording process.

[0091] During user operation, after selecting a topic in the "Topic List" and clicking "Add", the selected topic will be moved to the "Recording List". After selecting a topic in the "Recording List", clicking "Remove" will remove the selected topic from the "Recording List". After clicking the "Record" button, the system will begin recording the topic data in the "Recording List". After clicking the "End" button, the system ends recording and stores the recorded topic data in the system-specified directory. Through these functions, users can flexibly select and manage the sensor data to be recorded for subsequent analysis and application.

[0092] Figure 6 The following schematically shows a schematic diagram of a security and reliability evaluation device for an intelligent perception system according to an embodiment of the present invention.

[0093] like Figure 6 As shown, a security reliability evaluation device for an intelligent perception system according to an embodiment of the present invention uses Figure 1 The evaluation system shown includes: a sensor for generating initially fault-free sensor data; a controller for collecting the initially fault-free sensor data, injecting fault data into the initially fault-free sensor data to obtain sensor fault data, transmitting the initial fault-free sensor data and the sensor fault data to an intelligent perception system, and receiving and analyzing test data from the intelligent perception system to obtain test results; a display for displaying an operation interface to enable human-computer interaction; and a power supply for powering the sensor, controller, and display. The operation interface includes a sensor program startup interface, a data display interface, and a data recording interface. Optionally, the controller includes an industrial computer.

[0094] In some exemplary embodiments, the sensors include: a lidar for collecting point cloud data; an IMU for collecting inertial measurement unit data containing inertial attitude; an RTK for collecting differential positioning signal data containing positioning information; a camera for collecting image data; a millimeter-wave radar for collecting millimeter-wave radar data containing position information; and an ultrasonic radar for collecting ultrasonic radar data containing position information.

[0095] Figure 7 The structural diagram of an integrated sensor quick-release mechanism 100 according to an embodiment of the present invention is schematically shown.

[0096] like Figure 7As shown, a quick-release mechanism 100 integrates a lidar, IMU, RTK, camera, millimeter-wave radar, and ultrasonic radar. The housing of the sensor quick-release mechanism 100 is equipped with holes 101 for the camera, 102 for the IMU, 103 for the millimeter-wave radar, 106 for the ultrasonic radar, 107 for the RTK, 108 for the lidar, and 104 for the ultrasonic radar wiring harness to ensure neat wiring. Optionally, the bottom of the integrated sensor quick-release mechanism 100 is equipped with quick-release mechanism fixing holes 105 for securing the integrated sensor quick-release mechanism 100. The sensor quick-release mechanism 100 is constructed of an aluminum profile frame and aluminum plate housing, providing superior rigidity.

[0097] Figure 8 The following schematically shows a schematic diagram of the housing structure of a device for evaluating the safety and reliability of an intelligent sensing system according to an embodiment of the present invention; Figure 9 Schematically shows a front view of a housing of a device for evaluating safety and reliability of an intelligent sensing system according to an embodiment of the present invention; and Figure 10 A side view of a housing of a device for evaluating safety and reliability of an intelligent perception system according to an embodiment of the present invention is schematically shown.

[0098] like Figure 8 、 Figure 9 as well as Figure 10 As shown, a housing 200 of a safety and reliability evaluation device for an intelligent sensing system according to an embodiment of the present invention includes a three-layer structure from top to bottom, wherein the top of the first layer 210 includes a liftable bracket 211, which is used to place a display and is equipped with a folding hinge mechanism to achieve storage of the display; a switch display area 212 is provided on the side of the first layer 210, and the switch display area 212 includes an emergency stop button and a main power switch. The main power switch includes a DC switch and an AC switch. Corresponding DC and AC indicator lights are provided between the switch and the emergency stop button. The switch display area 212 is used to control the start and stop of the device and determine the operating status of the device; the bottom of the second layer 220 includes a pull-out tray 221, which is used to place the sensor quick-release mechanism 100. When in use, the tray can be pulled out. If actual vehicle testing is required, the integrated sensor mechanism can be conveniently removed completely through the quick-release device; and the third layer 230 is used to place the controller. The third layer 230 is connected to the second layer 220 to facilitate the arrangement of the sensor harness. Optionally, the housing 200 is made of metal and spray-painted with silver paint.

[0099] In some exemplary embodiments, the housing 200 also includes: a power bus interface 260 for connecting a power source to power the sensor, controller, and display; an exhaust hole 222 for dissipating heat from the sensor quick-release mechanism 100 and the controller; a storage layer 240, disposed between the first layer 210 and the second layer 220. Optionally, a slide rail is installed under the storage layer 240, which is opened by pulling and sliding, and is simple and convenient to operate; and a wheel structure 250 for moving the device.

[0100] For example, the sensor quick-release mechanism 100 consists of a lidar for collecting point cloud information, an IMU for collecting inertial attitude information, an RTK for collecting positioning information, a camera for collecting image information, and millimeter-wave radar and ultrasonic radar for collecting position information. Sensor operation instructions and fault occurrence instructions are input to a controller (industrial computer) via a mouse and keyboard. The sensor quick-release mechanism 100 transmits the data to the controller (industrial computer) via a sensor data cable, and the display ultimately displays sensor operating and fault information.

[0101] The multimodal intelligent perception system safety and reliability assessment device designed according to an embodiment of the present invention can realize multi-sensor data collection, and can also implement various types of fault injection and data visualization for each sensor. The data collected by the device already covers lidar point cloud, camera images, IMU, millimeter-wave radar, ultrasonic radar, and RTK data. The data can be recorded in real time and viewed offline. It can realize real-time injection of 29 types of faults for the above six types of sensor data. Based on the rosbag data packet, the recorded PCD point cloud file containing the time series, png image file, and IMU data, RTK latitude and longitude, ultrasonic radar data, and millimeter-wave radar data can be generated. XLSX file.

[0102] The above specific embodiments further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An intelligent perception system security and reliability evaluation system, characterized by: include: The data acquisition module is used to collect initial fault-free sensor data using lidar, camera, IMU, millimeter-wave radar, ultrasonic radar and RTK, and transmit the initial fault-free sensor data to the intelligent perception system to achieve multi-sensor data time synchronization. The sensor data collected by the data acquisition module includes lidar point cloud data, camera image data, IMU data, millimeter wave radar data, ultrasonic radar data and RTK data; a fault injection module, configured to simulate fault noise according to a physical model, generate a fault type for the initially fault-free sensor data, superimpose the fault noise on the initially fault-free sensor data to inject fault data, obtain sensor fault data, and transmit the sensor fault data to the intelligent perception system; The fault types include hardware faults and environmental faults. The hardware faults include circuit breakage and circuit jump. The environmental faults include dusty weather and rainy and foggy weather. The fault data injection includes: The fault data injected into the point cloud data includes random noise faults, disconnection faults and intermittent faults; The fault data injected into the image includes Gaussian noise fault, salt and pepper noise fault, gamma noise fault, uniform noise fault, Ruili noise fault, Poisson noise fault, rain and fog noise fault, disconnection fault and intermittent noise fault; The fault data injected by the IMU data includes deviation noise fault, random noise fault, disconnection fault and intermittent fault; The fault data injected by the millimeter wave radar data includes random noise fault, disconnection fault and intermittent fault; The fault data injected by the ultrasonic radar data includes random error distance fault, distance attenuation fault, disconnection fault and intermittent fault; and The fault data injected into the RTK data includes deviation noise fault, random noise fault, disconnection fault and intermittent fault; The processing of the fault type includes generating a fault for each sensor according to a noise mathematical model corresponding to the fault type: The circuit is broken, by interrupting the sensor data or sending the disconnection fault data; The circuit jump causes abnormal values ​​or mutations to appear in the sensor data by randomly or periodically injecting the Gaussian noise fault or the Poisson noise fault into the sensor data; In the dusty weather, random noise faults are injected into the LiDAR point cloud data or part of the point cloud is lost, and salt and pepper noise faults are added at the same time; In the rainy and foggy weather, the distance attenuation fault or the rainy and foggy noise fault occurs by superimposing a dynamic water drop layer on the sensor data; A result analysis module is used to receive and analyze the test data of the intelligent perception system to obtain the test results; and The interactive module is used to control the operations of the data acquisition module, the fault injection module and the result analysis module.

2. The evaluation system according to claim 1, characterized in that: The interactive module includes a sensor program startup interface, a data display interface, and a data recording interface. The sensor program startup interface is used to start the sensor driver and operate the sensor to publish the sensor data it collects; The data display interface is used to display the sensor data and fault injection status; and The data recording interface is used to record and collect the normal state data of the sensor and the fault state data of the sensor.

3. The evaluation system according to claim 1, characterized in that: The data acquisition module is capable of saving the initial fault-free sensor data to enable offline storage and viewing of the initial fault-free sensor data; The fault injection module is capable of saving the sensor fault data to enable offline storage and viewing of the sensor fault data; The result analysis module can save the test data and the test results to achieve offline storage and viewing of the test data and the test results.

4. An intelligent perception system security and reliability evaluation device based on the evaluation system according to any one of claims 1 to 3, characterized in that: include: Sensors for generating initial fault-free sensor data; a controller, configured to collect the initially fault-free sensor data, inject fault data into the initially fault-free sensor data to obtain sensor fault data, transmit the initially fault-free sensor data and the sensor fault data to an intelligent perception system, and receive and analyze test data from the intelligent perception system to obtain a test result; Display, used to display the operation interface and realize human-computer interaction; as well as a power supply for supplying power to the sensor, the controller, and the display; The operation interface includes a sensor program startup interface, a data display interface, and a data recording interface.

5. The device according to claim 4, characterized in that The sensor comprises: LiDAR, used to collect point cloud data; IMU, used to collect inertial measurement unit data including inertial attitude; RTK, used to collect differential positioning signal data containing positioning information A camera, used to collect image data; Millimeter-wave radar, used to collect millimeter-wave radar data containing position information; and Ultrasonic radar, used to collect ultrasonic radar data containing position information.

6. The device according to claim 5, characterized in that The laser radar, the IMU, the RTK, the camera, the millimeter-wave radar and the ultrasonic radar are integrated with a sensor quick-release mechanism, and the housing of the sensor quick-release mechanism is provided with camera holes, IMU holes, millimeter-wave radar holes, ultrasonic radar holes, RTK holes, laser radar holes and ultrasonic radar wiring harness holes.

7. The device according to claim 6, characterized in that The device further comprises a housing, The shell comprises a three-layer structure from top to bottom, wherein: The top of the first layer includes a liftable bracket, the liftable bracket is used to place the display, and the liftable bracket is equipped with a folding hinge mechanism to achieve storage of the display; A switch display area is set on the side of the first layer. The switch display area includes an emergency stop button and a main power switch. The main power switch includes a DC switch and an AC switch. Corresponding DC and AC indicator lights are set between the switch and the emergency stop button. The switch display area is used to control the start and stop of the device and determine the operating status of the device; The bottom of the second layer includes a pull-out tray, and the pull-out tray is used to place the sensor quick-release mechanism; and The third layer is used to place the controller, and the third layer is connected to the second layer to facilitate the arrangement of sensor harnesses.

8. The device according to claim 7, characterized in that The housing further comprises: A power bus interface, used to connect to a power source to supply power to the sensor, the controller, and the display; Exhaust holes, used for heat dissipation of the sensor quick-release mechanism and the controller; a storage layer, disposed between the first layer and the second layer; and A wheel set structure is used to move the device.

Images