System and device for evaluating safety and reliability of intelligent sensing system

Abstract

The invention provides a safety and reliability evaluation system for an intelligent sensing system, which can be applied to the technical field of automatic driving evaluation and comprises a data acquisition module for acquiring initial fault-free sensor data and transmitting the initial fault-free sensor data to the intelligent sensing system; the fault injection module is used for performing fault data injection on the initial fault-free sensor data to obtain sensor fault data and transmitting the sensor fault data to the intelligent sensing system; the result analysis module is used for receiving and analyzing the test data of the intelligent sensing system to obtain a test result; and the interactive module is used for controlling the work of the data acquisition module, the fault injection module and the result analysis module. Fault information of different formats is generated according to data formats of different types of sensors, and real-time sensor data reading, fault signal injection and data recording and analysis can be achieved. The invention further provides a device for evaluating the safety and reliability of the intelligent sensing system.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving evaluation, and in particular to a safety and reliability evaluation system and device for an intelligent perception system. Background Art

[0002] Self-driving cars rely on a variety of sensors to perceive and navigate the environment, including cameras, LiDAR, millimeter-wave radar, ultrasonic sensors, differential positioning signals (RTK), and inertial measurement units (IMUs). LiDAR provides precise distance and shape information by emitting lasers and measuring their reflection time; millimeter-wave radar uses electromagnetic waves in the millimeter wave band to measure the distance, speed, and angle of objects, which is particularly suitable for severe weather conditions; cameras capture images to identify traffic signs, lane lines, vehicles, and other targets; ultrasonic sensors detect close obstacles in low-speed environments; RTK data is used for navigation and path planning; IMU monitors the speed, direction, and inclination of the vehicle, assisting RTK in providing more accurate positioning. These sensors work together to ensure that the autonomous driving system can operate safely and efficiently through multi-sensor data fusion technology, and maintain high functionality and accuracy even in complex driving environments.

[0003] However, despite the high degree of integration and technical advantages these sensors offer, they may still encounter a variety of faults, such as sensor failure, inconsistent data, or signal interference. The fault tolerance and diagnostic capabilities of the perception system to these faults are important factors affecting the safety and reliability of autonomous driving. Therefore, it is necessary to test and verify the performance of the perception system when some sensors fail.

[0004] Current test systems mainly assume that sensor data is normal and undisturbed, or only rely on limited environmental interference conditions such as rain and fog contained in the data set for testing. This approach makes it difficult to ensure that all possible fault types can be fully covered. In addition, the types of sensor perception information vary, and the complex amount of data makes it difficult to achieve real-time data reception and transmission. It is urgent to inject different types of perception faults in real time for different types of perception information. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In order to solve at least one of the above-mentioned technical problems arising from the safety and reliability testing of autonomous driving perception systems in the prior art, an embodiment of the present invention provides an intelligent perception system safety and reliability evaluation system and device, which generates fault information in different formats according to the data formats of different types of sensors, and can realize real-time sensor data reading, fault signal injection, and data recording and analysis, and further assist in a more comprehensive fault tolerance and fault diagnosis capability evaluation at the perception system level, so as to test the design redundancy and adaptive recovery mechanism of the algorithm in response to possible faults, and further improve the safety and reliability of the system in the face of faults.

[0007] (II) Technical solution

[0008] In view of the above technical problems, an embodiment of the present invention proposes a security reliability evaluation system and device for an intelligent perception system.

[0009] According to the first aspect of the present invention, a safety and reliability evaluation system for an intelligent perception system is provided, comprising: a data acquisition module, used to acquire initial fault-free sensor data, and transmit the initial fault-free sensor data to the intelligent perception system; a fault injection module, used to inject fault data into the initial fault-free sensor data, obtain sensor fault data, and transmit the sensor fault data to the intelligent perception system; a result analysis module, used to receive and analyze test data of the intelligent perception system to obtain test results; and an interactive module, used to control the operation of the data acquisition module, the fault injection module and the result analysis module.

[0010] In some exemplary embodiments, the sensor data collected by the data acquisition module includes lidar point cloud data, camera image data, IMU data, millimeter wave radar data, ultrasonic radar data and RTK data.

[0011] In some exemplary embodiments, the fault data injected into the point cloud data includes random noise faults, disconnection faults and intermittent faults; the fault data injected into the image includes Gaussian noise faults, salt and pepper noise faults, gamma noise faults, uniform noise faults, Rayleigh noise faults, Poisson noise faults, rain and fog noise faults, disconnection faults and intermittent noise faults; the fault data injected into the IMU data includes deviation noise faults, random noise faults, disconnection faults and intermittent faults; the fault data injected into the millimeter wave radar data includes random noise faults, disconnection faults and intermittent faults; the fault data injected into the ultrasonic radar data includes random error distance faults, distance attenuation faults, disconnection faults and intermittent faults; and the fault data injected into the RTK data includes deviation noise faults, random noise faults, disconnection faults and intermittent faults.

[0012] In some exemplary embodiments, the interactive module includes a sensor program startup interface, a data display interface, and a data recording interface, wherein the sensor program startup interface is used to start the sensor driver and operate the sensor to publish the sensor data it collects; the data display interface is used to display the sensor data and fault injection status; and the data recording interface is used to record the collected sensor normal status data and sensor fault status data.

[0013] In some exemplary embodiments, the data acquisition module is capable of saving initial fault-free sensor data to achieve offline storage and viewing of the initial fault-free sensor data; the fault injection module is capable of saving sensor fault data to achieve offline storage and viewing of the sensor fault data; the result analysis module is capable of saving test data and test results to achieve offline storage and viewing of the test data and test results.

[0014] According to a second aspect of the present invention, there is provided a safety and reliability evaluation device for an intelligent perception system based on the above-mentioned evaluation system, comprising: a sensor for generating initial fault-free sensor data; a controller for collecting the initial fault-free sensor data, injecting fault data into the initial fault-free sensor data to obtain sensor fault data, and transmitting the initial fault-free sensor data and the sensor fault data to the intelligent perception system, and receiving and analyzing the test data of the intelligent perception system to obtain test results; a display for displaying an operation interface to realize human-computer interaction; and a power supply for supplying power to the sensor, the controller and the display, wherein the operation interface includes a sensor program startup interface, a data display interface and a data recording interface.

[0015] In some exemplary embodiments, the sensors include: a lidar for collecting point cloud data; an IMU for collecting inertial measurement unit data containing inertial attitude; an RTK for collecting differential positioning signal data containing positioning information; a camera for collecting image data; a millimeter wave radar for collecting millimeter wave radar data containing position information; and an ultrasonic radar for collecting ultrasonic radar data containing position information.

[0016] In some exemplary embodiments, laser radar, IMU, RTK, camera, millimeter-wave radar and ultrasonic radar are integrated with a sensor quick-release mechanism, and the shell of the sensor quick-release mechanism is provided with camera holes, IMU holes, millimeter-wave radar holes, ultrasonic radar holes, RTK holes, laser radar holes and ultrasonic radar wiring harness holes.

[0017] In some exemplary embodiments, the device also includes a shell, which includes a three-layer structure from top to bottom, wherein the top of the first layer includes a liftable bracket, which is used to place a display, and the liftable bracket is equipped with a folding hinge mechanism to achieve storage of the display; a switch display area is set on the side of the first layer, and the switch display area includes an emergency stop button and a main power switch, and the main power switch includes a DC switch and an AC switch. Corresponding DC indicator lights and AC indicator lights are set between the switch and the emergency stop button, and the switch display area is used to control the start and stop of the device and judge the operating status of the device; the bottom of the second layer includes a pull-out tray, which is used to place a sensor quick-release mechanism; and the third layer is used to place a controller, and the third layer is connected to the second layer to facilitate the arrangement of sensor wiring harnesses.

[0018] In some exemplary embodiments, the housing further comprises: a power bus interface for connecting a power source to power the sensor, the controller, and the display; an exhaust hole for dissipating heat from the sensor quick-release mechanism and the controller; a storage layer disposed between the first layer and the second layer; and a wheel structure for moving the device. According to a fourth aspect of the present invention, a storage medium is provided, characterized in that: the storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the above method.

[0019] (III) Beneficial effects

[0020] It can be seen from the above technical solutions that the intelligent perception system safety and reliability evaluation system and device provided by the embodiments of the present invention have at least the following beneficial effects:

[0021] (1) It can realize the time-synchronous transmission and reception of multi-sensor serial data, and can also realize the offline storage and viewing of six types of sensor data.

[0022] (2) Fault noise is simulated based on the physical model under the ROS platform, and faults are injected into the above sensor data, including hardware faults such as circuit disconnection and circuit jump, and environmental faults such as sandstorm weather, rain and fog weather, etc., to comprehensively test and verify the fault tolerance and fault diagnosis capabilities of the perception system. For the simulation of circuit noise, circuit disconnection indicates faults by interrupting the sensor data stream and setting data to send empty data. The signal loss when the circuit is disconnected is simulated by combining the physical event trigger mechanism. Circuit jumps inject Gaussian or Poisson noise into the data stream randomly or regularly, and make the sensor data have abnormal values ​​or mutations to simulate signal instability. For the simulation of environmental noise, sandstorm weather can be achieved by injecting random noise into the LiDAR point cloud data or losing part of the point cloud. At the same time, blur or granular noise is added to the camera image stream to reduce the contrast and simulate the visual blur effect. In rain and fog weather, the contrast and resolution of the image are reduced by superimposing a dynamic water droplet layer on the optical sensor data, and the signal attenuation and point cloud loss are simulated in the LiDAR data to achieve the effect of reduced sensor perception ability.

[0023] (3) The operation interface can realize the start and stop of different sensors, real-time data collection and transmission, real-time visualization of normal data, injection of different fault types and real-time visualization of injected fault data. At the same time, it can realize offline data storage for subsequent operation and viewing. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0025] Figure 1 A structural block diagram of a security reliability evaluation system for an intelligent perception system according to an embodiment of the present invention is schematically shown;

[0026] Figure 2 A schematic diagram schematically shows the fault types corresponding to the sensor according to an embodiment of the present invention;

[0027] Figure 3 A schematic diagram schematically shows a sensor program startup interface according to an embodiment of the present invention;

[0028] Figure 4 A schematic diagram schematically shows a sensor data display interface according to an embodiment of the present invention;

[0029] Figure 5 A schematic diagram of a sensor data recording interface according to an embodiment of the present invention is schematically shown;

[0030] Figure 6A schematic diagram of the structure of a security reliability evaluation device for an intelligent perception system according to an embodiment of the present invention is shown;

[0031] Figure 7 A schematic diagram of the structure of an integrated sensor quick-release mechanism according to an embodiment of the present invention is shown;

[0032] Figure 8 A schematic diagram of the shell structure of a device for evaluating the safety and reliability of an intelligent sensing system according to an embodiment of the present invention is shown;

[0033] Fig. 9 A schematic diagram shows a front view of a housing of a device for evaluating safety and reliability of an intelligent sensing system according to an embodiment of the present invention; and

[0034] Fig.10 A side view of a housing of a device for evaluating safety and reliability of an intelligent perception system according to an embodiment of the present invention is schematically shown.

[0035] Reference numerals:

[0036] 100-sensor quick-release mechanism; 101-camera hole; 102-IMU hole; 103-millimeter-wave radar hole; 104-ultrasonic radar harness hole; 105-quick-release mechanism fixing hole; 106-ultrasonic radar hole; 107-RTK hole; 108-lidar hole; 200-housing; 210-first layer; 211-liftable bracket; 212-switch display area; 220-second layer; 221-pull-out tray; 222-exhaust hole; 230-third layer; 240-storage layer; 250-wheel structure; 260-power bus interface. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] Figure 1 The structural block diagram of a security and reliability evaluation system 800 of an intelligent perception system according to an embodiment of the present invention is schematically shown.

[0039] like Figure 1As shown, according to an embodiment of the present invention, a safety and reliability evaluation system 800 of an intelligent perception system according to an embodiment of the present invention includes: a data acquisition module 810, used to acquire initial fault-free sensor data, and transmit the initial fault-free sensor data to the intelligent perception system; a fault injection module 820, used to inject fault data into the initial fault-free sensor data, obtain sensor fault data, and transmit the sensor fault data to the intelligent perception system; a result analysis module 830, used to receive and analyze test data of the intelligent perception system to obtain test results; and an interactive module 840, used to control the work of the data acquisition module 810, the fault injection module 820 and the result analysis module 830.

[0040] In some exemplary embodiments, the sensor data collected by the data collection module 810 includes lidar point cloud data, camera image data, IMU data, millimeter wave radar data, ultrasonic radar data, and RTK data.

[0041] In some exemplary embodiments, the fault data injected into the point cloud data includes random noise faults, disconnection faults and intermittent faults; the fault data injected into the image includes Gaussian noise faults, salt and pepper noise faults, gamma noise faults, uniform noise faults, Rayleigh noise faults, Poisson noise faults, rain and fog noise faults, disconnection faults and intermittent noise faults; the fault data injected into the IMU data includes deviation noise faults, random noise faults, disconnection faults and intermittent faults; the fault data injected into the millimeter wave radar data includes random noise faults, disconnection faults and intermittent faults; the fault data injected into the ultrasonic radar data includes random error distance faults, distance attenuation faults, disconnection faults and intermittent faults; and the fault data injected into the RTK data includes deviation noise faults, random noise faults, disconnection faults and intermittent faults.

[0042] In some exemplary embodiments, the data acquisition module 810 is capable of saving initial fault-free sensor data to achieve offline storage and viewing of the initial fault-free sensor data; the fault injection module 820 is capable of saving sensor fault data to achieve offline storage and viewing of the sensor fault data; the result analysis module 830 is capable of saving test data and test results to achieve offline storage and viewing of the test data and test results.

[0043] In some exemplary embodiments, the interactive module 840 includes a sensor program startup interface, a data display interface, and a data recording interface, wherein the sensor program startup interface is used to start the sensor driver and operate the sensor to publish the sensor data it collects; the data display interface is used to display the sensor data and fault injection status; and the data recording interface is used to record the collected sensor normal status data and sensor fault status data.

[0044] Figure 2 The schematic diagram schematically shows the fault types corresponding to the sensor according to the embodiment of the present invention.

[0045] like Figure 2 As shown in the figure, under the Robot Operating System (ROS), fault noise is simulated according to the physical model, and faults are injected into the above sensor data, including hardware faults such as circuit break and circuit jump, and environmental factor faults such as sandstorm weather, rain and fog weather, and 29 types of faults can be generated; the designed operating system can realize the collection of different sensor data, the injection of different fault types and the visualization of injected fault data; the output of normal non-injected fault data and randomly generated sensor fault data, the test data recording of the external perception system and the offline storage and viewing of sensor data can be realized through the above process.

[0046] Specifically, the working principles of common sensors in autonomous driving systems and the various fault forms that are prone to occur are classified and analyzed in detail, including:

[0047] Camera failure: Gaussian noise failure, salt and pepper noise failure, gamma noise failure, uniform noise failure, Ruili noise failure, Poisson noise failure, rain and fog noise failure, disconnection failure, intermittent noise failure, a total of 9 types of failures.

[0048] LiDAR faults: three types of random noise faults simulating dust and rain, disconnection faults, and intermittent faults, a total of five types of faults.

[0049] Ultrasonic faults: random error distance fault, distance attenuation fault, disconnection fault, intermittent fault, a total of 4 types of faults.

[0050] IMU failures: deviation noise failure, random noise failure, disconnection failure, and intermittent failure, a total of 4 types of failures.

[0051] RTK faults: deviation noise fault, random noise fault, disconnection fault, and intermittent fault, a total of 4 types of faults.

[0052] Millimeter wave faults: random noise fault, disconnection fault, and intermittent fault, a total of 3 types of faults.

[0053] Implement data injection of 29 types of fault forms, covering the above six types of sensor data.

[0054] Specifically, the fault generation method is as follows:

[0055] According to the noise mathematical model corresponding to the sensor fault type, detailed fault generation is performed for each sensor. First, the noise mathematical model is clarified as follows:

[0056] The mathematical representation of Gaussian noise is as follows:

[0057] Normal distribution: The noise value n of Gaussian noise has a mean of μ and a standard deviation of The normal distribution of:

[0058] (1)

[0059] in, represents a normal distribution, μ and are the mean and standard deviation respectively.

[0060] Gaussian noise is usually considered to be additive noise, that is, the noise value is directly added to the image pixel value:

[0061] (2)

[0062] in, is the original image pixel value, is the pixel value after adding noise, is Gaussian noise.

[0063] Mathematical representation of uniform noise:

[0064] Let the original image be , the image with uniform noise is , then the uniform noise can be expressed as:

[0065] (3)

[0066] in: is the pixel value of the original image, is uniform noise, obeying uniform distribution , where a and b are the lower and upper limits of the uniform distribution, respectively.

[0067] The probability density function of the uniform distribution is:

[0068] (4)

[0069] Mathematical representation of deviation noise:

[0070] Bias noise is a common fixed bias or drift in IMU. The mathematical model is expressed as

[0071] (5)

[0072] (6)

[0073] in: and are the measured acceleration and angular velocity, and are the real acceleration and angular velocity, and is a constant deviation, which represents the inherent deviation of the IMU sensor. and is random noise, usually assumed to be Gaussian noise with a mean of 0 and .

[0074] This model assumes that the IMU has a fixed bias and is affected by random noise. After clarifying the mathematical models of various types of noise, fault injection is performed according to project requirements and the working principles of each sensor.

[0075] For the above-mentioned Gaussian noise image, the image pixels are normalized to independent values ​​n, and a set of standard normally distributed random numbers are generated based on the random number generator. These standard normally distributed random numbers are converted into random numbers with a specified mean through linear transformation. and standard deviation Gaussian noise.

[0076] The generation formula is as follows:

[0077] (7)

[0078] Here, z is a random number from a standard normal distribution.

[0079] For the uniform noise in the above image, determine the noise range in the project application: Select the range of uniform noise , Generate uniformly distributed random numbers: Use a random number generator to generate uniformly distributed random numbers The generated uniform noise value is superimposed on the pixel value of the original image to form an image with uniform noise.

[0080] In response to the above IMU bias noise, random bias noise is added to the linear acceleration. The bias value of each axis is randomly generated within the specified range and added to the corresponding attitude axis of the fault data.

[0081] The xlsx file containing the recorded PCD point cloud file, png image file, IMU data, RTK longitude and latitude, ultrasonic radar data, and millimeter wave radar data containing time series can be generated according to the rosbag data package.

[0082] The interactive module 840 provides the start and stop functions of the sensors of the laser radar, camera, IMU, RTK, ultrasonic radar and millimeter wave radar, as well as the corresponding fault injection program start and stop functions. The user can choose to enable the function as needed. The interactive module 840 includes the sensor program startup interface, data display interface, and data recording interface.

[0083] Users can start the corresponding sensor driver through the sensor program startup interface and operate the sensor to publish the data topic it collects; the data display interface can be used to visualize the data of each sensor, display the sensor working data and fault injection status; the data recording interface can be used to record and collect the sensor's normal state data and fault state data. Through these functions, users can flexibly select and manage the sensor data that needs to be recorded for subsequent analysis and application.

[0084] Figure 3 The schematic diagram of the sensor program startup interface according to an embodiment of the present invention is schematically shown.

[0085] like Figure 3 As shown, the sensor program startup interface according to an embodiment of the present invention provides sensor start and stop functions for lidar, camera, IMU, RTK, ultrasonic radar and millimeter-wave radar, as well as corresponding fault injection program start and stop functions, and users can choose to turn on the functions as needed. After clicking the "Open" button of the specified sensor, the system will start the corresponding sensor driver and start publishing the data topic it collects, and at the same time feed back the sensor drive information to the sensor status bar. After clicking the "Open" button of the specified fault injection program, the system starts the fault injection program of the corresponding sensor, subscribes to the data collected by the sensor, and publishes the corresponding fault program after processing. In addition, when the "Open" button is clicked, it will enter a locked state to prevent the program from being started repeatedly and forming a conflict until the corresponding "Close" button is clicked to unlock it.

[0086] Figure 4 The figure schematically shows a sensor data display interface according to an embodiment of the present invention.

[0087] like Figure 4 As shown, the sensor data display interface according to the embodiment of the present invention includes the sensor data display interface of the image data area, the point cloud data area, the IMU data area, the RTK data area, the ultrasonic radar data area, and the millimeter wave radar data area. Click the "Status" button to enter the data display page, which can display the original sensor data, fault data and the corresponding fault type, and each area has a "Refresh" button.

[0088] The image data area displays the original image on the left, and the image after fault injection on the right. The fault type is displayed in real time above the fault image, including eight types: Gaussian noise, salt and pepper noise, gamma noise, uniform noise, Rayleigh noise, Poisson noise, disconnection and discontinuity. Click "Refresh" during operation to reset the area to the original state and receive data again; the original point cloud data in the point cloud data area is displayed as a white point cloud, and the fault point cloud is displayed as a red point cloud. In addition, the display viewing angle can be controlled by operations such as translation, rotation, zooming in and out with the mouse. The fault type is displayed in real time in the upper left corner of the data display area, including six types: random noise, simulated dust, simulated rain, zero cycle, disconnection and discontinuity; the data types in the IMU data area include quaternions, acceleration and angular velocity of three axes, and can simultaneously display the original data, fault data and the difference between the two, including deviation noise, random noise, zero cycle and disconnection; the data in the RTK data area includes the original longitude and latitude data and the faulty longitude and latitude data, as well as the difference between the two. The fault type is displayed in real time in the upper left corner of the data display area, including Gaussian noise, random noise, disconnection and discontinuity. During operation, click "Refresh" to reset the area to its original state and receive data again; the data displayed in the ultrasonic data area includes the original distance data measured by the four ultrasonic sensors, the fault data, and the difference between the two, in mm, and the minimum measurement distance is 250. The fault type is displayed in real time in the upper left corner of the data display area, including deviation noise, random noise, zero cycle, and disconnection; the millimeter wave data area displays the first six target data detected by the millimeter wave radar, and "NULL" is displayed for the insufficient part. The displayed data includes the target ID, the original XY coordinates, the XY coordinates of the fault, and the corresponding difference. The fault type is displayed in real time in the upper left corner of the data display area, including deviation noise, random noise, zero cycle, and disconnection. During operation, click "Refresh" to reset the area to its original state and receive data again.

[0089] Figure 5 A schematic diagram of a sensor data recording interface according to an embodiment of the present invention is schematically shown.

[0090] like Figure 5 As shown, the sensor data recording interface according to an embodiment of the present invention is used to display and record sensor data. The "Topic List" in the page displays the topic data in the current ROS system, the first column is the topic name, and the second column is the topic format. Click the "Refresh" button to refresh the current data topic in the "Topic List". The "Recording List" displays the topic data to be recorded, and its format is consistent with the "Topic List". The "Output Bar" will display the system information generated during the recording process.

[0091] During the user operation, after selecting a topic in the "topic list", click "Add" to move the selected topic into the "recording list"; after selecting a topic in the "recording list", click "Remove" to move the selected topic out of the "recording list"; after clicking the "Record" button, the system starts recording the topic data in the "recording list"; after clicking the "End" button, the system ends recording and stores the recorded topic data in the system-specified directory. Through these functions, users can flexibly select and manage the sensor data that needs to be recorded for subsequent analysis and application.

[0092] Figure 6 A schematic diagram of a security and reliability evaluation device for an intelligent perception system according to an embodiment of the present invention is schematically shown.

[0093] like Figure 6 As shown, a security reliability evaluation device for an intelligent perception system according to an embodiment of the present invention uses Figure 1 The evaluation system shown includes: a sensor for generating initial fault-free sensor data; a controller for collecting initial fault-free sensor data, injecting fault data into the initial fault-free sensor data to obtain sensor fault data, and transmitting the initial fault-free sensor data and sensor fault data to the intelligent perception system, and receiving and analyzing the test data of the intelligent perception system to obtain test results; a display for displaying an operation interface to achieve human-computer interaction; and a power supply for supplying power to the sensor, the controller and the display, wherein the operation interface includes a sensor program startup interface, a data display interface and a data recording interface. Optionally, the controller includes an industrial computer.

[0094] In some exemplary embodiments, the sensors include: a lidar for collecting point cloud data; an IMU for collecting inertial measurement unit data containing inertial attitude; an RTK for collecting differential positioning signal data containing positioning information; a camera for collecting image data; a millimeter wave radar for collecting millimeter wave radar data containing position information; and an ultrasonic radar for collecting ultrasonic radar data containing position information.

[0095] Figure 7 The structural diagram of an integrated sensor quick-release mechanism 100 according to an embodiment of the present invention is schematically shown.

[0096] like Figure 7As shown, a quick release mechanism 100 is integrated with a laser radar, an IMU, an RTK, a camera, a millimeter-wave radar, and an ultrasonic radar. The housing of the sensor quick release mechanism 100 is provided with a camera hole 101, an IMU hole 102, a millimeter-wave radar hole 103, an ultrasonic radar hole 106, an RTK hole 106, a laser radar hole 108, and an ultrasonic radar harness hole 104 to ensure that the mechanism wiring is neat. Optionally, a quick release mechanism fixing hole 105 is provided at the bottom of the integrated sensor quick release mechanism 100 for fixing the integrated sensor quick release mechanism 100. Optionally, the sensor quick release mechanism 100 uses an aluminum profile frame and an aluminum plate housing, which has strong rigidity.

[0097] Figure 8 A schematic diagram of the shell structure of a device for evaluating the safety and reliability of an intelligent sensing system according to an embodiment of the present invention is shown; Fig. 9 A schematic diagram shows a front view of a housing of a device for evaluating safety and reliability of an intelligent sensing system according to an embodiment of the present invention; and Fig.10 A side view of a housing of a device for evaluating safety and reliability of an intelligent perception system according to an embodiment of the present invention is schematically shown.

[0098] like Figure 8 , Fig. 9 as well as Fig.10 As shown, a housing 200 of a safety and reliability evaluation device for an intelligent perception system according to an embodiment of the present invention includes a three-layer structure from top to bottom, wherein the top of the first layer 210 includes a liftable bracket 211, the liftable bracket 211 is used to place a display, and the liftable bracket 211 is equipped with a folding hinge mechanism to achieve storage of the display; a switch display area 212 is arranged on the side of the first layer 210, the switch display area 212 includes an emergency stop button and a main power switch, the main power switch includes a DC switch and an AC switch, and corresponding DC indicator lights and AC indicator lights are arranged between the switch and the emergency stop button, and the switch display area 212 is used to control the start and stop of the device and judge the operating status of the device; the bottom of the second layer 220 includes a pull-out tray 221, the pull-out tray 221 is used to place the sensor quick-release mechanism 100, and the tray can be pulled out when in use. If a real vehicle test is required, the integrated sensor mechanism can be conveniently completely removed through the quick-release device; and the third layer 230 is used to place the controller, and the third layer 230 is connected to the second layer 220 to facilitate the arrangement of the sensor harness. Optionally, the housing 200 is made of metal and sprayed with silver paint.

[0099] In some exemplary embodiments, the housing 200 also includes: a power bus interface 260 for connecting a power source to power the sensor, controller and display; an exhaust hole 222 for dissipating heat from the sensor quick-release mechanism 100 and the controller; a storage layer 240, disposed between the first layer 210 and the second layer 220, optionally, a slide rail is installed under the storage layer 240, which is opened by pulling and pulling, and is simple and convenient to operate; and a wheel structure 250 for moving the device.

[0100] For example, the sensor quick release mechanism 100 is composed of a laser radar for collecting point cloud information, an IMU for collecting inertial attitude information, an RTK for collecting positioning information, a camera for collecting image information, and a millimeter wave radar and an ultrasonic radar for collecting position information. Sensor operation instructions and fault occurrence instructions are input to the controller (industrial computer) through the mouse and keyboard, and the sensor quick release mechanism 100 transmits data to the controller (industrial computer) through the sensor data line, and finally displays the sensor working information and sensor fault information on the display.

[0101] According to the multimodal intelligent perception system safety and reliability evaluation device designed in accordance with the embodiment of the present invention, multi-sensor data collection can be realized, and various types of fault injection and data visualization can be realized for each sensor. The data collected by the device has covered lidar point cloud, camera image, IMU, millimeter wave radar, ultrasonic radar, and RTK data. The data can be recorded in real time and viewed offline. Real-time injection of 29 types of faults for the above six types of sensor data can be realized. The recorded PCD point cloud file containing time series, png image file, IMU data, RTK longitude and latitude, ultrasonic radar data, and millimeter wave radar data can be generated according to the rosbag data packet. xlsx file.

[0102] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent perception system safety and reliability evaluation system, characterized in that: include: A data acquisition module, used for acquiring initial fault-free sensor data and transmitting the initial fault-free sensor data to an intelligent perception system; A fault injection module is used to inject fault data into the initial fault-free sensor data to obtain sensor fault data, and transmit the sensor fault data to the intelligent perception system; A result analysis module is used to receive and analyze the test data of the intelligent perception system to obtain the test results; as well as The interactive module is used to control the operation of the data acquisition module, the fault injection module and the result analysis module.

2. The evaluation system according to claim 1, characterized in that: The sensor data collected by the data acquisition module includes lidar point cloud data, camera image data, IMU data, millimeter wave radar data, ultrasonic radar data and RTK data.

3. The evaluation system according to claim 2, characterized in that: The fault data injected into the point cloud data includes random noise faults, disconnection faults and intermittent faults; The fault data injected into the image includes Gaussian noise fault, salt and pepper noise fault, gamma noise fault, uniform noise fault, Rayleigh noise fault, Poisson noise fault, rain and fog noise fault, disconnection fault and intermittent noise fault; The fault data injected by the IMU data includes deviation noise fault, random noise fault, disconnection fault and intermittent fault; The fault data injected by the millimeter wave radar data includes random noise fault, disconnection fault and intermittent fault; The fault data injected by the ultrasonic radar data includes random error distance fault, distance attenuation fault, disconnection fault and intermittent fault; as well as The fault data injected into the RTK data includes deviation noise fault, random noise fault, disconnection fault and intermittent fault.

4. The evaluation system according to claim 1, characterized in that: The interactive module includes a sensor program startup interface, a data display interface, and a data recording interface. The sensor program startup interface is used to start the sensor driver program and operate the sensor to publish the sensor data collected by it; The data display interface is used to display the sensor data and fault injection status; and The data recording interface is used to record and collect the normal state data of the sensor and the fault state data of the sensor.

5. The evaluation system according to claim 1, characterized in that: The data acquisition module is capable of saving the initial fault-free sensor data to realize offline saving and viewing of the initial fault-free sensor data; The fault injection module is capable of saving the sensor fault data to realize offline storage and viewing of the sensor fault data; The result analysis module can save the test data and the test results to achieve offline storage and viewing of the test data and the test results.

6. A security reliability evaluation device for an intelligent perception system based on the evaluation system according to any one of claims 1 to 5, characterized in that: include: A sensor for generating initial fault-free sensor data; A controller, used for collecting the initial fault-free sensor data, injecting fault data into the initial fault-free sensor data to obtain sensor fault data, transmitting the initial fault-free sensor data and the sensor fault data to an intelligent perception system, and receiving and analyzing test data of the intelligent perception system to obtain a test result; Display, used to display the operation interface and realize human-computer interaction; as well as a power supply, used to supply power to the sensor, the controller and the display, The operation interface includes a sensor program startup interface, a data display interface and a data recording interface.

7. The device according to claim 6, characterized in that The sensor comprises: LiDAR, used to collect point cloud data; IMU, used to collect inertial measurement unit data including inertial attitude; RTK, used to collect differential positioning signal data containing positioning information A camera, used to collect image data; Millimeter wave radar, used to collect millimeter wave radar data containing position information; and Ultrasonic radar, used to collect ultrasonic radar data containing position information.

8. The device according to claim 7, characterized in that The laser radar, the IMU, the RTK, the camera, the millimeter-wave radar and the ultrasonic radar are integrated with a sensor quick-release mechanism, and the housing of the sensor quick-release mechanism is provided with camera holes, IMU holes, millimeter-wave radar holes, ultrasonic radar holes, RTK holes, laser radar holes and ultrasonic radar wiring harness holes.

9. The device according to claim 8, characterized in that The device also includes a housing, The shell comprises a three-layer structure from top to bottom, wherein: The top of the first layer includes a liftable bracket, the liftable bracket is used to place the display, and the liftable bracket is equipped with a folding hinge mechanism to achieve storage of the display; A switch display area is set on the side of the first layer, and the switch display area includes an emergency stop button and a main power switch. The main power switch includes a DC switch and an AC switch. A corresponding DC indicator light and AC indicator light are set between the switch and the emergency stop button. The switch display area is used to control the start and stop of the device and judge the operating status of the device; The bottom of the second layer includes a pull-out tray, and the pull-out tray is used to place the sensor quick-release mechanism; and The third layer is used to place the controller, and the third layer is connected to the second layer to facilitate the arrangement of sensor harnesses.

10. The device according to claim 9, characterized in that The housing also includes: A power bus interface, used to connect a power source to supply power to the sensor, the controller and the display; Exhaust holes, used for heat dissipation of the sensor quick-release mechanism and the controller; a storage layer, disposed between the first layer and the second layer; and A wheel set structure is used to move the device.

Images