Intelligent vehicle fault-tolerant detection method based on multi-sensor fusion

Abstract

This invention provides a fault-tolerant detection method for intelligent vehicles based on multi-sensor fusion, belonging to the field of autonomous driving technology. The method includes the following steps: real-time acquisition of obstacle position information through multiple sensors; fusion of the obstacle position information acquired by the sensors using a fusion algorithm; determination of sensor malfunction by calculating position deviation; and finally, identification of sensor malfunction and output of obstacle position information. This invention not only achieves more accurate estimation of surrounding obstacle information through multi-sensor information fusion, but also corrects obstacle position information in the event of sensor malfunction, demonstrating good fault-tolerant detection capabilities.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically to a fault-tolerant detection method for intelligent vehicles based on multi-sensor fusion. Background Technology

[0002] With the development of technology, autonomous driving technology for vehicles is gradually emerging. In autonomous driving technology, environmental detection is a prerequisite for decision-making and control, and the foundation for the entire autonomous driving system. Real-world traffic scenarios are complex, and driverless vehicles face numerous complex and uncertain factors when driving autonomously in such environments. The complexity mainly lies in the fact that real-world environments often contain a wealth of driving scenario elements. The uncertainty mainly lies in the fact that sensors have specific detection ranges, making it difficult to guarantee that all environmental information can be obtained at any time and from any angle. Furthermore, during environmental detection, noise inherent in the sensors themselves, errors caused by sensor calibration errors, etc., mean that data from a single sensor cannot be used as the result of environmental detection. Therefore, a multi-sensor fusion approach is used for environmental detection. However, if the hardware of the environmental detection system itself malfunctions, the intelligent vehicle will also be unable to obtain reliable environmental information. The uncertainty of the environmental detection system poses challenges to the accuracy of the intelligent vehicle's situation assessment system's evaluation results and the rationality of the driving decision-making system's decision-making scheme, thus affecting the safe driving of the vehicle.

[0003] Chinese invention patent document CN111797701A discloses a "Road Obstacle Detection Method and System for Vehicle Multi-Sensor Fusion System". This method acquires or calculates four lane lines on the left and right sides of the vehicle according to a set lane width; divides the vehicle's own lane and the two adjacent lanes based on the four lane lines; determines whether the adjacent lanes are located within the drivable road area; detects only targets within the observation area that pose a risk of entering the region of interest; and detects all targets within the region of interest. However, this method has shortcomings:

[0004] 1. This method only collects obstacle information in a specified area, and cannot respond in a timely manner if an emergency occurs during actual driving;

[0005] 2. This method does not consider the error information transmitted to the fusion system when the sensor fails, which will cause a large obstacle position prediction error.

[0006] 3. This algorithm does not consider the significant driving risks that may arise when sensors malfunction, fail to acquire four lane line information, or transmit incorrect lane line information.

[0007] Chinese invention patent document CN115933646A discloses a "Method and System for Obstacle Avoidance and Bypass Based on Multi-Sensor Fusion for Obstacle Perception." This method first performs preliminary fusion of visual data, lidar data, and ultrasonic data. Then, based on DS evidence theory, it performs global fusion of the preliminary fusion results of visual data, lidar data, and ultrasonic data to obtain the final obstacle information around the robot. Finally, it performs obstacle avoidance control based on the obtained obstacle information. This method has shortcomings:

[0008] 1. The DS evidence theory is used to fuse sensor data. However, the DS evidence theory is highly sensitive to outliers. If sensor faults exist, these fault values ​​may adversely affect the fusion results through their contribution, leading to inaccuracies and significant obstacle information errors. Summary of the Invention

[0009] The technical problem to be solved by this invention is to overcome the shortcomings of the prior art. This invention provides a fault-tolerant detection method for intelligent vehicles based on multi-sensor fusion. This method can not only fuse multi-source information based on the surrounding obstacle information collected by multiple sensors using the extended Kalman algorithm to achieve more accurate estimation of surrounding obstacle information, but also can correct sensor information in fault conditions, thus exhibiting good fault-tolerant detection capabilities.

[0010] The objective of this invention is achieved as follows: This invention provides a fault-tolerant detection method for intelligent vehicles based on multi-sensor fusion. The fault-tolerant detection system involved in this method includes a multi-sensor detection module, a fusion module, a position deviation calculation module, a fault identification module, and a reset module. The multi-sensor detection module includes n sensors, which are integrated and installed at the same position on the front bumper of the intelligent vehicle. Each of the n sensors is unidirectionally electrically connected to the fusion module. The fusion module is sequentially unidirectionally electrically connected to the position deviation calculation module, the fault identification module, and the reset module. The reset module is unidirectionally electrically connected to each of the n sensors. Any one of the n sensors is denoted as sensor Y. i , i = 1, 2, ..., n;

[0011] A pre-set interval detection time f is established. When the interval detection time f expires, a fault-tolerant detection operation is initiated. The steps of one fault-tolerant detection operation are as follows:

[0012] Step 1: Record the current time as time k; establish a plane coordinate system with the center of mass of the intelligent vehicle in motion as the origin. The positive direction of the vertical axis of this plane coordinate system is the direction in which the front of the intelligent vehicle is pointing, and the positive direction of the horizontal axis is rotated 90° clockwise.

[0013] At time k, the positions of obstacles around the vehicle are detected by n sensors respectively; assuming that for the same obstacle, the n sensors detect n position coordinates, these n position coordinates are marked as the current obstacle position coordinates (x, k). i y i );

[0014] The multi-sensor detection module will detect the coordinates of n current obstacles (x, y, y). i y i Transmitted to the fusion module;

[0015] Step 2: The fusion module uses a preset extended Kalman filter algorithm to process the coordinates (x, y, y) of the n current obstacle positions. i y i The information is fused to obtain n current obstacle position fusion information Γ i (x ki y ki ), where (x ki y ki () represents the position coordinates of the current obstacle location fusion information;

[0016] The specific fusion method is as follows: sequentially integrate sensor Y... i The detected current obstacle position coordinates (x) i y i ) and sensor Y i+1 The detected current obstacle position coordinates (x) i+1 y i+1 The current obstacle position fusion information is obtained by fusing the data. i (x ki y ki ), i≤n-1; sensor Y n The obstacle position coordinates (x) obtained at time k are detected n y n The obstacle position fusion information Γ at time k is obtained by fusing the obstacle position coordinates (x1, y1) detected by sensor Y1 at time k. n (x kn y kn );

[0017] The fusion module will fuse the n current obstacle positions into Γ. i (x ki y ki Send to the position deviation calculation module:

[0018] Step 3: The position deviation calculation module calculates the fusion position deviation D of n obstacles according to the preset program. i Then the calculation results are sent to the fault identification module;

[0019] The obstacle fusion position deviation D i The Euclidean distance is calculated as follows:

[0020]

[0021] Where, x (k-f)i y (k-f)i The coordinates of the obstacle position fusion information at time kf;

[0022] Step 4: The fault identification module identifies sensor faults according to a preset program; if a sensor fault is found, a reset signal is sent to the faulty sensor through the reset module.

[0023] The sensor fault identification process is as follows:

[0024] Given a threshold M, M = v 2 f, where v is the current speed of the intelligent vehicle;

[0025] Obstacle fusion position deviation D i Compare with threshold M:

[0026] If the fusion position deviation of n obstacles is D i All values ​​are less than the threshold M, indicating no sensor malfunction;

[0027] If D j ≥M, and D j-1 ≥M, j=2,3,...,n, sensor Υ i A malfunction occurred;

[0028] If D1≥M, and D n ≥M, sensor Y1 has malfunctioned.

[0029] Preferably, the fusion module in step 2 uses a preset extended Kalman filter algorithm to process the coordinates (x, y, y) of the n current obstacle positions. i y i The information is fused to obtain n current obstacle position fusion information Γ i (x ki y ki The process is as follows:

[0030] Step 2.1, establish the prediction equation for the extended Kalman filter algorithm as follows:

[0031] P′=FPF T +Q

[0032] Where F is the state transition matrix, P is the state covariance matrix, Q is the process noise, and P' is the estimated value of the state covariance matrix P at time k.

[0033] Step 2.2, establish the update equation for the extended Kalman filter algorithm:

[0034]

[0035] Where Z(k) is the value passed through sensor Y at time k. i+1 The vector of the current obstacle position coordinates obtained from the detection. X(k) is the vector of the current obstacle position coordinates detected by sensor Yi at time k. Y(k) is the position vector deviation, H is the measurement matrix, R is the measurement Gaussian noise matrix, K is the Kalman gain; I is the update transformation matrix, and P″ is the estimated value of the state covariance matrix P at time k+f, which is used for calculation at time k+f and forms the closed loop of the extended Kalman filter algorithm.

[0036] Step 2.3, Current obstacle position fusion information Γ i (x ki y ki The formula for calculating ) is:

[0037] Γ i (x ki y ki )=X(k)+KY(k).

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] 1. The method of the present invention can still accurately measure obstacle information in complex environments through the complementarity of multiple sensors.

[0040] 2. The method of the present invention can still accurately measure obstacle information when the sensor malfunctions, demonstrating strong robustness.

[0041] 3. In the method of the present invention, the error threshold changes with the speed of the vehicle, which can improve the accuracy of the threshold setting. Attached Figure Description

[0042] Figure 1 This is a block diagram of the intelligent vehicle fault-tolerant detection system in an embodiment of the present invention. Detailed Implementation

[0043] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0044] Figure 1This is a block diagram of an intelligent vehicle fault-tolerant detection system according to an embodiment of the present invention. As shown in the diagram, the intelligent vehicle fault-tolerant detection system of the present invention includes a multi-sensor detection module, a fusion module, a position deviation calculation module, a fault identification module, and a reset module. The multi-sensor detection module includes n sensors, which are integrated and installed at the same position on the front bumper of the intelligent vehicle. Each of the n sensors is unidirectionally electrically connected to the fusion module. The fusion module is sequentially unidirectionally electrically connected to the position deviation calculation module, the fault identification module, and the reset module. The reset module is unidirectionally electrically connected to each of the n sensors. Any one of the n sensors is denoted as sensor Y. i , i = 1, 2, ..., n.

[0045] In this embodiment, n=3, sensor Y1 is a millimeter-wave radar, sensor Y2 is a lidar, and sensor Y3 is a camera. The millimeter-wave radar, lidar, and camera are all installed in the middle of the front bumper of the intelligent vehicle.

[0046] The detection method of this invention pre-sets a detection interval f. When the detection interval f expires, a fault-tolerant detection operation is initiated. The steps of a fault-tolerant detection operation are as follows:

[0047] Step 1: Record the current time as time k; establish a plane coordinate system with the center of mass of the intelligent vehicle in motion as the origin. The positive direction of the vertical axis of this plane coordinate system is the direction in which the front of the intelligent vehicle is pointing, and the positive direction of the horizontal axis is rotated 90° clockwise.

[0048] At time k, the positions of obstacles around the vehicle are detected by n sensors respectively; assuming that for the same obstacle, the n sensors detect n position coordinates, these n position coordinates are marked as the current obstacle position coordinates (x, k). i y i ).

[0049] The multi-sensor detection module will detect the coordinates of n current obstacles (x, y, y). i y i It is transmitted to the fusion module.

[0050] Step 2: The fusion module uses a preset extended Kalman filter algorithm to process the coordinates (x, y, y) of the n current obstacle positions. i y i The information is fused to obtain n current obstacle position fusion information Γ i (x ki y ki ), where (x ki y ki ) represents the position coordinates of the current obstacle location fusion information.

[0051] The specific fusion method is as follows: sequentially integrate sensor Y... i The detected current obstacle position coordinates (x) i y i ) and sensor Y i+1 The detected current obstacle position coordinates (x) i+1 y i+1 The current obstacle position fusion information is obtained by fusing the data. i (x ki y ki ), i≤n-1; sensor Y n The obstacle position coordinates (x) obtained at time k are detected n y n The obstacle position fusion information Γ at time k is obtained by fusing the obstacle position coordinates (x1, y1) detected by sensor Y1 at time k. n (x kn y kn ).

[0052] The fusion module will fuse the n current obstacle positions into Γ. i (x ki y ki It is sent to the position deviation calculation module.

[0053] In this embodiment, the current obstacle position coordinates of the millimeter-wave radar and the lidar are fused to obtain the current obstacle position fusion information Γ1(x). k1 y k1 The obstacle position fusion information Γ2(x) is obtained by fusing the current obstacle position coordinates of the LiDAR and camera. k2 y k2 The obstacle location fusion information Γ3(x) is obtained by fusing the current obstacle location coordinates of the camera and millimeter-wave radar. k3 y k3 ).

[0054] Step 3: The position deviation calculation module calculates the fusion position deviation D of n obstacles according to the preset program. i Then the calculation results are sent to the fault identification module;

[0055] The obstacle fusion position deviation D i The Euclidean distance is calculated as follows:

[0056]

[0057] Where, x (k-f)i y (k-f)i The coordinates are the position coordinates of the obstacle position fusion information at time kf.

[0058] In this embodiment, three obstacle fusion position deviations are calculated and denoted as D1, D2 and D3, respectively.

[0059] Step 4: The fault identification module identifies sensor faults according to a preset program; if a sensor fault is found, a reset signal is sent to the faulty sensor through the reset module.

[0060] The sensor fault identification process is as follows:

[0061] Given a threshold M, M = v 2 f, where v is the current speed of the intelligent vehicle.

[0062] Obstacle fusion position deviation D i Compare with threshold M:

[0063] If the fusion position deviation of n obstacles is D i All values ​​are less than the threshold M, indicating no sensor malfunction;

[0064] If D j ≥M, and D j-1 ≥M, j=2,3,...,n, sensor Υ i A malfunction occurred;

[0065] If D1≥M, and D n ≥M, sensor γ1 has malfunctioned.

[0066] In this embodiment, only three sensors are included, and the identification is as follows:

[0067] D1, D2, and D3 are all less than the threshold M, indicating that no sensor has malfunctioned.

[0068] If D1≥M and D3≥M, the millimeter-wave radar malfunctions.

[0069] If D1≥M and D2≥M, the lidar has malfunctioned;

[0070] If D2≥M and D3≥M, the camera is malfunctioning.

[0071] In this embodiment, the fusion module in step 2 uses a preset extended Kalman filter algorithm to process the coordinates (x, y, y) of the n current obstacle positions. i y i The information is fused to obtain n current obstacle position fusion information F. i (x ki y ki The process is as follows:

[0072] Step 2.1, establish the prediction equation for the extended Kalman filter algorithm as follows:

[0073] P′=FPF T +Q

[0074] Where F is the state transition matrix, P is the state covariance matrix, Q is the process noise, and P' is the estimated value of the state covariance matrix P at time k.

[0075] Step 2.2, establish the update equation for the extended Kalman filter algorithm:

[0076]

[0077] Where Z(k) is the value passed through sensor Y at time k. i+1 The vector of the current obstacle position coordinates obtained from the detection. X(k) is the value obtained by sensor Y at time k. i The vector of the current obstacle position coordinates obtained from the detection. Y(k) is the position vector deviation, H is the measurement matrix, R is the measurement Gaussian noise matrix, K is the Kalman gain, I is the update transformation matrix, and P″ is the estimated value of the state covariance matrix P at time k+f, which is used for calculation at time k+f and forms the closed loop of the extended Kalman filter algorithm.

[0078] Step 2.3, Current obstacle position fusion information Γ i (x ki y ki The formula for calculating ) is:

[0079] Γ i (x ki y ki )=X(k)+KY(k).

[0080] In this embodiment, f = 50 seconds, K = 29.

Claims

1. A fault-tolerant detection method for intelligent vehicles based on multi-sensor fusion, characterized in that, The intelligent vehicle fault-tolerant detection system involved in this detection method includes a multi-sensor detection module, a fusion module, a position deviation calculation module, a fault identification module, and a reset module. The multi-sensor detection module includes n sensors, which are integrated and installed at the same position on the front side of the intelligent vehicle. Each of the n sensors is unidirectionally electrically connected to the fusion module. The fusion module is then sequentially unidirectionally electrically connected to the position deviation calculation module, the fault identification module, and the reset module. The reset module is unidirectionally electrically connected to each of the n sensors. Any one of the n sensors is denoted as sensor Y. i , i = 1, 2, ..., n; A pre-set interval detection time f is established. When the interval detection time f expires, a fault-tolerant detection operation is initiated. The steps of one fault-tolerant detection operation are as follows: Step 1: Record the current time as time k; establish a plane coordinate system with the center of mass of the intelligent vehicle in motion as the origin. The positive direction of the vertical axis of this plane coordinate system is the direction in which the front of the intelligent vehicle is pointing, and the positive direction of the horizontal axis is rotated 90° clockwise. At time k, the positions of obstacles around the vehicle are detected by n sensors respectively; assuming that for the same obstacle, the n sensors detect n position coordinates, these n position coordinates are marked as the current obstacle position coordinates (x, k). i y i ); The multi-sensor detection module will detect the coordinates of n current obstacles (x, y, y). i y i Transmitted to the fusion module; Step 2: The fusion module uses a preset extended Kalman filter algorithm to process the coordinates (x, y, y) of the n current obstacle positions. i y i The information is fused to obtain n current obstacle position fusion information Γ i (x ki y ki ), where (x ki y ki () represents the position coordinates of the current obstacle location fusion information; The specific fusion method is as follows: sequentially integrate sensor Y... i The detected current obstacle position coordinates (x) i y i ) and sensor Y i+1 The detected current obstacle position coordinates (x) i+1 y i+1 The current obstacle position fusion information is obtained by fusing the data. i (x ki y ki ), i≤n-1; sensor Y n The obstacle position coordinates (x) obtained at time k are detected n y n The obstacle position fusion information Γ at time k is obtained by fusing the obstacle position coordinates (x1, y1) detected by sensor Y1 at time k. n (x kn y kn ); The fusion module will fuse the n current obstacle positions into Γ. i (x ki y ki Send to the position deviation calculation module: Step 3: The position deviation calculation module calculates the fusion position deviation D of n obstacles according to the preset program. i Then the calculation results are sent to the fault identification module; The obstacle fusion position deviation D i The Euclidean distance is calculated as follows: Where, x (k-f)i y (k-f)i The coordinates of the obstacle position fusion information at time kf; Step 4: The fault identification module identifies sensor faults according to a preset program; if a sensor fault is found, a reset signal is sent to the faulty sensor through the reset module. The sensor fault identification process is as follows: Given a threshold M, M = v 2 f, where v is the current speed of the intelligent vehicle; Obstacle fusion position deviation D i Compare with threshold M: If the fusion position deviation of n obstacles is D i All values ​​are less than the threshold M, indicating no sensor malfunction; If D j ≥M, and D j-1 ≥M, j=2,3,...,n, sensor Υ i A malfunction occurred; If D1≥M, and D n ≥M, sensor Y1 has malfunctioned.

2. The intelligent vehicle fault-tolerant detection method based on multi-sensor fusion according to claim 1, characterized in that, Step 2 describes the fusion module that, based on a preset extended Kalman filter algorithm, processes the coordinates (x, y, y) of the n current obstacle positions. i y i The information is fused to obtain n current obstacle position fusion information Γ i (x ki y ki The process is as follows: Step 2.1, establish the prediction equation for the extended Kalman filter algorithm as follows: P′=FPF T +Q Where F is the state transition matrix, P is the state covariance matrix, Q is the process noise, and P' is the estimated value of the state covariance matrix P at time k. Step 2.2, establish the update equation for the extended Kalman filter algorithm: Where Z(k) is the value passed through sensor Y at time k. i+1 The vector of the current obstacle position coordinates obtained from the detection. X(k) is the vector of the current obstacle position coordinates detected by sensor Yi at time k. Y(k) is the position vector deviation, H is the measurement matrix, R is the measurement Gaussian noise matrix, K is the Kalman gain; I is the update transformation matrix, and P″ is the estimated value of the state covariance matrix P at time k+f, which is used for calculation at time k+f and forms the closed loop of the extended Kalman filter algorithm. Step 2.3, Current obstacle position fusion information Γ i (x ki y ki The formula for calculating ) is: Γ i (x ki ,y ki )=X(k)+KY(k)。

Images