Target detection method based on infrared camera and three-dimensional laser radar

Abstract

The invention relates to a target anomaly detection method, in particular to a target detection method based on an infrared camera and a three-dimensional laser radar, and solves the technical problem that an existing single detection means is poor in target anomaly detection result effect in a specific scene. The target detection method based on the infrared camera and the three-dimensional laser radar comprises the following steps: 1) building an electronic fence system; 2) respectively detecting the boundary of the calibration plate and each point on the boundary to obtain an infrared image and point cloud data; 3) performing data registration on the infrared image and the point cloud data to obtain a rigid transformation matrix; 4) collecting data of the same scene at the same time, and obtaining target infrared data and target point cloud data; 5) obtaining an infrared detection result; 6) obtaining a point cloud detection result; and 7) performing decision-level fusion on an infrared detection result and a point cloud detection result by using the rigid transformation matrix to obtain a final dynamic target fusion anomaly detection result, and completing all-day target anomaly detection.

Description

Technical Field

[0001] The present invention relates to a target anomaly detection method, and in particular to a target detection method based on an infrared camera and a three-dimensional laser radar. Background Art

[0002] At present, target anomaly detection methods based on various detection methods are emerging in an endless stream, and their detection effects have also achieved good accuracy, leaving less and less room for improvement. However, most single detection methods are only applicable to some specific scenarios, have certain limitations, and cannot maintain a full-time working state, so it is necessary to integrate different detection methods and combine their respective advantages to adapt to diverse application scenarios.

[0003] With the development of security, autonomous driving, drone navigation and other fields, there is an increasing demand for detection systems that can operate reliably at any time and in any weather conditions. All-day anomaly detection requires the detection system to work not only during the day but also at night, and to maintain a certain degree of precision and accuracy, even in harsh weather conditions. However, the existing single detection method is difficult to meet the above requirements for target anomaly detection results in specific scenarios.

[0004] Infrared imaging technology uses infrared energy radiated by objects for imaging. It is a passive detection method that is not affected by visible light and can provide clear images at night or in bad weather conditions. Infrared sensors can capture the thermal characteristics of targets, which is particularly useful for detecting hidden targets. However, due to the low resolution of infrared imaging when facing a background with similar temperature, poor ability to finely depict targets, and lack of depth information, its application range is limited. On the other hand, three-dimensional laser scanning (Lidar) technology constructs a three-dimensional model of an object by emitting laser pulses and measuring the reflection time. It is an active detection method that can provide accurate distance measurement and high-resolution spatial information. Although Lidar may not perform well in bad weather conditions and its effect will be affected, it has an incomparable advantage in providing detailed spatial data.

[0005] In view of the limitations of a single detection method, research on combining infrared imaging with three-dimensional laser scanning technology is extremely important. Summary of the invention

[0006] The purpose of the present invention is to solve the technical problem that the existing single detection means has poor effect on target abnormality detection results in specific scenarios, and to provide a target detection method based on infrared camera and three-dimensional laser radar.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A target detection method based on an infrared camera and a three-dimensional laser radar, which is special in that it includes the following steps:

[0009] 1) Use infrared cameras and 3D laser radar to build an electronic fence system;

[0010] 2) Prepare the calibration plate, use the infrared camera and 3D laser radar in the electronic fence system to detect the boundary of the calibration plate and each point on it, and obtain infrared images and point cloud data;

[0011] 3) Perform data registration on the infrared image and point cloud data to obtain a rigid transformation matrix;

[0012] 4) Use the infrared camera and 3D laser radar in the electronic fence system to collect the same scene data at the same time to obtain target infrared data and target point cloud data;

[0013] 5) Send the target infrared data to the infrared data processing module of the infrared camera for processing to obtain the infrared detection results;

[0014] 6) Send the target point cloud data to the point cloud data processing module of the 3D laser radar for processing to obtain the point cloud detection result;

[0015] 7) The infrared detection results and point cloud detection results are fused at the decision level using the rigid transformation matrix to obtain the final dynamic target fusion anomaly detection results and complete all-day target anomaly detection.

[0016] Further, step 1) is specifically as follows:

[0017] The infrared camera is placed on the three-dimensional laser radar, calibrated and fixed in parallel to complete the construction of the electronic fence system.

[0018] Further, step 3) is specifically as follows:

[0019] 3.1. Calculate the 3D plane of the calibration plate in the camera coordinate system by back-projecting the infrared image and its boundaries Wherein, C represents the infrared camera; i represents the number, i=1,2,3,…N, N represents the number of infrared images; j represents the number of the four boundaries of the calibration plate, j=1,2,3,4;

[0020] 3.2. Use the RANSAC algorithm to detect the plane where the calibration plate is located in the point cloud coordinate system and its boundaries L represents three-dimensional laser radar;

[0021] 3.3. Calculate the initial value of the rotation matrix using the minimization function

[0022]

[0023] In the formula, It's the border The direction vector, It's the border The direction vector, Is a plane The normal vector of Is a plane The normal vector of is the rotation matrix from the point cloud coordinate system to the camera coordinate system, argmin represents the minimization function;

[0024] 3.4. Use the linear least squares method to solve the initial value of the translation matrix

[0025]

[0026] Where H is the coefficient matrix, A ij is the process calculation amount, I is the unit matrix, b is the column matrix, For plane The center of mass, For plane The centroids of the four boundaries of For plane The distance to the origin of the camera coordinate system, For plane Data points on

[0027] 3.5. According to the initial value of the rotation matrix and translation matrix Correct the rotation matrix using the minimization function and translation matrix Get the rigid transformation matrix

[0028]

[0029] Where m is the loop variable; N i Is a plane The number of data points on ; It's the border data points on the loop, k is the loop variable; K ij It's the border The number of data points on .

[0030] Further, step 5) is specifically as follows:

[0031] 5.1. Send the target infrared data to the infrared data processing module of the infrared camera, and record the nth frame image and the n-1th frame image as fn and f n-1 , the grayscale value of the corresponding pixel of the two frames of image is recorded as f n (x,y) and f n-1 (x, y), the difference image D is calculated by the following formula n :

[0032] D n (x,y)=|f n (x,y)-f n-1 (x,y)|

[0033] In the formula, x represents the horizontal coordinate on the image plane, and y represents the vertical coordinate on the image plane;

[0034] 5.2. Difference Image D n (x, y) is morphologically processed to obtain the target image;

[0035] 5.3. Set a first threshold. If the pixel value of the target image is greater than or equal to the first threshold, the pixel is retained; otherwise, the pixel is removed; and all the retained pixels are used as infrared detection results.

[0036] Further, step 6) is specifically as follows:

[0037] 6.1. Send the target point cloud data to the point cloud data processing module of the 3D laser radar, minimize the distance between the target point cloud data, and match the target point cloud data;

[0038] 6.2. Search the matched target point cloud data to obtain the nearest neighbor point;

[0039] 6.3. Set the second threshold. If the nearest neighbor point is greater than or equal to the second threshold, it will be used as the target point; otherwise, no processing will be done;

[0040] 6.4. Extract the coordinates of the target points and perform cluster analysis to obtain different types of targets;

[0041] 6.5. Visualize the target and obtain the point cloud detection results.

[0042] Furthermore, in step 6.1, the distance between target point cloud data is minimized using an ICP algorithm.

[0043] Furthermore, in step 6.2, the search is performed using a KD tree search.

[0044] Furthermore, in step 6.4, the clustering is DBSCAN clustering analysis.

[0045] Beneficial effects of the present invention:

[0046] 1. The present invention discloses a target detection method based on an infrared camera and a three-dimensional laser radar, which integrates an infrared camera and a three-dimensional laser radar for all-day anomaly detection of dynamic targets, thereby solving the problem that a single method has poor dynamic target detection effect in specific scenarios, and realizing efficient recognition and tracking of dynamic targets under various environmental conditions throughout the day.

[0047] 2. The target detection method based on infrared camera and three-dimensional laser radar of the present invention not only integrates the all-weather detection capability of infrared imaging and the high-precision three-dimensional spatial information acquisition capability of three-dimensional laser radar, but also realizes the efficient fusion of the two through algorithms, which significantly improves the robustness and detection accuracy.

[0048] 3. The present invention provides a target detection method based on infrared camera and 3D laser radar. The infrared camera and 3D laser radar can not only obtain complementary information, but also infrared imaging can make up for the shortcomings of 3D laser radar at night or in low visibility; and when detailed spatial data is needed, 3D laser radar can make up for the shortcomings of infrared imaging. In addition, it can enhance adaptability and flexibility, so that it can perform well in a wide range of environments.

[0049] 4. The target detection method based on infrared camera and three-dimensional laser radar of the present invention can provide stable and reliable moving target detection and tracking services under various weather conditions, and is particularly suitable for key areas that require continuous monitoring, such as airports, borders, and the periphery of important facilities.

[0050] 5. The target detection method based on infrared camera and three-dimensional laser radar of the present invention not only improves the accuracy of detection, but also reduces the false alarm rate, ensuring effective monitoring around the clock, and bringing new possibilities and development directions to the fields of security prevention and intelligent traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of an embodiment of a target detection method based on an infrared camera and a three-dimensional laser radar of the present invention;

[0052] Figure 2 It is a flow chart of processing target infrared data by an infrared data processing module in an embodiment of a target detection method based on an infrared camera and a three-dimensional laser radar of the present invention;

[0053] Figure 3 It is a flow chart of processing target point cloud data by a point cloud data processing module in an embodiment of a target detection method based on an infrared camera and a three-dimensional laser radar of the present invention. DETAILED DESCRIPTION

[0054] like Figure 1 As shown, a target detection method based on an infrared camera and a three-dimensional laser radar includes the following steps:

[0055] 1) Use infrared cameras and three-dimensional laser radars to build an electronic fence system;

[0056] Set the infrared camera on the 3D laser radar, calibrate it in parallel and fix it to complete the construction of the electronic fence system;

[0057] 2) Use the infrared camera and 3D laser radar in the electronic fence system to detect the boundary of the calibration plate and each point on it, and obtain infrared images and point cloud data;

[0058] 3) Perform data registration on the infrared image and point cloud data to obtain a rigid transformation matrix;

[0059] 3.1. Calculate the 3D plane of the calibration plate in the camera coordinate system by back-projecting the infrared image and its boundaries Wherein, C represents the infrared camera; i represents the number, i=1,2,3,…N, N represents the number of infrared images; j represents the number of the four boundaries of the calibration plate, j=1,2,3,4;

[0060] 3.2. Use the RANSAC algorithm to detect the plane where the calibration plate is located in the point cloud coordinate system and its boundaries L represents three-dimensional laser radar;

[0061] 3.3. Calculate the initial value of the rotation matrix using the minimization function

[0062]

[0063] In the formula, It's the border The direction vector, It's the border The direction vector, Is a plane The normal vector of Is a plane The normal vector of is the rotation matrix from the point cloud coordinate system to the camera coordinate system, argmin represents the minimization function;

[0064] 3.4. Obtain the initial value of the translation matrix based on the following two constraints:

[0065]

[0066] To avoid deviation, use a flat The centroid and the centroid of each of the four boundaries Calculate the translation matrix

[0067] make Then for each pose we have:

[0068]

[0069] All the postures are accumulated through the above two equations to form a linear equation system:

[0070]

[0071] Use linear least squares to solve for the initial value of the translation matrix Find Make Minimize. For this type of problem, there is a closed-form solution:

[0072]

[0073] Where I is the identity matrix; It's the border The data points on, k is the loop variable; Is a plane Data points on Is a plane Data points on Is a plane The distance to the origin of the camera coordinate system; is the translation matrix from the point cloud coordinate system to the camera coordinate system; T is the transposed matrix; H is the coefficient matrix; b is the column matrix; where,

[0074]

[0075] 3.5. According to the initial value and initial value Correct the rotation matrix using the minimization function and translation matrix Get the rigid transformation matrix

[0076]

[0077] Where m is the loop variable, m = 1, 2, ..., N i ; N i Is a plane The number of data points on K ij It's the border The number of data points on ;

[0078] 4) Use the infrared camera and 3D laser radar in the electronic fence system to collect the same scene data at the same time to obtain target infrared data and target point cloud data;

[0079] 5) If Figure 2 As shown, the target infrared data is sent to the infrared data processing module of the infrared camera for processing to obtain the infrared detection result;

[0080] 5.1. Send the target infrared data to the infrared data processing module of the infrared camera, and record the nth frame image and the n-1th frame image as f n and f n-1 , the grayscale value of the corresponding pixel of the two frames of image is recorded as f n (x,y) and f n-1 (x, y), the difference image D is calculated by the following formula n :

[0081] D n (x,y)=|f n (x,y)-f n-1 (x,y)|

[0082] In the formula, x represents the horizontal coordinate on the image plane, and y represents the vertical coordinate on the image plane;

[0083] 5.2. Difference Image D n (x, y) is morphologically processed to obtain the target image;

[0084] 5.3. Perform threshold processing on the target image;

[0085] A first threshold is set. If the pixel value of the target image is greater than or equal to the first threshold, the pixel is retained; otherwise, the pixel is removed; and all the retained pixels are used as infrared detection results;

[0086] 6) If Figure 3 As shown, the target point cloud data is sent to the point cloud data processing module of the 3D laser radar for processing to obtain the point cloud detection result;

[0087] 6.1. Send the target point cloud data to the point cloud data processing module of the 3D laser radar, use the ICP algorithm to minimize the distance between the target point cloud data, and match the target point cloud data;

[0088] 6.2. Use KD (K-Demensional, KD) tree to search the matched target point cloud data to obtain the nearest neighbor point;

[0089] 6.3. Set the second threshold. If the nearest neighbor point is greater than or equal to the second threshold, it will be used as the target point; otherwise, no processing will be done;

[0090] 6.4. Extract the coordinates of the target points and perform DBSCAN (Density Based Spatial Clustering of Applications with Noise) clustering analysis to obtain different types of targets;

[0091] 6.5. Visualize the target and obtain the point cloud detection results;

[0092] 7) The infrared detection results and point cloud detection results are fused at the decision level using the rigid transformation matrix to obtain the final dynamic target fusion anomaly detection results and complete all-day target anomaly detection.

Claims

1. A target detection method based on infrared camera and three-dimensional laser radar, characterized in that: The following steps are involved: 1) Use infrared cameras and 3D laser radar to build an electronic fence system; 2) Prepare the calibration plate, use the infrared camera and 3D laser radar in the electronic fence system to detect the boundary of the calibration plate and each point on it, and obtain infrared images and point cloud data; 3) Perform data registration on the infrared image and point cloud data to obtain a rigid transformation matrix; 4) Use the infrared camera and 3D laser radar in the electronic fence system to collect the same scene data at the same time to obtain target infrared data and target point cloud data; 5) Send the target infrared data to the infrared data processing module of the infrared camera for processing to obtain the infrared detection results; 6) Send the target point cloud data to the point cloud data processing module of the 3D laser radar for processing to obtain the point cloud detection result; 7) The infrared detection results and point cloud detection results are fused at the decision level using the rigid transformation matrix to obtain the final dynamic target fusion anomaly detection results and complete all-day target anomaly detection.

2. The target detection method based on infrared camera and three-dimensional laser radar according to claim 1, characterized in that: Step 1) is specifically as follows: The infrared camera is placed on the three-dimensional laser radar, calibrated and fixed in parallel to complete the construction of the electronic fence system.

3. The target detection method based on infrared camera and three-dimensional laser radar according to claim 2, characterized in that: Step 3) is specifically: 3.

1. Calculate the 3D plane of the calibration plate in the camera coordinate system by back-projecting the infrared image and its boundaries Wherein, C represents the infrared camera; i represents the number, i=1,2,3,…N, N represents the number of infrared images; j represents the number of the four boundaries of the calibration plate, j=1,2,3,4; 3.

2. Use the RANSAC algorithm to detect the plane where the calibration plate is located in the point cloud coordinate system and its boundaries L represents three-dimensional laser radar; 3.

3. Calculate the initial value of the rotation matrix using the minimization function In the formula, It's the border The direction vector of It's the border The direction vector of Is a plane The normal vector of Is a plane The normal vector of is the rotation matrix from the point cloud coordinate system to the camera coordinate system, argmin represents the minimization function; 3.

4. Use the linear least squares method to solve the initial value of the translation matrix Where H is the coefficient matrix, A ij is the process calculation amount, I is the unit matrix, b is the column matrix, For plane The center of mass, For plane The centroids of the four boundaries of For plane The distance to the origin of the camera coordinate system, For plane Data points on 3.

5. According to the initial value of the rotation matrix and translation matrix Correct the rotation matrix using the minimization function and translation matrix Get the rigid transformation matrix Where m is the loop variable; N i Is a plane The number of data points on ; It's the border data points on the loop, k is the loop variable; K ij It's the border The number of data points on .

4. The target detection method based on infrared camera and three-dimensional laser radar according to claim 3 is characterized in that: Step 5) is specifically as follows: 5.

1. Send the target infrared data to the infrared data processing module of the infrared camera, and record the nth frame image and the n-1th frame image as f n and f n-1 , the grayscale value of the corresponding pixel of the two frames of image is recorded as f n (x,y) and f n-1 (x, y), the difference image D is calculated by the following formula n : D n (x,y)=|f n (x,y)-f n-1 (x,y)| In the formula, x represents the horizontal coordinate on the image plane, and y represents the vertical coordinate on the image plane; 5.

2. Difference Image D n (x, y) is morphologically processed to obtain the target image; 5.

3. Set a first threshold. If the pixel value of the target image is greater than or equal to the first threshold, the pixel is retained; otherwise, the pixel is removed; and all the retained pixels are used as infrared detection results.

5. The target detection method based on infrared camera and three-dimensional laser radar according to claim 4, characterized in that: Step 6) is specifically as follows: 6.

1. Send the target point cloud data to the point cloud data processing module of the 3D laser radar, minimize the distance between the target point cloud data, and match the target point cloud data; 6.

2. Search the matched target point cloud data to obtain the nearest neighbor point; 6.

3. Set the second threshold. If the nearest neighbor point is greater than or equal to the second threshold, it will be used as the target point. Otherwise, no processing is done; 6.

4. Extract the coordinates of the target points and perform cluster analysis to obtain different types of targets; 6.

5. Visualize the target and obtain the point cloud detection results.

6. The target detection method based on infrared camera and three-dimensional laser radar according to claim 5, characterized in that: In step 6.1, the distance between target point cloud data is minimized using the ICP algorithm.

7. The target detection method based on infrared camera and three-dimensional laser radar according to claim 6, characterized in that: In step 6.2, the search is performed using a KD tree search.

8. The target detection method based on infrared camera and three-dimensional laser radar according to claim 7, characterized in that: In step 6.4, the clustering is DBSCAN clustering analysis.

Images