Near-field array three-dimensional SAR and laser radar calibration method
By designing special calibration targets and using a variety of filtering and matching methods, the coordinate system alignment of the near-field array three-dimensional SAR and lidar is achieved, solving the problem of difficulty in finding correspondence in the existing technology, and achieving high-precision calibration effect.
Patent Information
- Application Number
- CN202311372023.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-07-11
- Filing Date
- 2023-10-23
- Publication Date
- 2025-03-11
AI Technical Summary
The prior art is difficult to effectively find the correspondence between the three-dimensional SAR and lidar of the near-field array, making it difficult to assist in the scattering diagnosis of the three-dimensional SAR images of the near-field array.
A method for calibration of near-field array three-dimensional SAR and lidar is proposed. By designing special calibration targets, such as penetrating metal pushpins attached to square reflective patches, and using direct-through filtering, conditional filtering, left-hand coordinate system conversion, statistical filtering, M-estimating sample consistency and coherent point drifting methods, the coordinate system alignment of near-field array three-dimensional SAR and lidar is achieved.
The coordinate system alignment of the three-dimensional SAR and lidar in the near-field array is realized, and the calibration error reaches the order of centimeters, effectively assisting the scattering diagnosis of the three-dimensional SAR images in the near-field array.
Smart Images
Figure CN119596248A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of synthetic aperture radar (SAR) image interpretation, and relates to a near-field array three-dimensional SAR and laser radar calibration method. Background Art
[0002] The near-field array 3D SAR imaging system can effectively measure the target radar cross-section (RCS), and has gradually been applied in the fields of military target stealth performance analysis and evaluation. For details, see the document "Li Liang. Research on Near-field 3D Synthetic Aperture Radar Imaging and Target Extraction Technology [D]. University of Electronic Science and Technology of China.". However, the imaging defects of near-field array 3D SAR have brought troubles to the diagnosis of scattering characteristics of near-field array 3D SAR images. Lidar has excellent 3D environmental perception and measurement capabilities, and can accurately describe the geometric shape, structure and size of the target. Fusion of Lidar image characteristics can construct the 3D space of the detection scene, locate and distinguish targets and interference, enhance the understanding of the detection scene, and effectively assist the scattering diagnosis of near-field array 3D SAR images. For details, see the document "Fu Bo. Multi-sensor calibration of mobile vehicles [D]. Zhejiang University, 2022.".
[0003] Calibration is a technical means to achieve the alignment of multi-source fusion coordinate systems. The key to solving the calibration problem is to find the corresponding relationship between the data obtained from the two sensors. For details, see the document "Han Meng. Research on joint calibration and correlation method of millimeter-wave radar and camera for traffic monitoring [D]. North China University of Technology, 2022.".
[0004] Currently, the existing calibration methods hardly involve the near-field array 3D SAR system, and due to the large differences in the imaging characteristics of the near-field array 3D SAR and LiDAR, it is often difficult to find the corresponding relationship. The existing calibration targets are not suitable for the calibration of the near-field array 3D SAR imaging system and LiDAR, and cannot complete the calibration of the near-field array 3D SAR and LiDAR.
[0005] Therefore, in order to solve the problem of difficulty in finding the correspondence between near-field array 3D SAR and LiDAR in the prior art and to assist in the scattering diagnosis of near-field array 3D SAR images, the present invention proposes a near-field array 3D SAR and LiDAR calibration method. Summary of the invention
[0006] The present invention belongs to the technical field of synthetic aperture radar (SAR) image interpretation. The present invention discloses a near-field array three-dimensional SAR and laser radar calibration method, which is used to solve the problem of difficulty in finding the corresponding relationship between the near-field array three-dimensional SAR and the laser radar in the prior art, and assist in the diagnosis of the scattering characteristics of the near-field array three-dimensional SAR image. The method can be divided into three stages, namely, the design of the calibration target, the extraction of the calibration target, and the alignment of the calibration target. After the near-field array three-dimensional SAR and the laser radar go through the above three stages in sequence, the coordinate system alignment of the near-field array three-dimensional SAR and the laser radar can be finally achieved, and the root mean square error can reach the centimeter level.
[0007] In order to facilitate the description of the present invention, the following terms are first defined:
[0008] Definition 1: Calibration target
[0009] Calibration is a method of aligning multiple source sensors in space by operations such as rotation and translation. In the calibration method based on calibration targets, the calibration target is the carrier of the correspondence. A good calibration target has the following characteristics: (1) easy to detect; (2) high positioning accuracy of the two sensors. In the near-field array 3D SAR, targets with high RCS provide good detection rate. In the lidar, targets with high reflectivity provide good detection rate. For details, see the reference " J, I, I.Extrinsic 6DoF calibration of 3D lidar and radar[C].2017European Conference on Mobile Robots(ECMR).IEEE,2017:1-6."
[0010] Definition 2: Through-filtering method
[0011] The function of the straight-through filter is to filter out points whose values in the specified dimension direction are not within the given range. The implementation principle is as follows: first, specify a dimension and the range under the dimension; second, traverse each point in the point cloud, determine whether the value of the point in the specified dimension is within the range, and delete the points whose values are not within the range; finally, the traversal ends, and the remaining points constitute the filtered point cloud. The straight-through filter is simple and efficient, and is suitable for operations such as eliminating background. For details, please see the website "https: / / blog.csdn.net / luolaihua2018 / article / details / 120215014".
[0012] Definition 3: Conditional filtering method
[0013] Conditional filtering is filtering by setting filtering conditions, similar to piecewise function, to determine whether the point cloud is within the specified range, if not, it is discarded. Commonly used filtering conditions include intensity, reflectivity and range. For details, please refer to the website "https: / / zhuanlan.zhihu.com / p / 351411512".
[0014] Definition 4: Left-handed coordinate system conversion method
[0015] The right index finger is in the positive direction of the Y axis, the middle finger is in the positive direction of the Z axis, and the thumb is in the positive direction of the X axis. This coordinate system is called a right-handed coordinate system. If the right hand is replaced by the left hand, it is a left-handed coordinate system. If one axis of a coordinate system is reversed, the chirality is changed; if two axes are reversed, it is equivalent to rotating 180 degrees around the third axis; if all three axes are reversed, it is the superposition of the previous two, changing the chirality and rotating. For details, see the document "https: / / www.jianshu.com / p / a58d941dec2d".
[0016] Definition 5: Statistical filtering method
[0017] Statistical filtering can be used to remove outliers or gross errors caused by measurement errors. It performs statistical analysis on the neighborhood of each point, and filters out some outliers that do not meet the requirements based on the distance distribution characteristics from the point to all neighboring points. First, traverse the point cloud and calculate the average distance between each point and its nearest k neighboring points; second, calculate the mean μ and standard deviation σ of all average distances, then the distance threshold dmax can be expressed as dmax=μ+α×σ, α is a constant, which can be called the proportional coefficient; finally, traverse the point cloud again and remove points whose average distance to the k neighboring points is greater than dmax. For details, see the website "https: / / blog.csdn.net / xiachong27 / article / details / 90238293".
[0018] Definition 6: M-estimation sample consistency method
[0019] The M-estimator Sample Consensus (MSAC) algorithm uses an iterative optimization method to estimate the parameters of the mathematical model to be solved from a set of observation data containing outliers. It is more adaptable to the interference of outliers than the least squares method. Compared with the random sampling consensus algorithm, the MSAC algorithm improves the loss function, taking the actual distance between the inner point and the mathematical model into account for the loss, and replacing the actual distance between the outer point and the mathematical model with a fixed threshold into the loss, which often results in better mathematical model fitting results. For details, see the literature "Torr PHS, Zisserman A. MLESAC: A new robust estimator with application to estimating image geometry [J]. Computer vision and image understanding, 2000, 78 (1): 138-156.".
[0020] Definition 7: Point-to-plane distance calculation method
[0021] Given a plane Ax+By+Cz+D=0, find the distance d from a point outside the plane (x0, y0, z0) to the plane using the following formula.
[0022]
[0023] For details, please visit the website "https: / / blog.csdn.net / u011483307 / article / details / 51034169".
[0024] Definition 8: Coherence Point Drift Method
[0025] The coherent point drift algorithm transforms point cloud registration into a probability density estimation problem, where the probability of a correct pair of corresponding points is 1 and the probability of an incorrect pair of corresponding points is 0. The algorithm can match point sets with similar shapes and long distances, and does not require a very high initial pose of the point cloud. For details, see the paper "Myronenko A, Song X. Point set registration: Coherent point drift [J]. IEEE transactions on pattern analysis and machine intelligence, 2010, 32 (12): 2262-2275.".
[0026] Definition 9: Root mean square error calculation method
[0027] Root Mean Squared Error (RMSE) is a commonly used indicator to measure the difference between the model's predicted value and the actual observed value. It is used to evaluate the degree of fit of the model on the given data. RMSE is obtained by calculating the mean of the squares of the differences between the predicted value and the actual observed value and taking the square root. The value of RMSE has the same unit as the original observed value. It can measure the average size of the model's prediction error. A smaller RMSE means that the difference between the model's predicted value and the actual observed value is smaller, that is, the model fits better. For details, please see the website "https: / / blog.csdn.net / weixin_43155435 / article / details / 131181780".
[0028] The present invention provides a near-field array three-dimensional SAR and laser radar calibration method, which includes the following steps:
[0029] Step 1: Calibration target design
[0030] According to Definition 1, a metal pushpin with strong scattering and a checkered reflective sticker with high reflectivity are selected as calibration target materials, and the metal pushpin is penetrated through the center of the checkered reflective sticker to obtain the calibration target.
[0031] At this point, the calibration target design is completed.
[0032] Step 2: Calibrate the scene placement
[0033] Prepare the 6 calibration targets designed in step 1.
[0034] Prepare two rectangular polystyrene foams (40cm×40cm×160cm), and place the two foams at the preset positions, staggered 40cm front and back, and adjacent to each other left and right.
[0035] Fix the prepared calibration targets on the vertical surfaces of two rectangular foam blocks. There are three calibration targets on each rectangular foam block, 40 cm apart, distributed in a triangle, and the lowest calibration target is 1.2 m above the ground.
[0036] Place the near-field array three-dimensional SAR and lidar at preset locations.
[0037] At this point, the calibration scene is complete.
[0038] Step 3: Data Collection
[0039] Start the LiDAR and collect LiDAR point cloud data for the calibration scene.
[0040] Start the near-field array 3D SAR system and collect near-field array 3D SAR point cloud data of the same calibration scene.
[0041] At this point, the calibration data collection is completed.
[0042] Step 4: Calibration target extraction
[0043] Step 4.1: Calibrate target SAR point cloud extraction
[0044] The near-field array three-dimensional SAR point cloud data collected in step 3 is processed using the straight-through filtering method in Definition 2 to obtain the SAR point cloud straight-through filtering result, which is recorded as S1.
[0045] The conditional filtering method in Definition 3 is used to set the scattering intensity threshold to process the SAR point cloud direct filtering result S1, and the SAR point cloud conditional filtering result is obtained, which is recorded as S2.
[0046] The left-handed coordinate system conversion method in Definition 4 is used to process the SAR point cloud conditional filtering result S2 to obtain the SAR point cloud of the calibration target, which is denoted as P.
[0047] At this point, the extraction of the calibration target SAR point cloud is completed.
[0048] Step 4.2: Calibrate target lidar point cloud extraction
[0049] The laser radar point cloud data collected in step 3 is processed by the straight-through filtering method in Definition 2 to obtain the laser radar point cloud straight-through filtering result, which is recorded as L1.
[0050] The statistical filtering method in Definition 5 is used to process the direct filtering result L1 of the LiDAR point cloud to obtain the statistical filtering result of the LiDAR point cloud, which is recorded as L2.
[0051] The conditional filtering method in Definition 3 is used to set the reflectivity threshold to process the statistical filtering result L2 of the lidar point cloud, and the conditional filtering result of the lidar point cloud is obtained, which is recorded as L3; at the same time, the filtered lidar point cloud is obtained, which is recorded as L4.
[0052] The M-estimation sample consistency method in Definition 6 is used to process the filtered lidar point cloud L4 to obtain two fitting planes, which are denoted as C1 and C2.
[0053] According to the point-to-plane distance formula in Definition 7, the distance from all points of the calibrated target in the conditional filtering result L3 of the LiDAR point cloud to the corresponding fitting plane C1 or C2 is calculated. The average distance from the calibrated target to the corresponding fitting plane in the conditional filtering result L3 of the LiDAR point cloud is calculated, recorded as D1 and D2, and the calculation formula is as follows:
[0054]
[0055] where d meanrepresents the average distance from the calibration target to the corresponding fitting plane, N represents the total number of points in the calibration target point cloud, and d i Represents the distance from the i-th point in the calibration target point cloud to its corresponding fitting plane.
[0056] The conditional filtering result L3 of the lidar point cloud is translated to the corresponding fitting plane to obtain the lidar point cloud of the calibration target, which is recorded as Q. The translation formula is as follows:
[0057]
[0058] Where P a is the translated point, P b is the point before translation, Represents the normal vector corresponding to the fitted plane.
[0059] At this point, the extraction of the calibrated target lidar point cloud is completed.
[0060] Step 5: Calibration target alignment
[0061] The coherent point drift method in Definition 8 is used to process the calibration target SAR point cloud P and the calibration target lidar point cloud Q to obtain the calibration result, which is recorded as K.
[0062] At this point, the calibration target alignment is complete.
[0063] Step 6: Calibration error evaluation
[0064] The root mean square error calculation method in Definition 9 is used to calculate the root mean square error of the calibration method, which is recorded as RMSE.
[0065] The innovation of the present invention is that a calibration method is proposed to solve the problem of difficulty in finding the corresponding relationship between near-field array 3D SAR and LiDAR. The method designs a special calibration target, that is, a metal thumbtack that penetrates the attached checkered reflective sticker, which can be effectively observed by the near-field array 3D SAR and LiDAR. Then, the position deviation problem existing in the LiDAR imaging result of the calibration target is solved by fitting the plane and translating. Finally, the corresponding point selection problem is converted into a point cloud matching problem, and the near-field array 3D SAR and LiDAR are effectively aligned. Finally, the effectiveness of the proposed method is verified in the actual field experiment of near-field array 3D SAR and solid-state LiDAR in the microwave band. It can effectively align the near-field array 3D SAR and LiDAR, and the calibration error is 0.0113m.
[0066] The advantage of the present invention is that it can realize the coordinate system alignment of the near-field array three-dimensional SAR and the laser radar, overcoming the problem of difficulty in finding the corresponding relationship between the near-field array three-dimensional SAR and the laser radar in the prior art, and the calibration error is at the centimeter level, which can effectively assist the scattering diagnosis of the near-field array three-dimensional SAR image. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] Figure 1 This is a flow chart of the near-field array three-dimensional SAR and lidar calibration method in the present invention.
[0068] Figure 2 Schematic diagram of the near-field array three-dimensional SAR and lidar calibration scene in the present invention.
[0069] Figure 3 These are the calibration results and performance evaluation indicators of the present invention. DETAILED DESCRIPTION
[0070] The following is combined with Figure 1 and attached Figure 2 The present invention is described in further detail.
[0071] Step 1: Calibration target design
[0072] like Figure 1 As shown in Figure 1, the calibration process is shown. According to Definition 1, a metal pushpin with strong scattering and a square reflective sticker with high reflectivity are selected as the calibration target materials, and the metal pushpin is penetrated through the center of the square reflective sticker to obtain the calibration target.
[0073] At this point, the calibration target design is completed.
[0074] Step 2: Calibrate the scene placement
[0075] like Figure 2 As shown, according to Figure 2 The schematic diagram sets the scene.
[0076] Prepare the 6 calibration targets designed in step 1.
[0077] Prepare two rectangular polystyrene foams (40cm×40cm×160cm), and place the two foams at the preset positions, staggered 40cm front and back, and adjacent to each other left and right.
[0078] Fix the prepared calibration targets on the vertical surfaces of two rectangular foam blocks. There are three calibration targets on each rectangular foam block, 40 cm apart, distributed in a triangle, and the lowest calibration target is 1.2 m above the ground.
[0079] Place the near-field array three-dimensional SAR and lidar at preset locations.
[0080] At this point, the calibration scene is complete.
[0081] Step 3: Data Collection
[0082] Start the LiDAR and collect LiDAR point cloud data for the calibration scene.
[0083] Start the near-field array 3D SAR system and collect near-field array 3D SAR point cloud data of the same calibration scene.
[0084] At this point, the calibration data collection is completed.
[0085] Step 4: Calibration target extraction
[0086] Step 4.1: Calibrate target SAR point cloud extraction
[0087] The near-field array three-dimensional SAR point cloud data collected in step 3 is processed using the straight-through filtering method in Definition 2 to obtain the SAR point cloud straight-through filtering result, which is recorded as S1.
[0088] The conditional filtering method in Definition 3 is used to set the scattering intensity threshold to process the SAR point cloud direct filtering result S1, and the SAR point cloud conditional filtering result is obtained, which is recorded as S2.
[0089] The left-handed coordinate system conversion method in Definition 4 is used to process the SAR point cloud conditional filtering result S2 to obtain the SAR point cloud of the calibration target, which is denoted as P.
[0090] At this point, the extraction of the calibration target SAR point cloud is completed.
[0091] Step 4.2: Calibrate target lidar point cloud extraction
[0092] The laser radar point cloud data collected in step 3 is processed by the straight-through filtering method in Definition 2 to obtain the laser radar point cloud straight-through filtering result, which is recorded as L1.
[0093] The statistical filtering method in Definition 5 is used to process the direct filtering result L1 of the LiDAR point cloud to obtain the statistical filtering result of the LiDAR point cloud, which is recorded as L2.
[0094] The conditional filtering method in Definition 3 is used to set the reflectivity threshold to process the statistical filtering result L2 of the lidar point cloud, and the conditional filtering result of the lidar point cloud is obtained, which is recorded as L3; at the same time, the filtered lidar point cloud is obtained, which is recorded as L4.
[0095] The M-estimation sample consistency method in Definition 6 is used to process the filtered lidar point cloud L4 to obtain two fitting planes, which are denoted as C1 and C2.
[0096] According to the point-to-plane distance formula in Definition 7, the distance from all points of the calibrated target in the conditional filtering result L3 of the LiDAR point cloud to the corresponding fitting plane C1 or C2 is calculated. The average distance from the calibrated target to the corresponding fitting plane in the conditional filtering result L3 of the LiDAR point cloud is calculated, recorded as D1 and D2, and the calculation formula is as follows:
[0097]
[0098] where d mean represents the average distance from the calibration target to the corresponding fitting plane, N represents the total number of points in the calibration target point cloud, and d i Represents the distance from the i-th point in the calibration target point cloud to its corresponding fitting plane.
[0099] The conditional filtering result L3 of the lidar point cloud is translated to the corresponding fitting plane to obtain the lidar point cloud of the calibration target, which is recorded as Q. The translation formula is as follows:
[0100]
[0101] Where P a is the translated point, P b is the point before translation, Represents the normal vector corresponding to the fitted plane.
[0102] At this point, the extraction of the calibrated target lidar point cloud is completed.
[0103] Step 5: Calibration target alignment
[0104] The coherent point drift method in Definition 8 is used to process the calibration target SAR point cloud P and the calibration target lidar point cloud Q to obtain the calibration result, which is recorded as K.
[0105] At this point, the calibration target alignment is complete.
[0106] Step 6: Calibration error evaluation
[0107] The root mean square error calculation method in Definition 9 is used to calculate the root mean square error of the calibration method, which is recorded as RMSE.
[0108] like Figure 3 As shown, the present invention realizes the calibration of near-field array three-dimensional SAR and lidar, and can effectively align near-field array three-dimensional SAR and lidar data, with the calibration error at the centimeter level.
Claims
1. A near-field array three-dimensional SAR and lidar calibration method, characterized in that it The following steps are involved: Step 1: Calibration target design The traditional target calibration method is adopted, and metal pushpins with strong scattering and square reflective stickers with high reflectivity are selected as calibration target materials. The metal pushpin is penetrated through the center of the square reflective sticker to obtain the calibration target. At this point, the calibration target design is complete; Step 2: Calibrate the scene placement Prepare the calibration targets designed in step 1. Prepare two rectangular polystyrene foams (40cm×40cm×160cm), and place them at the preset positions, staggered 40cm front and back, and adjacent to each other left and right; Fix the prepared calibration targets on the vertical surfaces of two rectangular foam blocks; there are three calibration targets on each rectangular foam block, 40 cm apart, distributed in a triangle, and the lowest calibration target is 1.2 m above the ground; Place the near-field array 3D SAR and lidar at the preset locations; At this point, the calibration scene is complete; Step 3: Data Collection Start the LiDAR and collect the LiDAR point cloud data of the calibration scene; Start the near-field array 3D SAR system to collect near-field array 3D SAR point cloud data of the same calibration scene; At this point, the calibration data collection is completed; Step 4: Calibration target extraction Step 4.1: Calibrate target SAR point cloud extraction The near-field array three-dimensional SAR point cloud data collected in step 3 is processed using a traditional straight-through filtering method to obtain a SAR point cloud straight-through filtering result, which is recorded as S1; The traditional conditional filtering method is used to set the scattering intensity threshold to process the SAR point cloud direct filtering result S1, and the SAR point cloud conditional filtering result is obtained, which is recorded as S2; The traditional left-handed coordinate system conversion method is used to process the SAR point cloud conditional filtering result S2 to obtain the SAR point cloud of the calibration target, which is recorded as P; At this point, the SAR point cloud extraction of the calibration target is completed; Step 4.2: Calibrate target lidar point cloud extraction The laser radar point cloud data collected in step 3 is processed using the traditional straight-through filtering method to obtain the laser radar point cloud straight-through filtering result, which is recorded as L1; The traditional statistical filtering method is used to process the laser radar point cloud direct filtering result L1, and the laser radar point cloud statistical filtering result is obtained, which is recorded as L2; The traditional conditional filtering method is used to set the reflectivity threshold to process the statistical filtering result L2 of the lidar point cloud, and the conditional filtering result of the lidar point cloud is obtained, which is recorded as L3; at the same time, the filtered lidar point cloud is obtained, which is recorded as L4; The traditional M-estimation sample consistency method is used to process the filtered lidar point cloud L4 and obtain two fitting planes, which are denoted as C1 and C2. The traditional point-to-plane distance calculation method is used to calculate the distance from all points of the calibrated target in the lidar point cloud conditional filtering result L3 to the corresponding fitting plane C1 or C2; The traditional point-to-plane distance calculation method is used to calculate the average distance from the calibrated target to the corresponding fitting plane in the lidar point cloud conditional filtering result L3, recorded as D1 and D2, and the calculation formula is as follows: where d mean represents the average distance from the calibration target to the corresponding fitting plane, N represents the total number of points in the calibration target point cloud, and d i Indicates the distance from the i-th point in the calibration target point cloud to its corresponding fitting plane; The conditional filtering result L3 of the lidar point cloud is translated to the corresponding fitting plane to obtain the lidar point cloud of the calibration target, which is recorded as Q. The translation formula is as follows: Where P a is the translated point, P b is the point before translation, represents the normal vector corresponding to the fitting plane; At this point, the calibration target lidar point cloud extraction is completed; Step 5: Calibration target alignment The traditional coherent point drift method is used to process the calibration target SAR point cloud P and the calibration target lidar point cloud Q to obtain the calibration result, which is recorded as K; At this point, the calibration target alignment is complete; Step 6: Calibration error evaluation The traditional root mean square error calculation method is used to calculate the root mean square error of the calibration method, which is recorded as RMSE.