Camera / laser radar calibration method for galloping monitoring of power transmission conductor

Abstract

The invention provides a camera / laser radar calibration method for galloping monitoring of a transmission conductor, and belongs to the technical field of crossing of multi-sensor fusion sensing and power system monitoring, and the method comprises the steps: reconstructing a dense point cloud through binocular stereo matching, and automatically segmenting a multi-plane structure existing in a near field and a far field; and normal vector clustering grouping is carried out on the laser point cloud to obtain near and far field multi-plane fragments, then cross-correlation of the image reconstruction multi-plane point cloud and the laser radar multi-plane point cloud is realized based on an SPFH descriptor constructed by local voxels, and finally, a camera / laser radar calibration algorithm based on active robust of a multi-plane structure is combined to estimate high-precision external parameters. According to the method, no calibration target is needed, a multi-distance and multi-angle plane structure in a scene is fully utilized, and full-environment and full-automatic calibration is achieved; and meanwhile, through multi-plane cross-correlation and active anti-noise optimization, the registration error is remarkably reduced, the external parameter precision is improved, and the robustness of image distortion and shielding noise in a complex environment is enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of intersection of multi-sensor fusion perception and power system monitoring, and specifically relates to a camera / lidar calibration method for transmission line galloping monitoring, which is applied to the technical field of intersection of multi-sensor fusion perception and power system monitoring for wire galloping monitoring. Background Art

[0002] Transmission lines are exposed to both natural and human factors in an open environment for a long time. Frequent extreme weather events cause lines to swing violently in strong winds, bounce after ice buildup, and even significantly increase the risk of damage from external forces, posing a significant threat to the safe and stable operation of the power grid. Real-time and accurate monitoring of conductor windage is a key measure to prevent transmission line crossover accidents. Currently, traditional power line vibration monitoring relies on fixed-angle visual cameras: camera-based methods can only qualitatively identify whether the conductor is vibrating, but cannot quantify the amplitude. While radar point cloud-based monitoring offers excellent ranging accuracy, its point cloud lacks intuitiveness, making it difficult to quickly interpret. Therefore, a new power line dance monitoring solution with both high-precision measurement and intuitive visualization is urgently needed to improve the real-time and accuracy of monitoring and help on-site personnel quantify the wind deviation amplitude. The power line dance monitoring device that integrates binocular cameras and lidar has the advantages of both cameras and lidar, and is the optimal solution for power line dance monitoring tasks at this stage. However, efficiently and accurately calibrating the binocular cameras and lidar sensors on the monitoring device is a challenging task. The existing calibration method relies on high-precision calibration plates, which is unrealistic in daily work. Summary of the Invention

[0003] In order to solve the problem that the existing methods of camera and lidar extrinsic parameter calibration rely on high-precision calibration plates and are difficult to adapt to daily construction sites, the present invention proposes a camera / lidar calibration method for transmission line dancing monitoring. The method reconstructs a dense point cloud through binocular stereo matching and automatically segments the multi-planar structures in the near and far fields. The laser point cloud is clustered and grouped by normal vectors to obtain near and far field multi-planar fragments. The SPFH descriptor constructed based on local voxels is then used to realize the correlation between the image reconstructed multi-planar point cloud and the lidar multi-planar point cloud. Finally, a camera / lidar calibration algorithm based on active anti-error of multi-planar structure is combined to estimate high-precision extrinsic parameters. The method makes full use of the on-site planar environment characteristics, and through robust plane extraction and geometric constraint fusion, realizes an automated calibration process without the need for a high-precision calibration plate, greatly simplifies the equipment deployment and calibration steps, and is suitable for conventional transmission line dancing monitoring tasks. It is used to provide high-precision extrinsic parameters for the transmission line dancing monitoring device that integrates lidar and binocular camera, thereby realizing convenient operation.

[0004] In order to achieve the above object, the technical scheme adopted by the present application is as follows: a camera / laser radar calibration method for transmission line galloping monitoring, comprising the following steps:

[0005] S1: based on the binocular camera internal parameter, the left eye image and the right eye image obtained are subjected to dense stereo reconstruction, the reconstructed dense three-dimensional point cloud is classified based on the RANSAC algorithm, the plane point cloud is obtained and the normal vector of the multi-cluster plane is calculated, and a plurality of local coordinate systems are defined on the extracted multi-image plane point cloud Wherein i represents the i-th camera plane, the multi Local coordinate system is re-calibrated, and the pose transformation between the plurality of local coordinate systems And the binocular camera coordinate system C left , here we define the left camera of the binocular camera as the binocular camera coordinate system;

[0006] S2: the point cloud data obtained by the laser radar is subjected to local normal vector calculation, then the point cloud is clustered based on the calculated normal vector attribute, considering that there are a large number of parallel planes in the environment, the laser point cloud of different planes may exist in the same normal vector cluster, the multi-cluster point cloud is filtered based on the plane fitting algorithm, and the final plane point cloud is obtained;

[0007] S3: an adaptive voxel is constructed with the laser radar and the binocular camera body coordinate system origin as the center, the extracted plane point cloud cluster is divided into the same voxel, the SPFH vector of each voxel is calculated, the descriptor is constructed for the local area voxel, and the heterogeneous point cloud inter-correlation is realized based on the descriptor matching;

[0008] S4: considering that a large number of noise points will be reproduced in the image-based point cloud stereo reconstruction process, the extracted plane and the normal vector have Gaussian noise, an active anti-noise external parameter optimization algorithm is used for iterative solution, and through the active anti-noise external parameter optimization algorithm, the optimization constraint quality can be monitored in real time, and the external parameter solving precision is improved.

[0009] Further, the step S1 comprises:

[0010] S101: assuming that the world coordinate system is {W}, the left camera coordinate system is {L}, and the right coordinate system is {R}, assuming that the internal parameter matrix K L And K R Of the left camera and the right camera are known, the P W (X,Y,Z) can be projected to obtain the pixel coordinates p L (x L ,y L ) T And p R (x R ,y R ) TUp,

[0011] p L = K L [R L |t L ]P W ,p R = K R [R R |t R ]P W

[0012] where R L ,t L and R R ,t R are the rotation matrix and translation vector of left and right camera respectively;

[0013] In ideal case, the coordinate system of left and right camera are parallel, the relationship between disparity d and depth Z is,

[0014] d = x L -x R

[0015]

[0016] where f is the focal length of left and right camera, B is the baseline length, d is the disparity, based on the disparity d and depth Z, the dense point cloud reconstructed based on binocular image can be easily obtained;

[0017] S102: Extract the plane cluster plane i based on RANSAC algorithm for the reconstructed dense image point cloud, and calculate the normal vector of each cluster plane, further to obtain the transformation relationship between each plane cluster and binocular camera coordinate system and Assuming that the binocular camera takes the left camera as the reference, a local coordinate system is defined on the plane point cloud The transformation relationship between each plane cluster and binocular camera coordinate system is obtained based on Zhang's calibration method.

[0018] Further, the step S2 comprises:

[0019] S201: First, principal component analysis is performed on the obtained radar point cloud, and the normal vector of each radar point is estimated, and the specific steps are as follows:

[0020] (a) Principal component analysis normal vector estimation

[0021] Assuming that the local neighborhood of each point cloud is N{P j},j = 1,2,3,…,n, the covariance matrix C is calculated:

[0022]

[0023] wherein m is the number of points in the neighborhood, is the mean of the neighborhood points, the corresponding eigenvalues λ1, λ2, λ3can be obtained by eigenvalue decomposition, and the larger eigenvalue is selected to represent the normal vector n of the point i

[0024] (b) DBSCAN clustering algorithm

[0025] Considering the unevenness of the distribution in space, the DBSCAN clustering algorithm is used to cluster , and multiple clusters in a single frame of point cloud can be obtained:

[0026] C = {C1, C2, …, C i , C k}

[0027] For each point cloud cluster C i extracted, the RANSAC algorithm is used to fit a plane, and noise points that do not conform to the plane model are filtered out according to the threshold of the fitted plane.

[0028] Further, the step S3 comprises:

[0029] S301: Since the laser radar and the binocular camera are installed close to each other, taking the laser radar as an example, adaptive voxel division is performed with the origin of the body coordinate system as the center, the extracted plane point cloud cluster C i is divided into the same voxel grid, small voxel grids are removed, and the neighborhood covariance matrix is calculated for the voxel grid.

[0030]

[0031] Eigenvalue decomposition is performed on C p , and the vector v3corresponding to the smallest eigenvalue is taken as the direction of the plane normal vector:

[0032] C p v j = λ j v j , λ1≥ λ2≥ λ3, n p = v3, ‖n p ‖ = 1

[0033] Three geometric features are calculated for each plane voxel grid. First, the Darboux reference frame is constructed

[0034]

[0035] Three geometric features are calculated

[0036]

[0037] Statistically analyze (a, f, q) respectively to obtain three sub-histograms, and construct SPFH vector based on the obtained sub-histograms

[0038] SPFH(p) = [h α (p);h φ (p);h θ (p)]

[0039] Describe sub-cluster of k-plane cluster {C1,…C k} in local range:

[0040]

[0041] Wherein C j is the jth plane cluster in {C1,…C k};

[0042] For dense point cloud reconstructed based on binocular image, construct plane cluster descriptor in the same way, and given two sets of cluster descriptors Wherein is a plane point cloud cluster descriptor of an image, is a plane point cloud descriptor of a laser radar;

[0043]

[0044] For registered plane point cloud, filter based on the following geometric consistency conditions, and keep matching pairs that satisfy both conditions

[0045]

[0046] Further, the step S4 comprises:

[0047] S401: Assuming that the external parameters of laser radar to camera are known, and Project the extracted multi-plane point cloud to the camera coordinate system:

[0048]

[0049] Further utilize binocular camera coordinate system to self-defined local coordinate system transformation and Project the extracted multi-plane point cloud to multiple self-defined local coordinate systems:

[0050]

[0051] Wherein and are and inverse transformation;

[0052] Further because the projection of the past Located in the xoy plane of the custom local coordinate system, there is a natural constraint z = 0:

[0053]

[0054] Considering that there are errors in the local coordinate system self-definition and the plane point cloud extraction process, in order to obtain accurate and Using the extrinsic parameter optimization algorithm with active noise reduction to iteratively solve.

[0055] Further, the extrinsic parameter optimization algorithm with active noise reduction includes:

[0056] For the i = 1, …, N plane points of a plane, define the residual as:

[0057]

[0058] Where the parameter vector Around some initial estimate θ0, express the small perturbation in Lie algebra ξ, and perform a first-order Taylor expansion:

[0059] r i (θ0+δξ)≈r i 0 +J i δξ

[0060] Where, r i 0 =r(θ0), Ideally r i 0 +J i δξ=0

[0061] Write all i = 1, …, N linear constraints in matrix form:

[0062]

[0063] Construct the augmented matrix If there is no noise, there is a non-zero vector x = [δξ, 1] such that Mx = 0

[0064] Under any perturbation ΔM, find the smallest Frobenius norm perturbation that makes (M + ΔM) rank deficient, ensuring the existence of a non-zero solution x:

[0065]

[0066] Do singular value decomposition on M, M = UΣV T ,Σ=diag(σ1,…,σd+1 ),σ1≥... ≥σ d+1 ≥0.

[0067] Obtaining the optimal perturbation is equivalent to eliminating σ d+1 The corresponding right singular vector v d+1 The augmented solution is:

[0068] x=v d+1 , V=[v1,..., v d+1 ], Mv d+1 =σ d+1 u d+1 ≈0

[0069]

[0070] Then it satisfies

[0071]

[0072] Get the increment

[0073]

[0074] Update the estimated parameter θ by exponential mapping:

[0075] θ new =exp(δξ)·θ

[0076] Iterative optimization is performed until ||δξ||≤Thr, and the final transformation parameter is obtained by stopping updating Wherein Thr is a heuristic threshold set by man, generally 0.001.

[0077] The beneficial effects of the present application are: compared with the existing scheme which depends on artificial calibration object and is easily affected by image distortion, the present method does not need any calibration target, fully utilizes the multi-distance and multi-angle plane structure in the scene, realizes full environment and full automatic calibration; at the same time, through the mutual correlation of multiple planes and active anti-noise optimization, the registration error is significantly reduced, the external parameter precision is improved, and the robustness to image distortion and occlusion noise in complex environment is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0078] Figure 1 is the flow chart of the method of the present application;

[0079] Figure 2 Based on the plane extraction effect diagram of laser point cloud;

[0080] Figure 3 It is the effect diagram of stereo matching based on binocular images;

[0081] Figure 4 It is the camera and laser radar calibration result diagram. DETAILED DESCRIPTION

[0082] In order to make the objectives, technical solutions and advantages of the present application clearer and more comprehensible, the present application will be further described in detail below with reference to the drawings and examples. However, it should be understood that the specific examples described herein are only used to explain the present application and are not intended to limit the scope of the present application.

[0083] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application.

[0084] As shown in Figure 1 A camera / lidar calibration method for monitoring power transmission conductor galloping includes the following steps:

[0085] S1: Based on the binocular camera internal parameter, the acquired left eye image and right eye image are subjected to dense stereo reconstruction, the reconstructed dense three-dimensional point cloud is classified based on the RANSAC algorithm, the plane point cloud is acquired and the normal vector of the multi-cluster plane is calculated, and a plurality of local coordinate systems are defined on the extracted multi-image plane point cloud Where i represents the i-th camera plane, the multi- local coordinate system is re-calibrated, and the pose transformation between the plurality of local coordinate systems and the binocular camera coordinate system C is obtained, and the left camera of the binocular camera is defined as the binocular camera coordinate system; left

[0086] S2: The point cloud data acquired by the lidar is subjected to local normal vector calculation, then the point cloud is clustered based on the calculated normal vector attribute, considering that there are a large number of parallel planes in the environment, the laser point cloud of different planes may exist in the same normal vector cluster, the multi-cluster point cloud is filtered based on the plane fitting algorithm, and the final plane point cloud is obtained;

[0087] S3: An adaptive voxel is constructed with the origin points of the lidar and the binocular camera body coordinate system as the center, the extracted plane point cloud cluster is divided into the same voxel, the SPFH vector of each voxel is calculated, the descriptor is constructed for the local area voxel, and the heterogeneous point cloud inter-correlation is realized based on the descriptor matching;

[0088] S4: Considering that a large number of noise points will be reproduced in the image-based point cloud stereo reconstruction process, the extracted plane and normal vector have Gaussian noise, an active anti-noise extrinsic parameter optimization algorithm is used for iterative solution, and through the active anti-noise extrinsic parameter optimization algorithm, the optimization constraint quality can be monitored in real time, and the extrinsic parameter solving accuracy is improved.

[0089] Step S1 includes: ​

[0090] S101: assuming a world coordinate system as {W}, a left camera coordinate system as {L}, and a right coordinate system as {R}, assuming the intrinsic matrix K L and K R of the left camera and the right camera W (X,Y,Z) can be projected to obtain the pixel coordinates p L (x L ,y L ) T and p R (x R ,y R ) T on the image plane

[0091] p L = K L [R L |t L ]P W , p R = K R [R R |t R ]P W

[0092] where R L , t L and R R , t R are the rotation matrix and translation vector of the left and right cameras, respectively

[0093] Under ideal conditions, the left and right camera coordinate systems are parallel, and the relationship between the disparity d and the depth Z is

[0094] d = x L -x R

[0095]

[0096] where f is the focal length of the left and right cameras, B is the baseline length, and d is the disparity. Based on the disparity d and the depth Z, a dense point cloud based on binocular image reconstruction can be easily obtained

[0097] S102: For the reconstructed dense image point cloud, the plane cluster plane i is extracted based on the RANSAC algorithm, and the normal vector of each cluster plane is calculated to further obtain the transformation relationship between each plane cluster and the binocular camera coordinate system and Assuming that the binocular camera takes the left camera as the reference, a local coordinate system is defined on the plane point cloud The transformation relationship between each plane cluster and the binocular camera coordinate system is obtained based on Zhang's calibration method, and the stereo matching result based on the binocular camera is as follows Figure 3as shown.

[0098] Step S2 includes:

[0099] S201: First, principal component analysis is performed on the acquired radar point cloud, and the normal vector of each radar point is estimated, and the specific steps are as follows:

[0100] (a) Principal component analysis normal vector estimation

[0101] Assuming that the local neighborhood N{P j},j=1,2,3,…,n, calculate its covariance matrix C:

[0102]

[0103] Where m is the number of points in the neighborhood, is the mean of the neighborhood points, and the corresponding eigenvalues λ1, λ2, λ3 can be obtained by eigenvalue decomposition, and the larger eigenvalue represents the normal vector n i

[0104] (b) DBSCAN clustering algorithm

[0105] Considering the unevenness of the distribution in space, the DBSCAN clustering algorithm is used to cluster , and multiple clusters in a single frame of point cloud can be obtained:

[0106] C={C1,C2,…,C i ,…,C k}

[0107] For each point cloud cluster C i extracted, the RANSAC algorithm is used for plane fitting, and the noise points that do not conform to the plane model are filtered out according to the fitting plane threshold, and the plane cluster point cloud is extracted, as shown in Figure 2 .

[0108] Step S3 includes:

[0109] S301: Since the laser radar and the binocular camera are installed close to each other, taking the laser radar as an example, adaptive voxel division is performed with the origin of the body coordinate system as the center, and the extracted plane point cloud cluster C i is divided into the same voxel grid, and the voxel grid with small number of points is removed, and the neighborhood covariance matrix of the voxel grid is calculated:

[0110]

[0111] The C p is decomposed, and the vector v3 corresponding to the minimum eigenvalue is taken as the plane normal vector direction:

[0112] Cp v j = λ j v j , λ1≥ λ2≥ λ3, n p = v3,‖n p ‖ = 1

[0113] For each planar voxel grid, calculate its three geometric features, first construct the Darboux reference frame

[0114]

[0115] Calculate the three geometric features

[0116]

[0117] Statistically analyze (α, φ, θ) respectively to obtain three sub-histograms, and construct the SPFH vector based on the obtained sub-histograms

[0118] SPFH(p) = [h α (p); h φ (p); h θ (p)]

[0119] Describe the sub-cluster of the local range k plane cluster {C1,…C k} range:

[0120]

[0121] Where C j is the jth plane cluster in {C1,…C k};

[0122] For the dense point cloud reconstructed based on binocular images, the same method is used to construct the plane cluster descriptor, and given two sets of cluster descriptors Where is a certain image plane point cloud cluster descriptor, is a certain laser radar plane point cloud descriptor;

[0123]

[0124] For the registered plane point cloud, based on the following geometric consistency conditions, the matching pairs that satisfy both conditions are retained

[0125]

[0126] Step S4 includes:

[0127] S401: Assuming that the external parameters of the laser radar to the camera are known, and Project the extracted multi-plane point cloud to camera coordinate system:

[0128]

[0129] Further utilize the binocular camera coordinate system to self-defined local coordinate system transformation and Project the extracted multi-plane point cloud to multi self-defined local coordinate system:

[0130]

[0131] where and is and inverse transformation.

[0132] Further because the projected is located in the xoy plane of the self-defined local coordinate system, there is a natural constraint z = 0:

[0133]

[0134] Considering that there are errors in the process of local coordinate system self-definition and plane point cloud extraction, in order to obtain accurate and Use the extrinsic parameter optimization algorithm with active noise reduction to iteratively solve.

[0135] The extrinsic parameter optimization algorithm with active noise reduction includes:

[0136] For the i = 1, …, N plane points of a certain plane, define the residual as:

[0137]

[0138] where the parameter vector Around a certain initial estimate θ0, express the small perturbation in Lie algebra ξ, and perform first-order Taylor expansion to get:

[0139] r i (θ0+δξ)≈r i 0 +J i δξ

[0140] where, r i 0 = r(θ0), Ideally r i 0 +J i δξ=0

[0141] Write all i = 1, …, N linear constraints in matrix form:

[0142]

[0143] Constructing augmented matrix If there is no noise, then there exists a non-zero vector x = [δξ, 1] such that Mx = 0

[0144] Under any small perturbation ΔM, find the minimum Frobenius norm perturbation that reduces the rank of (M+ΔM) and ensures the existence of a non-zero solution x:

[0145]

[0146] Perform singular value decomposition on M, M=UΣV T ,Σ=diag(σ1,…,σ d+1 ),σ1≥…≥σ d+1 ≥0.

[0147] Obtaining the optimal perturbation is equivalent to eliminating σ d+1 The corresponding right singular vector v d+1 It is the augmented solution:

[0148] x=v d+1 ,V=[v1,…,v d+1 ],Mv d+1 =σ d+1 u d+1 ≈0

[0149]

[0150] Then it satisfies

[0151]

[0152] Get the increment

[0153]

[0154] Update the estimated parameters θ through the exponential mapping:

[0155] θ new =exp(δξ)2θ

[0156] Perform iterative optimization until ||δξ||≤Thr, stop updating and get the final transformation parameters Thr is a manually set heuristic threshold, usually 0.001.

[0157] like Figure 4 As shown in Figure 1, field experiments were conducted in an outdoor environment to verify the accuracy. The extracted planar point cloud and the extracted image reconstruction point cloud were used to test the camera / lidar calibration algorithm based on the proposed multi-plane structure active anti-error method, and the lidar and binocular camera calibration results shown in Table 1 were obtained.

[0158] Table 1 Camera / Lidar Calibration Results

[0159] 0.04802 -0.01981 0.99865 0.09111 -0.99884 -0.00332 0.04796 0.06278 0.00236 -0.99979 -0.01994 -0.07651 0 0 0 1

[0160] The above description is merely that of the preferred embodiments of the application and is not intended to limit its scope. Modifications and variations are conceivable within the spirit and scope of the application as can be seen by the appended claims.

Claims

1. A camera / lidar calibration method for power transmission line galloping monitoring, characterized by: It includes the following steps: S1: Based on the intrinsic parameters of the binocular camera, the acquired left and right eye images are densely stereo reconstructed. The reconstructed dense 3D point cloud is classified based on the RANSAC algorithm, the planar point cloud is obtained and the normal vectors of the multi-cluster planes are calculated. Multiple local coordinate systems are defined on the extracted multi-image planar point cloud, and the multiple local coordinate systems are recalibrated. The pose transformation between the multiple local coordinate systems and the binocular camera coordinate system is obtained, and the left camera of the binocular camera is defined as the binocular camera coordinate system. S2: Calculate local normal vectors for the point cloud data acquired by the lidar, then cluster the point cloud based on the calculated normal vector attributes, and then filter the multi-clustered point cloud based on the plane fitting algorithm to obtain the final plane point cloud; S3: Adaptive voxels are constructed with the origin of the laser radar and binocular camera coordinate systems as the center, the extracted planar point cloud is clustered into the same volume, the SPFH vector of each voxel is calculated, a descriptor is constructed for the local area voxels, and the heterogeneous point cloud is correlated based on descriptor matching; S4: Considering that a large number of noise points will be reproduced in the image-based point cloud stereo reconstruction process, the extracted planes and normal vectors have Gaussian noise, and an external parameter optimization algorithm with active noise reduction is used to iteratively solve the problem.

2. The camera / lidar calibration method for power transmission line galloping monitoring according to claim 1, characterized in that: The step S1 includes: S101: Assume that the world coordinate system is {W}, the left camera coordinate system is {L}, and the right coordinate system is {R}. Assume that the intrinsic parameter matrix K of the left and right cameras is L and K R It is known that P W (X, Y, Z) projection obtains the pixel coordinate p of the image plane L (x L ,y L ) T and p R (x R ,y R ) T superior, p L =K L [R L |t L ]P W ,p R =K R [R R |t R ]P W Among them, R L ,t L and R R ,t R The rotation matrix and translation vector of the left and right cameras respectively; Ideally, the left and right camera coordinate systems are parallel, and the relationship between the parallax d and the depth Z is, d=x L -x R Among them, f is the focal length of the left and right cameras, B is the baseline length, and d is the disparity. Based on the disparity d and the depth Z, it is easy to obtain a dense point cloud based on binocular image reconstruction; S102: Extract plane clusters based on the RANSAC algorithm for the reconstructed dense image point cloud i , and calculate the normal vector of each cluster plane, and further obtain the transformation relationship between each plane cluster and the binocular camera coordinate system and Assuming that the binocular camera takes the left camera as a reference, define the local coordinate system on the plane point cloud The transformation relationship between each plane cluster and the binocular camera coordinate system is obtained based on Zhang's calibration method.

3. The camera / lidar calibration method for power transmission line galloping monitoring according to claim 1, characterized in that: The step S2 comprises: S201: First, perform principal component analysis on the acquired radar point cloud to estimate the normal vector of each radar point. The specific steps are as follows: (a) Principal component analysis normal vector estimation Assume that the local neighborhood N{P j }, j = 1, 2, 3, ..., n, calculate its covariance matrix C: Among them, m is the number of points in the neighborhood, is the mean of the neighborhood points. The corresponding eigenvalues ​​λ1, λ2, and λ3 can be obtained by eigenvalue decomposition. The larger eigenvalue is selected to represent the normal vector n of the point. i (b) DBSCAN clustering algorithm Taking into account The DBSCAN clustering algorithm is used to cluster the non-uniform distribution of By performing clustering, you can obtain multiple clusters within a single frame point cloud: C={C1,C2,…,C i ,…,C k } For each point cloud cluster C extracted i ,The RANSAC algorithm is used to perform plane fitting, and noise points that do not conform to the plane model are filtered out according to the fitting plane threshold.

4. The camera / lidar calibration method for power transmission line galloping monitoring according to claim 1, characterized in that: The step S3 comprises: S301: Since the laser radar and the binocular camera are installed close to each other, taking the laser radar as an example, adaptive voxel division is performed with the origin of the body coordinate system as the center, and the extracted plane point cloud is clustered into C i Divide into the same voxel grid, remove the voxel grid with smaller points, and calculate the neighborhood covariance matrix of the voxel grid: C p Perform eigendecomposition and take the vector v3 corresponding to the minimum eigenvalue as the direction of the plane normal vector: C p v j =λ j v j ,λ1≥λ2≥λ3,n p =v3,‖n p ‖=1 For each planar voxel grid, calculate its three geometric features. First, construct the Darboux reference system. Calculate three geometric features Statistical analysis is performed on (α, φ, θ) to obtain three subhistograms, and the SPFH vector is constructed based on the obtained subhistograms. SPFH(p)=[h α (p);h φ (p);h θ (p)] For the local k-plane clustering {C1,…C k } range for descriptor clustering: Among them C j For {C1,…C k }jth plane cluster in; For dense point clouds reconstructed based on binocular images, the same method is used to construct planar cluster descriptors. Given two sets of cluster descriptors in is a clustering descriptor of a certain image plane point cloud, is a certain laser radar plane point cloud descriptor, For the registration of planar point clouds, we filter based on the following geometric consistency conditions and retain the matching pairs that meet both conditions.

5. The camera / lidar calibration method for power transmission line galloping monitoring according to claim 1, characterized in that: The step S4 comprises: S401: Assuming that the external parameters of the laser radar to the camera are known, and Project the extracted multi-plane point cloud to the camera coordinate system: Further use the binocular camera coordinate system to custom local coordinate system transformation and Project the extracted multi-planar point cloud into multiple custom local coordinate systems: in and for and The inverse transform of Further because of the projection of the past The xoy plane in the custom local coordinate system has a natural constraint z = 0: Considering the errors in the local coordinate system customization and plane point cloud extraction process, in order to obtain accurate and The solution is iteratively solved using an extrinsic parameter optimization algorithm with active noise reduction.

6. A camera / lidar calibration method for power transmission line galloping monitoring according to claim 1 or 5, characterized in that: The external parameter optimization algorithm for active noise reduction includes: For the i=1,…,Nth plane point of a certain plane, the residual is defined as: The parameter vector Around a certain initial estimate θ0, the small perturbation is represented by the Lie algebra ξ, and a first-order Taylor expansion is performed to obtain: r i (θ0+δξ)≈r i 0 +J i right Among them, r i 0 =r(θ0), Ideally, i 0 +J i δξ=0 Write all i=1,…,N linear constraints in matrix form: Constructing augmented matrix If there is no noise, then there exists a non-zero vector x = [δξ, 1] such that Mx = 0 Under any small perturbation ΔM, find the minimum Frobenius norm perturbation that reduces the rank of (M+ΔM) and ensures the existence of a non-zero solution x: Perform singular value decomposition on M, M=UΣV T ,Σ=diag(σ1,…,σ d+1 ),σ1≥…≥σ d+1 ≥0, Obtaining the optimal perturbation is equivalent to eliminating σ d+1 The corresponding right singular vector v d+1 It is the augmented solution: x=v d+1 ,V=[v1,…,v d+1 ],Mv d+1 =σ d+1 u d+1 ≈0 Then it satisfies Get the increment Update the estimated parameters θ through the exponential mapping: i new =exp(δξ)·θ Perform iterative optimization until ||δξ||≤Thr, stop updating and get the final transformation parameters Thr is a manually set heuristic threshold, usually 0.001.

Images