Lattice tower rod piece missing detection method based on three-dimensional reconstruction

Abstract

The invention belongs to the technical field of lattice tower structure health detection, and relates to a lattice tower rod piece missing intelligent detection method based on multi-view three-dimensional reconstruction. According to the method, an adaptive feature matching algorithm fusing geometric features and texture descriptors is provided, camera pose estimation is optimized by constructing a global depth consistency scoring mechanism, and high-precision sparse point cloud reconstruction is realized; designing a depth map optimization strategy based on energy minimization, and effectively overcoming the interference of complex illumination and a low-texture environment on three-dimensional reconstruction in combination with density threshold filtering and geometric structure feature analysis; an ordered / disordered point cloud adaptive noise reduction method is innovatively provided, and noise robustness processing is realized through gradient constraint and covariance matrix eigenvalue analysis; and finally, an improved neighborhood radius adaptive registration algorithm is adopted, and a rod piece missing region is accurately identified through a significance scoring mechanism. The method remarkably improves the automation level and reliability of rod piece missing detection, and can be widely applied to structural health detection of the lattice tower.

Description

Technical Field

[0001] This invention belongs to the field of structural health detection and three-dimensional vision detection technology, and relates to a method for detecting missing lattice tower members based on three-dimensional reconstruction. It is a method for detecting missing lattice tower members based on multi-view three-dimensional reconstruction and intelligent point cloud analysis, and is particularly suitable for automated safety status assessment of lattice steel structures such as power towers and communication towers. Background Technology

[0002] As the core supporting structure of infrastructure such as power transmission networks and communication base stations, lattice towers are exposed to complex environments for extended periods, making them prone to localized defects such as member breakage and bolt loosening due to material aging, mechanical damage, or extreme loads. Traditional manual inspections rely on visual observation and experience-based judgment, which suffers from drawbacks such as low efficiency, high-risk high-altitude operations, and a high rate of missing minute defects. In recent years, 3D reconstruction technology based on UAV imagery has provided a new approach for automated inspection of lattice towers, but it still faces the following technical bottlenecks:

[0003] (1) Insufficient feature matching accuracy: The lattice tower members have high similarity and dense repetitive textures. Traditional feature descriptors such as SIFT and ORB have poor distinguishability between linear structures and low-texture areas, resulting in a high mismatch rate and incomplete or distorted reconstructed point clouds.

[0004] (2) Severe noise interference: The reconstructed point cloud is easily affected by changes in ambient light, vegetation shading, metal reflection, etc. Existing filtering algorithms (such as statistical filtering and radius filtering) are not sensitive enough to structural noise (such as pseudo rods and discrete noise points), and are prone to accidentally deleting real rod point clouds.

[0005] (3) Low reliability of missing detection: Existing methods mostly rely on rigid registration between point cloud and CAD model, without considering geometric deviations caused by structural deformation and corrosion during service, and lack quantitative criteria for local missing, resulting in high false alarm and false negative rates. Summary of the Invention

[0006] To address the problems of low feature matching accuracy, weak noise suppression, and insufficient tolerance for geometric deviations in existing lattice tower member missing detection technologies, this invention proposes a member missing detection method that integrates multi-view 3D reconstruction and intelligent point cloud analysis. By designing a geometry-texture fusion descriptor to improve feature matching robustness, a dense reconstruction optimization model based on depth consistency scoring and structural constraints is constructed. Furthermore, a gradient-density dual-driven point cloud denoising algorithm and a local geometric feature missing detection framework are proposed, enabling accurate detection of lattice tower members in complex environments and effectively solving the problem of automated member missing detection in complex environments.

[0007] To achieve the above objectives, the specific technical solution of the present invention is as follows:

[0008] A method for detecting missing members in a lattice tower based on 3D reconstruction includes the following steps:

[0009] Step 1: 3D Reconstruction of Lattice Tower Members Based on Multiple Views

[0010] (1.1) Feature matching

[0011] For a series of images of the lattice tower captured by a single camera, the overlap between two images is determined by matching to determine whether they are adjacent, achieving the effect of simulating multi-view images with a single camera. After the simulation, to evaluate the matching degree of the simulated multi-view images, geometric feature points in the images of the lattice tower members are extracted to obtain geometric descriptors, based on the linear and symmetry features of the lattice tower members. Local adaptive filtering is then applied to the texture features to extract texture descriptors. The two images are used as the left and right images, respectively. For the left image, the pixel is p, and the calculation formula is as follows (the calculation method for pixel q in the right image is similar):

[0012]

[0013]

[0014] in, It is a geometric descriptor. It is a texture descriptor; It is the grayscale value of the image. It is a pixel. exist and Image gradient in the direction;

[0015] The gradient direction of a pixel is defined as:

[0016]

[0017] The filter weights, used to adjust the contribution of different points in the calculation to adapt to changes in photometric intensity, are defined as follows:

[0018]

[0019] in, Indicates the first The preset direction angles of each filter. These are predefined directions for each filter, used to capture texture features at different angles. K is limited to 1~5, meaning that the entire angle range is covered by filters in 5 different directions (each direction is spaced 72° apart). Lattice towers are typically composed of orthogonally or symmetrically distributed rods (such as horizontal, vertical, and diagonal lines), and their main directional angle distribution is relatively regular. Five evenly spaced directions (72° intervals) can cover the entire angle range, capturing the multi-directional linear features of the rods (such as horizontal and diagonal rods) while avoiding redundant calculations.

[0020] It is the first The local grayscale values ​​of each filter are defined as follows:

[0021]

[0022] in, , They are respectively exist and Sobel gradient kernel in the direction, For convolution operations, different directions It will vary depending on the direction of the gradient kernel.

[0023] The geometric and texture features are fused by uniformly distributing weights, and the combined feature points are defined as follows:

[0024]

[0025] After the comprehensive feature point extraction is completed, the Euclidean distance between the feature points is calculated based on the comprehensive feature points. The point with the smallest distance is selected as the initial matching feature point. The distance calculation formula is as follows:

[0026]

[0027] in, This represents the comprehensive feature point of pixel p in the left image; The above represents the comprehensive feature points of pixel q in the right image.

[0028] When the overlap of combined feature points between two images reaches 70%, they are considered adjacent images, thus completing feature matching. The overlap is determined by the following formula:

[0029]

[0030] in, , These represent the number of combined feature points in the left and right images, respectively. This is the set of interior point matching pairs selected using the random sampling consensus algorithm.

[0031] (1.2) Sparse Reconstruction

[0032] After selecting inlier points from the initial matching feature points using a random sampling consensus algorithm, the geometric relationship between the two images is estimated based on the coordinates of these inlier points. Estimating the geometric relationship requires calculating the fundamental matrix. and essential matrix ;

[0033] Fundamental matrix The calculation formula is as follows:

[0034]

[0035]

[0036] in, , These are the homogeneous coordinates of the combined feature points in the two images; The intrinsic parameter matrix of the camera is defined as follows:

[0037]

[0038] in, and Corresponding to the camera at and Focal length in direction, and For the camera main point at and pixel coordinates on;

[0039] The essential matrix Decomposed into the camera rotation matrix Translation vector The relative rotation and translation of the two images are obtained, and the formula is defined as follows:

[0040]

[0041] in, for An antisymmetric matrix.

[0042] This yields the essential matrix. The decomposition is not unique and needs to be verified using the depth values ​​of the triangulated points to obtain the unique correct pose; assuming the projection matrices of the two images are:

[0043]

[0044]

[0045] in, It is The identity matrix, yes The zero vector.

[0046] The homogeneous coordinates of each pair of matched feature points in three-dimensional space are obtained using the triangulation formula. The formula is as follows:

[0047]

[0048] in, Homogeneous coordinates are achieved by adding one dimension (usually 1) to a three-dimensional point. Represented as a four-dimensional vector, it allows rotation and translation changes to be represented simultaneously through a single matrix multiplication.

[0049] For each group of candidate poses Define global depth consistency score :

[0050]

[0051] in, and For the first Triangulation of depth values ​​in both images To the total number of matching point pairs, This is an indicator function that evaluates to true when the depth values ​​in both images are positive. Set the value to 1, otherwise set it to 0.

[0052] The global depth consistency scores of the candidate poses are compared, and the group with the highest score is selected as the final camera pose. The three-dimensional coordinates of each integrated feature point in the image are output to complete the sparse point cloud reconstruction.

[0053] (1.3) Dense Reconstruction

[0054] The disparity value of the 3D coordinates of the same composite feature point in two images is calculated using the following formula:

[0055]

[0056] Where S is the disparity value, and These are the x-coordinates of the pixels in the left and right images, respectively.

[0057] Calculate the depth value for each pixel using the intrinsic parameters and parallax values ​​defined in the camera calibration:

[0058]

[0059] in, For depth value, For camera focal length, The distance between the optical centers of the two cameras is given.

[0060] The depth values ​​are optimized using an energy minimization method to ensure global consistency. The energy function is defined as follows:

[0061]

[0062]

[0063]

[0064] in, Represents a pixel in a two-dimensional image. This represents the total energy function for depth value optimization. Its core objective is to minimize this function so that the depth estimation results simultaneously satisfy data consistency and spatial smoothness. This is a data item that measures the accuracy of the current image depth value matching the disparity. It is a smoothing term that constrains the smoothness of the depth map (which is constructed by taking the depth values ​​of each point in the input image) within a local region; , for , Grayscale value, , for , The offset of a pixel in the parallax direction. This indicates the pixel position after taking parallax into account. express for The neighboring points.

[0065] The depth map of each image is converted into a 3D point cloud, and the point clouds are then fused to eliminate redundant points. The formula for converting a depth map into a 3D point cloud is as follows:

[0066]

[0067]

[0068] in, For three-dimensional point world coordinates, The image coordinates of the pixels. This represents the image depth value.

[0069] By using the camera's pose matrix, point clouds from different viewpoints are mapped to a unified world coordinate system. :

[0070]

[0071]

[0072] Output the world coordinates of the 3D coordinates of all integrated feature points in the image to complete dense reconstruction.

[0073] Step 2: Denoising the point cloud model of the lattice tower members

[0074] (2.1) Ordered point cloud noise reduction processing

[0075] For ordered point clouds, noise reduction is performed based on their specific arrangement structure using the spatial gradient and mean filtering methods of the point cloud; firstly, the point cloud gradient is defined. as follows:

[0076]

[0077] in, It is a point The three-dimensional coordinates The coordinates of the neighboring points; It is a point The neighborhood of is consistent with the range of values ​​in the densely reconstructed neighborhood; It is the weight of the spatial proximity of the current point to the neighboring points, used to measure the grayscale difference between the current point and its neighbors; the calculation formula is as follows:

[0078]

[0079] in, It is a parameter sensitive to gradient changes.

[0080] On point The gradient exceeds the global mean gradient. Plus twice the standard deviation When a point is identified as a noise point, the formula is as follows:

[0081]

[0082]

[0083]

[0084] in, It represents the total number of point clouds. It is a point The magnitude of the gradient, , , It is a point The gradient in the x, y, and z directions.

[0085] For non-noise points, optimize them using the mean of their neighborhood points:

[0086]

[0087] in, It is a point The optimized three-dimensional coordinates.

[0088] (2.2) Noise Reduction Processing of Disordered Point Clouds

[0089] For unordered point cloud data, a noise identification and filtering method based on point density distribution and local geometric features is adopted. This is based on the global statistics of point density. and standard deviation Define the density threshold for points:

[0090]

[0091]

[0092]

[0093] in, The optimal rigid body transformation parameters include both translation and rotation. For the target point cloud (reference model) in the th The three-dimensional coordinates of the points The first point cloud (data to be registered) in the source point cloud The three-dimensional coordinates of each point; These are the weighting coefficients;

[0094] when Less than the threshold ( When judging the point Noise points; among them For point Local neighborhood density, The global density mean. To remove the global density standard deviation.

[0095] After removing noise points, to ensure the geometric consistency of the remaining point cloud, the points are... neighborhood Calculate the covariance matrix:

[0096]

[0097] in, Neighboring points The three-dimensional coordinates The index of the current center point. For neighborhood point index, It is the average value of the neighborhood points.

[0098] eigenvalues ​​of covariance matrix For point The local geometric distribution characteristics correspond to the three basic structural types of the point neighborhood (linear, planar, and spherical). When The value is When any one of them is more than 10 times the value of the others, the point distribution is a linear structure (such as the edge points of a rod). When and for When the number of points is more than 5 times the normal value, the point distribution becomes a planar structure (such as points on the surface of a rod). When the pairwise difference is within 0.1, the point distribution is spherical (e.g., noise points). Based on the geometric structure information of the points, the following characteristic indicators are defined:

[0099]

[0100]

[0101] when Determine if the point is a surface point; otherwise, discard it. This point is retained as a linear structure point.

[0102] Step 3: Detection of missing members in lattice towers based on point cloud registration

[0103] (3.1) Coarse registration of 3D point cloud

[0104] An improved intrinsic shape feature algorithm is used for coarse registration. The specific process is as follows:

[0105] First, extract the comprehensive feature points, for each pixel. The neighborhood radius is adaptively adjusted using the following formula:

[0106]

[0107] in, The neighborhood radius, The default value is 1.5, which is an empirically optimized value. The default value has been verified through a large number of experiments (such as S3DIS and KITTI datasets). It is a comprehensive trade-off between local geometric sensitivity, noise robustness and computational efficiency. yes by A spherical neighborhood with radius , , These are the three-dimensional coordinates of the current point and its neighboring points.

[0108] After calculating the covariance matrix and eigenvalues ​​of the point neighborhood according to the method for denoising disordered point clouds in step (2.2), a significance score is obtained:

[0109]

[0110] Calculate the local threshold:

[0111]

[0112] in, The default value is 1.5, a classic statistical threshold that strikes a balance between noise removal and preservation of true features. and Here, represents the mean and standard deviation of the significance score, respectively, as shown in the following formula:

[0113]

[0114]

[0115] reserve The points are identified, and geometric redundancy suppression is applied to the feature points in the point cloud. The significance scores of two points p and q are defined as similar if and only if |S(p)-S(q)|<δ, where δ is a preset threshold (usually 0.5 times the standard deviation of the significance score). Then, in the neighborhood of point p... Points that satisfy the conditions constitute a set of similar points. Calculate redundancy score .

[0116]

[0117] in, The significance score is calculated after redundancy suppression.

[0118] Keep each point The point cloud with the highest composite feature point value is assigned a significance score weight to each composite feature point. :

[0119]

[0120]

[0121]

[0122] in, For point curvature, This is the average curvature of all points.

[0123] By matching and combining feature point pairs, a weighted distance function is defined:

[0124]

[0125] Minimize the weighted distance objective function to obtain the initial transformation matrix, thus completing the coarse registration:

[0126]

[0127] in, For the target point cloud (reference model) in the th The three-dimensional coordinates of the points The first point cloud (data to be registered) in the source point cloud The three-dimensional coordinates of each point; , for Significance score weights are used to suppress redundant feature points. This is the coarse registration transformation matrix.

[0128] (3.2) Fine registration of 3D point clouds

[0129] Based on coarse registration, the nearest neighbor algorithm, which uses dynamic neighborhood weights and normal vector consistency constraints, is further optimized to improve registration accuracy. The specific algorithm flow is as follows: using the coarse registration transformation matrix... For each point Find the nearest neighbor Minimize the objective function And update the rigid body transformation matrix. The process is iterated until convergence to complete the fine registration. The specific formula is as follows:

[0130]

[0131]

[0132]

[0133] in, The objective function for precise registration; It is the spatial neighborhood weight of the weighted distance. (Experiments show that 1.5x resolution is robust to most sensor data). It is the normal vector consistency weight; , is the angular threshold of the normal vector deviation. The normal vector deviation of rigid structures such as lattice towers is usually less than 10°. For the target point cloud The unit normal vector at each point This is the unit normal vector of the corresponding matching point in the source point cloud.

[0134] (3.3) Visualization of missing members

[0135] After point cloud registration is completed, the source point cloud and the target model point cloud are compared point by point to detect missing members and visualize them. The residual distance of each point is calculated.

[0136]

[0137] Standardize the residuals:

[0138]

[0139] in, and These are the residual mean and standard deviation, respectively.

[0140] when ( When the default value is 2 (the standard value for determining the integrity of a member), the current point is considered a missing point. Color coding is used to visualize missing members by changing their color: green represents a complete area, and red represents a missing area.

[0141] The beneficial effects of this invention are:

[0142] This invention proposes an adaptive feature matching algorithm that integrates geometric features and texture descriptors. It optimizes camera pose estimation by constructing a global depth consistency scoring mechanism, achieving high-precision sparse point cloud reconstruction. A depth map optimization strategy based on energy minimization is designed, combining density threshold filtering and geometric structure feature analysis to effectively overcome the interference of complex lighting and low-texture environments on 3D reconstruction. An innovative adaptive denoising method for ordered / unordered point clouds is proposed, achieving robust noise processing through gradient constraints and covariance matrix eigenvalue analysis. Finally, an improved neighborhood radius adaptive registration algorithm is employed, accurately identifying missing member regions through a saliency scoring mechanism. This method significantly improves the automation level and reliability of missing member detection and can be widely applied to the structural health monitoring of lattice towers. Detailed Implementation

[0143] The specific embodiments of the present invention will be further described below in conjunction with the technical solution.

[0144] A method for detecting missing members in lattice towers based on 3D reconstruction, the specific steps of which are as follows:

[0145] (1) Data collection

[0146] A drone equipped with a high-resolution monocular camera was used to fly around the lattice tower and take pictures, ensuring that the overlap of each image is ≥80% and the pitch angle is ≤15°.

[0147] (2) Multimodal feature extraction and fusion

[0148] Geometry-texture fusion descriptor computation: Calculating pixel gradients and gradient direction Generate geometric descriptors For texture features, five directional filters (0°, 72°, 144°, 216°, 288°) are used to extract local grayscale values. Combined with weights Generate texture descriptors Weighted fusion of geometric and texture features The Euclidean distance is used to match the feature points of adjacent images (with an overlap of more than 70%) to identify adjacent images.

[0149] (3) Three-dimensional reconstruction

[0150] Sparse reconstruction: Interior points are selected from matching feature points using a random sampling consensus algorithm. Rotation and translation vectors are obtained after decomposing the fundamental and essential matrices. A global depth consensus score is calculated for each candidate pose. The pose with the highest score is selected as the final result.

[0151] Dense Reconstruction: Utilizing the Disparity Values ​​of Left and Right Images and camera parameters (focal length) and baseline ) Calculate depth The depth map is optimized using an energy minimization function.

[0152] (4) Point cloud noise reduction

[0153] Ordered point cloud denoising: Based on spatial gradient and weight For ordered point clouds, noise reduction is achieved by utilizing the spatial gradient and mean filtering methods based on the specific arrangement structure of the point cloud. First, the point cloud gradient is defined as follows:

[0154]

[0155] in, It is a point The three-dimensional coordinates The coordinates of the neighboring points; It is a point The neighborhood of is consistent with the range of values ​​in the densely reconstructed neighborhood; It is the weight of the spatial proximity of the current point to the neighboring points, used to measure the grayscale difference between the current point and its neighbors; the calculation formula is as follows:

[0156]

[0157] in, It is a parameter sensitive to gradient changes. When point The gradient exceeds the global mean gradient. Plus twice the standard deviation When the time is right, it is determined to be a noise point.

[0158] Denoising of disordered point clouds: using local density thresholding The sum and covariance matrix eigenvalues ​​(when Determine if the point is a surface point; otherwise, discard it. (This point is retained as a linear structure point.)

[0159] (5) Point cloud registration

[0160] Coarse registration: Improved intrinsic shape feature algorithm, adaptive neighborhood radius (γ=1.5), based on significance score and redundancy score Filter feature points and minimize weighted distance Obtain the initial transformation matrix .

[0161] Fine-grained registration: based on dynamic neighborhood weights Consistency constraint with normal vector The rigid body transformation matrix T is iteratively optimized until convergence.

[0162] (6) Missing item detection and visualization

[0163] Calculate the residual distance between the source point cloud and the target model point cloud. After standardization, , when ( When the default value is 2 (the standard value for determining the integrity of a member), the current point is considered a missing point. Color coding is used to visualize missing members by changing their color: green represents a complete area, and red represents a missing area.

[0164] The specific steps of this embodiment are as follows:

[0165] I. Experimental Platform and Data Acquisition

[0166] 1. Equipment and Calibration

[0167] Raw images were acquired using a drone equipped with a monocular camera (20 megapixels, shutter speed ≥ 1 / 1000 s). The camera resolution was 5472×3648 pixels, with a pixel size of 3.45 μm. Intrinsic parameter calibration was performed using a calibration board.

[0168]

[0169] in, The principal point is located at the center of the image frame. This intrinsic parameter is used for estimation and triangulation of the fundamental and essential matrices.

[0170] 2. Flight path and shooting parameters

[0171] The drone circled the tower twice, taking panoramic photos at an altitude of 45–55m, with a lateral overlap of ≥80%, a pitch angle of ≤15°, and a flight speed of 1.5–2.0m / s. Each circle was spaced 8–10m apart to improve parallax. The "overlap ≥80%, pitch angle ≤15°" constraints are consistent with the aforementioned data acquisition constraints.

[0172] II. Feature Extraction and Neighbor Image Determination

[0173] 1. Geometry-texture fusion descriptor:

[0174] For each image pixel Calculate the gradient , With direction Constructing geometric descriptors The texture descriptor uses five directional filter angles (0°, 72°, 144°, 216°, 288°) and is arranged according to... Weighted Comprehensive descriptor Adjacent images are determined by a comprehensive feature matching overlap of ≥70%.

[0175] 2. Parameter settings:

[0176] Pyramid layer 3, Sobel kernel 3×3; initial matching threshold: 10th percentile of feature Euclidean distance; RANSAC 8-point method, inlier threshold ε=0.8 pixels, 5000 iterations; feature overlap threshold 0.70.

[0177] III. Sparse / Dense Reconstruction and Depth Optimization

[0178] 1. Pose estimation and triangulation

[0179] Interior set estimation of fundamental matrix With the essential matrix Decomposition yields And based on "global depth consistency score" Choose the optimal pose. Triangulation yields sparse points.

[0180] 2. Parallax and Depth Calculation

[0181] parallax Depth calculation based on camera focal length and baseline: .in, Take the distance between the optical centers of adjacent poses , Take pixel focal length ; Apply a credibility mask to areas with small parallax.

[0182] 3. Energy minimization optimization

[0183] Using energy function In this embodiment, The data terms employ the parallax consistency L2 norm, and the smoothing term uses the edge-preserving term of the guided image. After multi-view depth fusion, a dense point cloud is generated using voxel filtering (10mm).

[0184] IV. Point Cloud Noise Reduction

[0185] 1. Ordered point cloud (obtained by rasterization projection)

[0186] Calculate the gradient magnitude of the point cloud and use it as follows Determine noise, weights ; It is a gradient-sensitive parameter, typically set to a value that corresponds to a fraction of the point cloud resolution. For non-noise points, the coordinates are optimized using a weighted mean.

[0187] 2. Disordered point cloud (obtained by multi-view fusion)

[0188] Estimating local density using neighborhood ,by Remove sparse outliers; then calculate the neighborhood covariance eigenvalues. flatness linearity .reserve Surface points and Linear structure points are used to eliminate spherical noise.

[0189] V. Point Cloud Registration

[0190] 1. Coarse registration (improving intrinsic shape features)

[0191] by Define adaptive neighborhood radius Significance score Defined by spectral features, threshold Weighted by significance score Suppress redundancy. Obtain the initial rigid body transformation. .

[0192] 2. Precise registration:

[0193] based on and Iterative optimization continues until the rigid body transformation matrix converges.

[0194] VI. Missing Detection and Visual Output

[0195] Calculate the minimum residual distance from the source point to the target point by point. Standardize:

[0196]

[0197] when Determined as a missing point ( (Default value is 2). Cluster connected clusters according to the axis direction of the members to form "missing segments". Export *.ply and *.csv files (start and end points, length, and member number of the missing segment).

Claims

1. A method for detecting missing members in a lattice tower based on three-dimensional reconstruction, characterized in that, Includes the following steps: Step 1: 3D Reconstruction of Lattice Tower Members Based on Multiple Views (1.1) Feature matching For a series of images of a lattice tower captured by a single camera, the overlap between two images is determined by matching to identify whether they are adjacent, achieving a multi-view effect simulated by a single camera. After the simulation, to evaluate the matching degree of the simulated multi-view images, geometric feature points in the images of the lattice tower members are extracted to obtain geometric descriptors, based on the linear and symmetry features of the lattice tower members. Local adaptive filtering is then applied to the texture features to extract texture descriptors. Two images are used as the left and right images, respectively. For the left image, the number of pixels is p, and the calculation formula is as follows: ； ； in, It is a geometric descriptor. It is a texture descriptor; It is the grayscale value of the image. It is a pixel. exist and Image gradient in the direction; The gradient direction of a pixel is defined as: ； The filter weights, used to adjust the contribution of different points in the calculation to adapt to changes in photometric intensity, are defined as follows: ； in, Indicates the first The preset direction angle of each filter; k=1~5, that is, the full angle range is covered by filters in 5 different directions; It is the first The local grayscale values ​​of each filter are defined as follows: ； in, , They are respectively exist and Sobel gradient kernel in the direction, For convolution operations, different directions It will differ depending on the direction of the gradient kernel; The geometric and texture features are fused by uniformly distributing weights, and the combined feature points are defined as follows: ； After the comprehensive feature point extraction is completed, the Euclidean distance between the feature points is calculated based on the comprehensive feature points. The point with the smallest distance is selected as the initial matching feature point. The distance calculation formula is as follows: ； in, This represents the comprehensive feature point of pixel p in the left image; The comprehensive feature point of pixel q in the right figure; When the overlap of the combined feature points of the two images reaches 70%, they are determined to be adjacent images, and feature matching is completed. (1.2) Sparse Reconstruction After selecting inlier points from the initial matching feature points using a random sampling consensus algorithm, the geometric relationship between the two images is estimated based on the coordinates of these inlier points. Estimating the geometric relationship requires calculating the fundamental matrix. and essential matrix ; For each group of candidate poses Define global depth consistency score : ； in, and For the first Triangulation of depth values ​​in both images To the total number of matching point pairs, This is an indicator function that takes a positive depth value in either image as an indication when the condition is true. Set to 1, otherwise set to 0; The global depth consistency scores of the candidate poses are compared, and the group with the highest score is selected as the final camera pose. The three-dimensional coordinates of each comprehensive feature point in the image are output to complete the sparse point cloud reconstruction. (1.3) Dense Reconstruction The disparity value of the 3D coordinates of the same composite feature point in two images is calculated using the following formula: ； Where S is the disparity value, and These are the x-coordinates of pixels in the left and right images, respectively. Calculate the depth value for each pixel using the intrinsic parameters and parallax values ​​defined in the camera calibration: ； in, For depth value, For camera focal length, The distance between the optical centers of the two cameras; The depth values ​​are optimized using an energy minimization method to ensure global consistency. The energy function is defined as follows: ； ； ； in, Represents a pixel in a two-dimensional image. This represents the total energy function for depth value optimization, with the goal of minimizing this function to ensure that the depth estimation results simultaneously satisfy data consistency and spatial smoothness. This is a data item that measures the accuracy of the current image depth value matching the disparity. It is a smoothing term that constrains the smoothness of the depth map within a local region; , for , Grayscale value, , for , The offset of a pixel in the parallax direction. This indicates the pixel position after taking parallax into account. express for , neighborhood points; The depth map of each image is converted into a 3D point cloud, and the point clouds are then fused to eliminate redundant points. The formula for converting a depth map into a 3D point cloud is as follows: ； ； in, For three-dimensional point world coordinates, The image coordinates of the pixels. Image depth value; By using the camera's pose matrix, point clouds from different viewpoints are mapped to a unified world coordinate system. : ； ； Output the world coordinates of the 3D coordinates of all integrated feature points in the image to complete dense reconstruction; Step 2: Denoising the point cloud model of the lattice tower members (2.1) Ordered point cloud noise reduction processing For ordered point clouds, noise reduction is performed using the spatial gradient and mean filtering methods of the point cloud based on its specific arrangement structure. (2.2) Noise Reduction Processing of Disordered Point Clouds For unordered point cloud data, a noise identification and filtering method based on point density distribution and local geometric features is adopted; Step 3: Detection of missing members in lattice towers based on point cloud registration (3.1) Coarse registration of 3D point cloud An improved intrinsic shape feature algorithm is used for coarse registration. The specific process is as follows: First, extract the comprehensive feature points, for each pixel. The neighborhood radius is adaptively adjusted using the following formula: ； in, The neighborhood radius; yes by A spherical neighborhood with radius , , The three-dimensional coordinates of the current point and its neighboring points; After calculating the covariance matrix and eigenvalues ​​of the point neighborhood according to the method for denoising disordered point clouds in step (2.2), a significance score is obtained: ； Calculate the local threshold: ； in, It is a statistical threshold that strikes a balance between eliminating noise and preserving true features. and Here, represents the mean and standard deviation of the significance score, respectively, as shown in the following formula: ； ； reserve The points are identified, and geometric redundancy suppression is applied to the feature points in the point cloud. The significance scores of two points p and q are defined as similar if and only if |S(p)-S(q)|<δ, where δ is a preset threshold. Points that satisfy the conditions constitute a set of similar points. Calculate redundancy score ; ； in, The significance score after redundancy suppression; Keep each point The point cloud with the highest composite feature point value is assigned a significance score weight to each composite feature point. : ； ； ； in, For point curvature, The average curvature of all points; By matching and combining feature point pairs, a weighted distance function is defined: ； Minimize the weighted distance objective function to obtain the initial transformation matrix, thus completing the coarse registration: ； in, For the target point cloud The three-dimensional coordinates of the points The first point in the source cloud The three-dimensional coordinates of each point; , for Significance score weights are used to suppress redundant feature points and suppress... This is the coarse registration transformation matrix; (3.2) Fine registration of 3D point clouds Based on coarse registration, the nearest neighbor algorithm, which uses dynamic neighborhood weights and normal vector consistency constraints, is further optimized to improve registration accuracy. The specific process is as follows: using the coarse registration transformation matrix... For each point Find the nearest neighbor Minimize the objective function And update the rigid body transformation matrix. The process is iterated until convergence to complete the fine registration. The specific formula is as follows: ； ； ； in, The objective function for precise registration; It is the spatial neighborhood weight of the weighted distance. ; It is the normal vector consistency weight; , is the angular threshold of the normal vector deviation; For the target point cloud The unit normal vector at each point This is the unit normal vector of the corresponding matching point in the source point cloud; (3.3) Visualization of missing members After point cloud registration is completed, the source point cloud and the target model point cloud are compared point by point to detect missing members and visualize them.

2. The method for detecting missing members in a lattice tower based on three-dimensional reconstruction according to claim 1, characterized in that, In step (1.1), the degree of overlap is determined by the following formula: ； in, , These represent the number of combined feature points in the left and right images, respectively. This is the set of interior point matching pairs selected using the random sampling consensus algorithm.

3. The method for detecting missing members in a lattice tower based on three-dimensional reconstruction according to claim 1, characterized in that, In step (1.2), the fundamental matrix The calculation formula is as follows: ； ； in, , These are the homogeneous coordinates of the combined feature points in the two images; The intrinsic parameter matrix of the camera is defined as follows: ； in, and Corresponding to the camera at and Focal length in direction, and For the camera main point at and pixel coordinates on; The essential matrix Decomposed into the camera rotation matrix Translation vector The relative rotation and translation of the two images are obtained, and the formula is defined as follows: ； in, for The antisymmetric matrix; This yields the essential matrix. The decomposition is not unique and needs to be verified using the depth values ​​of the triangulated points to obtain the unique correct pose; assuming the projection matrices of the two images are: ； ； in, It is The identity matrix, yes The zero vector; The homogeneous coordinates of each pair of matched feature points in three-dimensional space are obtained using the triangulation formula. The formula is as follows: ； in, Homogeneous coordinates add a dimension to a three-dimensional point It is represented as a four-dimensional vector.

4. The method for detecting missing members in a lattice tower based on three-dimensional reconstruction according to claim 1, characterized in that, Step (2.1) is as follows: First, define the point cloud gradient. as follows: ； in, It is a point The three-dimensional coordinates The coordinates of the neighboring points; It is a point The neighborhood of is consistent with the range of values ​​in the densely reconstructed neighborhood; It is the weight of the spatial proximity of the current point to the neighboring points, used to measure the grayscale difference between the current point and its neighbors; the calculation formula is as follows: ； in, It is a parameter sensitive to gradient changes; On point The gradient exceeds the global mean gradient. Add twice the standard deviation When a point is identified as a noise point, the formula is as follows: ； ； ； in, It represents the total number of point clouds. It is a point The magnitude of the gradient, , , It is a point The gradient in the x, y, and z directions; For non-noise points, optimize them using the mean of their neighborhood points: ； in, It is a point The optimized three-dimensional coordinates.

5. The method for detecting missing members in a lattice tower based on three-dimensional reconstruction according to claim 1, characterized in that, Step (2.2) is as follows: Based on the global statistics of point density and standard deviation Define the density threshold for points: ； ； ； in, The optimal rigid body transformation parameters include both translation and rotation. For the target point cloud The three-dimensional coordinates of the points The first point in the source cloud The three-dimensional coordinates of each point; These are the weighting coefficients; when Less than the threshold ( When judging the point For noise points; where For point Local neighborhood density, The global density mean. To remove the global density standard deviation; After removing noise points, to ensure the geometric consistency of the remaining point cloud, the points are... neighborhood Calculate the covariance matrix: ； in, Neighboring points The three-dimensional coordinates The index of the current center point. For neighborhood point index, It is the average value of the neighborhood points; eigenvalues ​​of covariance matrix For point The local geometric distribution characteristics correspond to the three basic structural types of the point neighborhood: linear, planar, and spherical; when The value is When any one of them is more than 10 times the value of the others, the point distribution has a linear structure; when and for When the number of points is more than 5 times the size of the point distribution, the point distribution becomes a planar structure. When the pairwise difference is within 0.1, the point distribution is spherical; based on the geometric structure information of the points, the following characteristic indices are defined: ； ； when Determine if the point is a surface point; otherwise, discard it. This point is retained as a linear structure point.

6. The method for detecting missing members in a lattice tower based on three-dimensional reconstruction according to claim 1, characterized in that, Step (3.3) is as follows: Calculate the residual distance for each point: ； Standardize the residuals: ； in, and These are the residual mean and standard deviation, respectively. when When determining if the current point is a missing point, where, To determine the standard value for the integrity of the members, color coding is used to visualize the missing members by changing the color of the missing points, with green representing complete areas and red representing missing areas.

Images