Tower structure defect identification and three-dimensional space positioning method based on multi-source fusion

Abstract

The invention discloses a tower structure defect identification and three-dimensional space positioning method based on multi-source fusion. The method comprises the following steps: constructing multi-dimensional acquisition data; unifying multi-dimensional data to the same coordinate system; establishing an enhanced data set based on the multi-dimensional data after unifying the coordinate system; constructing an independent single tower point cloud model, and carrying out reverse modeling to form a vectorized single tower three-dimensional DXF format model; detecting and analyzing the difference between the vectorized single tower three-dimensional DXF format model and the manually designed tower three-dimensional DXF format model by using a projection dimension reduction comparison method; and outputting the spatial position of the tower structure difference found in the matching process, and performing mapping association with the enhanced data set to realize defect identification and positioning of the tower structure. The method breaks through the subjectivity of a traditional method and the one-sidedness of single-source data, constructs a full-process automation system of data fusion, reverse modeling, intelligent comparison and visual positioning, and achieves the precise diagnosis of the three-dimensional deviation before the operation of the tower.

Description

Technical Field

[0001] The present invention relates to the technical field of target recognition and positioning, and specifically to a method for identifying tower structure defects and three-dimensional space positioning based on multi-source fusion. Background Technique

[0002] With the rapid development of the UHV power transmission network and the smart grid, the efficient identification and accurate three-dimensional positioning of tower structure defects have become the core challenges for ensuring the reliable operation of the power system. However, traditional detection means and the existing technical system face multiple bottlenecks under complex working conditions. Currently, the detection is still mainly dominated by manual inspection, which has defects such as low efficiency (the efficiency of conventional inspection is less than 3 towers per day), high danger, and large missed detection rate (it is impossible to perform three-dimensional tower structure detection). Although multi-sensor collaborative acquisition can improve the integrity of information, the spatio-temporal alignment of heterogeneous data and the mining of feature complementarity are still technical difficulties. Since the current identification based solely on images cannot effectively identify the defects of tower structure characteristics and lacks an adaptive fusion algorithm for tower structure characteristics, it seriously affects the quality of defect identification. The development trend of the industry's technology focuses on multi-sensor fusion, in-depth application of artificial intelligence, and high-precision three-dimensional reconstruction. It is urgent to break through key technologies such as deep fusion of multi-source data, highly robust defect identification models, and centimeter-level three-dimensional positioning, and build an intelligent solution for the entire "detection-identification-positioning" chain. Summary of the Invention

[0003] Aiming at the deficiencies of the existing technology, the present invention provides a method for identifying tower structure defects and three-dimensional space positioning based on multi-source fusion, aiming to overcome technical bottlenecks through multi-source data fusion and intelligent algorithms, promote the upgrade of power facility operation and maintenance towards precision and unmanned operation, and provide core support for the safe operation of the UHV power grid.

[0004] To achieve the above object, the present invention provides the following technical solution: A method for identifying tower structure defects and three-dimensional space positioning based on multi-source fusion, characterized by including the following steps: Step S1: Use a drone equipped with sensors to perform an all-round scan of the tower to construct multi-dimensional acquisition data, and the multi-dimensional acquisition data includes tower point cloud, tower visible light image, and tower panoramic image; Step S2: Remove the scattered points and isolated points in the tower point cloud to simplify the tower point cloud, and unify the simplified tower point cloud, tower visible light image, and tower panoramic image into the same coordinate system; Step S3: Perform spherical projection distortion removal on the tower panoramic image after unifying the coordinate system, calculate the mapping relationship between the spherical longitude and latitude coordinates and the pixel coordinates of the image after distortion removal; take panoramic images of the four sides of the tower, divide each panoramic image into pixel blocks, and store the pixel coordinates of the pixel blocks and the mapping relationship between the spherical longitude and latitude coordinates and the pixel coordinates of the image after distortion removal; Step S4: Use an adaptive clustering method to extract the features of the pole tower to form an independent single pole tower point cloud model; Step S5: Reverse model the extracted independent single pole tower point cloud model to form a vectorized single pole tower three-dimensional DXF format model; Step S6: Use the projection dimensionality reduction comparison method to detect and analyze the differences between the vectorized single pole tower three-dimensional DXF format model and the artificial designed pole tower three-dimensional DXF format model; Step S7: Output the spatial positions of the pole tower structure differences found in the matching process, and map and associate them with the mapping relationship between the pixel coordinates of the pixel block and the spherical longitude and latitude coordinates to the de-distorted image pixel coordinates in Step S3, so as to realize the defect identification and positioning of the pole tower structure.

[0005] Furthermore, the specific process of Step S3 is as follows: Step S3.1: Based on the internal parameters of the panoramic camera that collects the panoramic image of the pole tower, perform spherical projection transformation on the panoramic image of the pole tower after unifying the coordinate system to eliminate lens distortion and generate an undistorted pole tower equidistant cylindrical projection image; Among them, the internal parameters of the panoramic camera include: the focal length of the panoramic camera on the axis and the focal length on the , the principal point coordinates and the distortion coefficients , , , ; represents the quadratic term coefficient of radial distortion, represents the quartic term coefficient of radial distortion, , both represent the parameters of tangential distortion; Step S3.2: Calculate the mapping relationship from the spherical longitude and latitude coordinates to the pixel coordinates of the pole tower equidistant cylindrical projection image, expressed as: ; ; In the formula, represents the pixel coordinates of the pole tower equidistant cylindrical projection image; represents the spherical longitude and latitude coordinates; represents the principal point coordinates; and respectively represent the focal lengths of the panoramic camera on the axis and the focal length on the ; Step S3.3: Set observation points on the four sides of the pole tower, take panoramic images, divide the panoramic images of each observation point into pixel blocks, and store the mapping relationship between the pixel coordinates of the pixel blocks and the spherical longitude and latitude coordinates and the pixel coordinates of the undistorted image.

[0006] Furthermore, the specific process of step S4 is as follows: Step S4.1: Collect the point cloud of the single pole tower to be processed through the sensor carried by the UAV; Step S4.2: Separate the ground point cloud and non-ground point cloud of the single pole tower to be processed. Specifically: Generate an elevation constraint surface through a simulation filtering algorithm, combined with the elevation information of the single pole tower point cloud: ; In the formula, represents the elevation value of the elevation constraint surface at the position ; and respectively represent the second-order partial derivatives of the elevation in the , directions; represents the gradient of the elevation ; represents the preset slope threshold; Remove the ground point cloud less than the preset distance, and output the non-ground point cloud belonging to the pole tower structure, expressed as: ; In the formula, represents the th non-ground point cloud point obtained after screening; represents the elevation value of the th point of the single pole tower point cloud; represents the elevation value of the elevation constraint surface at the horizontal position of the th point of the single pole tower point cloud; Step S4.3: Calculate the average distance from the th non-ground point cloud point to other points within its own neighborhood: ; In the formula, represents 's average distance to other points within its own neighborhood; represents 's neighborhood, that is, centered on , with a radius of The set of all non-ground point cloud points within the sphere; Represents a preset neighborhood radius; Represents of The number of non-ground point cloud points within the neighborhood; Represents The th point in; Step S4.4: Based on , calculate the average distance and standard deviation of each point in the non-ground point cloud: ; ; In the formula, Represents the number of points in the non-ground point cloud; Step S4.5: Based on the average distance and standard deviation calculate the adaptive threshold, and filter the non-ground point cloud through the adaptive threshold to obtain candidate clustering point clouds similar to the tower structure characteristics; The adaptive threshold Is expressed as: ; In the formula, Represents the set coefficient used to adjust the size of the adaptive threshold ; Step S4.6: Calculate the covariance matrix of the candidate clustering point cloud and perform eigenvalue decomposition: ; ; In the formula, Represents the center point of the candidate clustering point cloud; Represents the transpose; Represents the th eigenvector of the covariance matrix; Represents the th eigenvalue of the candidate clustering point cloud; Step S4.7: When is less than the preset value, it is determined that the corresponding candidate clustering point cloud meets the eigenvalue and is retained. Otherwise, it is marked as a candidate clustering point cloud that does not meet the eigenvalue; When there are candidate clustering point clouds that do not meet the eigenvalue, adjust the value and repeat steps S4.3 - S4.5; Step S4.8: Merge all candidate clustering point clouds that meet the eigenvalue to form an independent single-tower point cloud model.

[0007] Furthermore, the specific process of step S5 is as follows: Step S5.1: Identify the feature edges of the point cloud model of an independent single-tower pole. For the edges of the point cloud model of an independent single-tower pole perform single adjacent edge screening, respectively representing two vertices of the point cloud model of an independent single-tower pole. When the number of adjacent edges of is 1 and it is located at the end of the rod, it is determined as a cross-section boundary edge. At the same time, determine the feature edges of the gusset plate of the point cloud model of an independent single-tower pole: ; In the formula, represents the angle between two normal vectors; , respectively represent the normal vectors of two adjacent edges of the point cloud model of an independent single-tower pole; represents the arccosine function; When , then it is determined that is a feature edge of the gusset plate; Step S5.2: Construct the contour of the point cloud model of an independent single-tower pole. For the point cloud at the end of the rod of the point cloud model of an independent single-tower pole perform plane fitting , and optimize the objective: ; In the formula, represents the fitted plane equation; are all coefficients of the plane equation; represents a point in with coordinates Step S5.3: Construct the axis of the rod of the point cloud model of an independent single-tower pole. Extract the axis of the rod of the point cloud model of an independent single-tower pole according to the axis equation : , represents any point on the axis of the rod; represents the midpoint of the axis of the rod; represents the distance along the axis direction; represents the direction vector of the axis; Step S5.4: Construct the cross-section of the rod of the point cloud model of an independent single-tower pole. Calculate the maximum distance between the contour vertices of the rod of the point cloud model of an independent single-tower pole in the cross-section plane: ; In the formula, Represents the maximum distance between the vertices of the bar profile within the cross-sectional plane; Represents the th vertex on the bar profile, Represents the th vertex on the bar profile, ; Calculate the minimum distance from the vertices of the bar profile to the opposite side: ; In the formula, Represents the bar edge thickness, that is, the minimum distance from the vertices of the bar profile to the opposite side; Represents the vertices of the bar profile; Step S5.5: Construct the gusset plate of the point cloud model of the independent single-tower pole; Determine the reference point according to the gusset plate characteristic contour point cloud , and calculate the polar angle of other points in except relative to , expressed as: ; ; ; In the formula, Represents the th point in , ; Represents the total number of points in; Represents the coordinate component of; Represents the standard two-dimensional arctangent function, used to calculate the angle of point relative to the reference point ; Represents the coordinate difference of on the axis; Represents the coordinate difference of on the axis; Represents the coordinate difference of on the axis; Sort in ascending order of polar angle , when the corresponding polar angles of two points in If they are the same, sort them in ascending order of the distances between these two points and the reference point ; After traversing all the points in , connect the remaining points to form the node plate of the single-tower point cloud model; Step S5.6: Perform vector modeling of the connection relationships of tower components through graph theory; Establish a component connection graph , and the vertex set consists of the rod vertices and the node plate vertices . The edge set consists of the coplanar connection constraint set and the coaxial connection constraint set ; ; ; In the formula, represents the direction vector of the rod; represents the normal vector of the node plate; respectively represent the vertices of the first rod and the second rod; respectively represent the direction vectors of the first rod and the second rod; Step S5.7: Output the vectorized 3D DXF format model of the single tower; Store the axis of the rod of the single-tower point cloud model, the cross-section of the rod of the single-tower point cloud model, and the node plate of the single-tower point cloud model as entities to establish a vectorized 3D DXF format model of the single tower.

[0008] Furthermore, the specific process of step S6 is as follows: Step S6.1: Convert the vectorized single-tower DXF model to the coordinate system consistent with the designed output tower DXF model through a transformation matrix; Take the point coordinates in the 3D DXF format model of the manually designed tower as , represents 's coordinate vector, represents the transpose; Take the point coordinates in the vectorized 3D DXF format model of the single tower as , represents 's coordinate vector; Perform transformation on the vectorized 3D DXF format model of the single tower through the rotation matrix and the translation vector so that the point coordinates of the transformed vectorized 3D DXF format model of the single tower In the same coordinate system as the point coordinates of the 3D DXF format model of the manually designed pole tower: ; Let be the point set of the 3D DXF format model of the manually designed pole tower, represent the th point in the point set of the 3D DXF format model of the manually designed pole tower, represent the number of points in the point set of the 3D DXF format model of the manually designed pole tower; Let be the point set of the 3D DXF format model of the vectorized single pole tower, represent the th point in the 3D DXF format model of the vectorized single pole tower, represent the number of points in the point set of the 3D DXF format model of the vectorized single pole tower; For each , find such that is minimized, and the objective function to be minimized is: ; In the formula, represents and the square of the Euclidean distance between the closest point in the DXF model of the vectorized single pole tower; Continuously iterate and update the rotation matrix and the translation vector until the objective function converges to the set threshold condition; Output the point set of the 3D DXF format model of the transformed vectorized single pole tower. At this time, the 3D DXF format model of the vectorized single pole tower and the 3D DXF format model of the manually designed pole tower are in the same coordinate system; Step S6.2: Project the points of the DXF model of the vectorized single pole tower and the 3D DXF format model of the manually designed pole tower onto three mutually perpendicular two-dimensional planes; Step S6.3: On the projected two-dimensional planes, respectively extract the two endpoints representing the rod segments of the DXF model of the vectorized single pole tower and the 3D DXF format model of the manually designed pole tower, and respectively extract the polygons representing the gusset plates of the DXF model of the vectorized single pole tower and the 3D DXF format model of the manually designed pole tower; Step S6.4: Perform two-dimensional geometric matching on the extracted rod segments; Compare the rod segments after projection of the 3D DXF format model of the manually designed pole tower and the 3D DXF format model of the vectorized single pole tower, and calculate the average value of the rod segment distances , and determine whether there is a match; ; In the formula, , represent both endpoints of , represent both endpoints of represents the rod segment after the projection of the 3D DXF format model of the manually designed pole tower; represents the rod segment after the projection of the 3D DXF format model of the vectorized single pole tower; When is less than the set threshold , then determine that , match; Step S6.5: Perform two-dimensional geometric matching on the extracted gusset plates; Compare the polygons after the projection of the 3D DXF format model of the manually designed pole tower and the 3D DXF format model of the vectorized single pole tower, and calculate the average value of the distances between the corresponding vertices , and determine whether there is a match; ; In the formula, , respectively represent and the th vertex of; respectively represent the polygons after the projection of the 3D DXF format model of the manually designed pole tower and the 3D DXF format model of the vectorized single pole tower; represents or the number of vertices of; When is less than the set threshold , then determine that , match.

[0009] Further, the specific process of step S7 is as follows: When a certain geometric element in the 3D DXF format model of the manually designed pole tower, i.e., a rod segment or a polygon, has no matching geometric element in the projection of the vectorized single pole tower 3D DXF format model, record the information of these missing geometric elements and their positions in the 3D DXF format model of the manually designed pole tower, and associate and correspond them with the mapping relationship between the pixel coordinates of the pixel block and the spherical longitude and latitude coordinates to the pixel coordinates of the undistorted image in step S3.3; When the difference in the size parameters of the matching geometric elements between the 3D DXF format model of the manually designed pole tower and the vectorized single pole tower 3D DXF format model exceeds the set threshold, record the information of these geometric elements with inconsistent sizes and the size difference value, and associate and correspond them with the mapping relationship between the pixel coordinates of the pixel block and the spherical longitude and latitude coordinates to the pixel coordinates of the undistorted image in step S3.3, so as to realize the defect identification and positioning of the pole tower structure.

[0010] Further, the specific process of unifying the simplified pole tower point cloud, the pole tower visible light image, and the pole tower panoramic image into the same coordinate system is as follows: Make the world coordinate system of the simplified pole tower point cloud and the coordinate system of the high-definition camera in the external parameters of the pole tower visible light image satisfy orthogonality through a rotation matrix to realize the unification of the coordinate systems; Perform a spherical projection transformation on the pole tower panoramic image to establish a mapping relationship between the pole tower panoramic image and the simplified pole tower point cloud, so as to realize the unification of the coordinate systems.

[0011] An electronic device includes a processor, a memory, and a bus. The processor and the memory are connected through the bus. Among them, the memory is used to store a set of program codes, and the processor is used to call the program codes stored in the memory to execute the method for identifying pole tower structure defects and three-dimensional space positioning based on multi-source fusion.

[0012] A non-volatile computer storage medium stores computer-executable instructions, and these computer-executable instructions execute the method for identifying pole tower structure defects and three-dimensional space positioning based on multi-source fusion.

[0013] Compared with the existing technologies, the present invention has the following beneficial effects: The present invention realizes the lightweight reverse vector construction of the pole tower through the multi-source fusion of point cloud, visible light, panoramic image, and design data, and completes the defect visualization positioning of the pole tower by comparing with the artificial pole tower model; The present invention breaks through the subjectivity of traditional manual detection and the one-sidedness of single-source data, constructs a full-process automation system of "data fusion - reverse modeling - intelligent comparison - visualization positioning", realizes the accurate diagnosis of the three-dimensional deviation before the operation of the pole tower, and provides a new technical solution for the digital delivery and full-life cycle management of transmission facilities. Description of the Drawings

[0014] Figure 1 This is the flowchart of the method of the present invention. Specific embodiments

[0015] As Figure 1 shown, the present invention provides a technical solution: a method for identifying tower structure defects and three-dimensional space positioning based on multi-source fusion, including the following steps:

[0016] Step S1: Use a drone equipped with sensors to perform an all-round scan of the tower, and construct multi-dimensional acquisition data, where the multi-dimensional acquisition data includes tower point cloud, tower visible light image, and tower panoramic image.

[0017] Among them, the sensors carried by the drone include a three-dimensional laser scanner, a 20-million-pixel high-definition camera, and a dual-fisheye lens panoramic camera.

[0018] Among them, the three-dimensional laser scanner obtains a tower point cloud with millimeter-level accuracy at an angular resolution of 0.1°; the high-definition camera synchronously captures a tower visible light image with a ground resolution of 0.5 cm; the dual-fisheye lens panoramic camera (covering 360° horizontally and ±60° in pitch) generates a tower panoramic image and performs real-time positioning through RTK technology.

[0019] Establish a sensor coordinate system transformation matrix, and calculate the space propagation delay by calibrating the position offsets of each sensor (three-dimensional laser scanner, 20-million-pixel high-definition camera, dual-fisheye lens panoramic camera) relative to the drone inertial measurement unit to ensure the consistency of the multi-dimensional acquisition data collected by the sensors in time and space, expressed as: (1); In the formula, represents the time delay, that is, the time required for the signal to travel from the sensor to the drone inertial measurement unit; represents the electromagnetic wave velocity; 、 、 respectively represent the position offsets on the 、 、 coordinate axes, that is, the position changes of the sensor relative to the drone inertial measurement unit in three-dimensional space.

[0020] Step S2: Remove the hash points and isolated points in the tower point cloud, simplify the tower point cloud, and unify the simplified tower point cloud, tower visible light image, and tower panoramic image into the same coordinate system.

[0021] Among them, the specific process of unifying the simplified tower point cloud, tower visible light image, and tower panoramic image into the same coordinate system is: The world coordinate system of the simplified tower point cloud and the coordinate system of the high-definition camera in the external parameters of the tower visible light image satisfy orthogonality through the rotation matrix to achieve coordinate system unification, which is expressed as: (2); In the formula, represents the rotation matrix, which is a 3×3 matrix used to describe the rotation operation in three-dimensional space; represents transpose of; represents the identity matrix.

[0022] Perform spherical projection transformation on the tower panoramic image to establish the mapping relationship between the tower panoramic image and the simplified tower point cloud, and achieve coordinate system unification.

[0023] Step S3: Perform spherical projection distortion removal on the tower panoramic image in the unified coordinate system, calculate the mapping relationship from the spherical longitude and latitude coordinates to the pixel coordinates of the undistorted image; capture the panoramic images of the four sides of the tower, divide each panoramic image into pixel blocks, and store the mapping relationship between the pixel coordinates of the pixel blocks and the spherical longitude and latitude coordinates to the pixel coordinates of the undistorted image.

[0024] The specific process of Step S3 is as follows:

[0025] Step S3.1: Based on the internal parameters of the panoramic camera, perform spherical projection transformation on the tower panoramic image in the unified coordinate system to eliminate lens distortion and generate an undistorted equidistant cylindrical projection image of the tower.

[0026] Among them, the internal parameters of the panoramic camera include: the focal length of the panoramic camera on the axis and the focal length on the axis, the principal point coordinates and the distortion coefficients , , ; represents the quadratic term coefficient of radial distortion, represents the quartic term coefficient of radial distortion, , respectively represent different parameters of tangential distortion, which are used to correct the geometric distortion caused by tangential distortion in the image.

[0027] Step S3.2: Calculate the mapping relationship from the spherical longitude and latitude coordinates to the pixel coordinates of the tower equidistant cylindrical projection image, which is expressed as: (3); (4); In the formula, Denote the pixel coordinates of the equally-spaced columnar projection image of the pole tower; Denote the spherical longitude and latitude coordinates; Denote the principal point coordinates; and respectively denote the focal lengths of the panoramic camera on the axis and axis .

[0028] Step S3.3: Set observation points on the four sides of the pole tower, and capture panoramic images. Cut each panoramic image of the observation points into 256×256 pixel blocks, and simultaneously store the mapping relationship between the pixel coordinates of the pixel blocks and the spherical longitude and latitude coordinates to the de-distorted image pixel coordinates.

[0029] Step S4: Use the adaptive clustering method to extract the features of the pole tower to form an independent single-pole tower point cloud model.

[0030] The specific process of Step S4 is as follows:

[0031] Step S4.1: Collect the point cloud of the single pole tower to be processed through a drone carrying a sensor (3D laser scanner).

[0032] Step S4.2: Separate the ground point cloud and the non-ground point cloud of the single pole tower to be processed. Specifically: Generate an elevation constraint surface through the simulated filtering (CSF) algorithm, combined with the elevation information of the single pole tower point cloud: (5); In the formula, Denote the elevation value of the elevation constraint surface at the position ; and respectively denote the second-order partial derivatives of the elevation in the , directions, which are used to measure the curvature of the elevation surface in the corresponding directions; Denote the gradient of the elevation ; Denote the preset slope threshold, which is used to limit the maximum slope of the generated elevation constraint surface.

[0033] Remove the ground point cloud less than the preset distance (0.5m), and output the non-ground point cloud belonging to the pole tower structure, expressed as: (6); In the formula, Denote the th non-ground point cloud point obtained after screening; Represents the point cloud of a single tower The elevation value of a point; Indicates the elevation constraint surface in the single tower point cloud The horizontal position of the point The elevation value at .

[0034] The ground point cloud occupies a large proportion, and filtering out the ground point cloud can reduce the complexity of subsequent extraction.

[0035] Step S4.3: Calculate the Non-ground point cloud points To oneself The average distance to other points in the neighborhood: (7); In the formula, express To oneself The average distance to other points in the neighborhood; express of Neighborhood, that is, is the center and the radius is The set of all non-ground point cloud points within the sphere; Indicates the preset neighborhood radius. ; express of The number of non-ground point cloud points in the neighborhood; express The points.

[0036] Step S4.4: Based on , calculate the average distance of each point in the non-ground point cloud With standard deviation : (8); (9); In the formula, Represents the number of points in the non-ground point cloud.

[0037] Step S4.5: Based on the average distance With standard deviation Calculate the adaptive threshold value, through the adaptive threshold value Filter non-ground point clouds (such as Exceed , the corresponding points are marked as noise points or abnormal points and removed), and the candidate cluster point cloud with similar structural features of the tower is obtained; adaptive threshold It is expressed as: (10); Wherein, represents the set coefficient for adjusting the adaptive threshold size. In this embodiment, set can cover about 99% of the normal distribution points.

[0038] Step S4.6: Calculate the covariance matrix of the candidate clustering point cloud and perform eigenvalue decomposition: (11); (12); Wherein, represents the center point of the candidate clustering point cloud; represents the transpose; represents the th eigenvector of the covariance matrix, ; represents the th eigenvalue of the candidate clustering point cloud, that is, the variance size of the covariance matrix in the direction of the corresponding eigenvector. In this embodiment, , , is the maximum eigenvalue, representing the variance along the tower height direction, is the intermediate eigenvalue, representing the variance in the tower width direction, is the minimum eigenvalue, representing the variance in the tower thickness direction.

[0039] Step S4.7: When is less than the preset value (0.25), it is determined that the corresponding candidate clustering point cloud conforms to the eigenvalue (tower structure feature) and is retained. Otherwise, it is marked as a candidate clustering point cloud that does not conform to the eigenvalue; when there is a candidate clustering point cloud that does not conform to the eigenvalue, adjust value and repeat steps S4.3 - S4.5.

[0040] Step S4.8: Merge all candidate clustering point clouds that conform to the eigenvalue to form an independent single - body tower point cloud model.

[0041] Step S5: Reverse - model the extracted independent single - body tower point cloud model to form a vectorized single - body tower three - dimensional DXF format model.

[0042] The specific process of step S5 is as follows:

[0043] Step S5.1: Identify the feature edges of the independent single - body tower point cloud model.

[0044] For the edges of the independent single - body tower point cloud model Perform single adjacent edge screening, respectively represent two vertices of the point cloud model of an independent single-tower pole, if the number of adjacent edges of and is located at the end of the member, it is determined as the cross-section boundary edge; at the same time, determine the characteristic edges of the gusset plate of the point cloud model of the independent single-tower pole (the edge contour line of the tower gusset plate (the component connecting different members)): (13); In the formula, represents the angle between two normal vectors; , respectively represent the normal vectors of two adjacent edges of the point cloud model of the independent single-tower pole; represents the inverse cosine function.

[0045] If , then determine as the characteristic edge of the gusset plate (the angle between the normal vectors in the connection area of the tower gusset plate and the member is usually greater than 90°).

[0046] Step S5.2: Construct the contour of the point cloud model of the independent single-tower pole.

[0047] For the point cloud at the end of the member of the point cloud model of the independent single-tower pole perform plane fitting , and optimize the objective: (14); In the formula, represents the plane equation of the fitting; are all coefficients of the plane equation; represents a point in .

[0048] Step S5.3: Construct the axis of the member of the point cloud model of the independent single-tower pole.

[0049] Extract the axis of the member of the point cloud model of the independent single-tower pole according to the axis equation : , represents any point on the axis of the member; represents the midpoint of the axis of the member; represents the distance along the axis direction; represents the direction vector of the axis.

[0050] Step S5.4: Construct the cross-section of the member of the point cloud model of the independent single-tower pole.

[0051] Calculate the maximum distance between the contour vertices of the member of the point cloud model of the independent single-tower pole in the cross-section plane: (15); Wherein, represents the maximum distance between the vertexes of the rod profile in the cross-section plane; represents the th vertex on the rod profile, represents the th vertex on the rod profile, .

[0052] Calculate the minimum distance from the vertex of the rod profile to the opposite side (the thickness of the rod edge): (16); Wherein, represents the thickness of the rod edge, that is, the minimum distance from the vertex of the rod profile to the opposite side; represents the vertex of the rod profile.

[0053] Step S5.5: Construct the gusset plate of the independent single-tower point cloud model.

[0054] Determine the reference point according to the gusset plate characteristic profile point cloud , and calculate the polar angle of other points relative to , expressed as: (17); (18); (19); Wherein, represents the th point in ; represents the total number of points in ; represents the coordinate component of ; represents the standard two-dimensional arctangent function, used to calculate the angle of point relative to the reference point ; represents the coordinate difference of on the axis; represents the coordinate difference of on the axis; represents the coordinate difference of on the axis; represents the coordinate difference of on the

[0055] According to the polar angle sort them in ascending order from small to large. If the corresponding polar angles of two points in are the same, then sort them in ascending order according to the distances between these two points and the reference point ; after traversing all points in , connect the remaining points to form the node plate of the single-pole tower point cloud model.

[0056] Step S5.6: Perform vector modeling of the connection relationship of tower components through graph theory methods.

[0057] Establish a component connection graph , the vertex set is composed of the rod vertices and the node plate vertices , and the edge set is composed of the coplanar connection constraint set and the coaxial connection constraint set ; (20); (21); In the formula, represents the direction vector of the rod; represents the normal vector of the node plate; respectively represent the vertices of the first rod and the second rod; respectively represent the direction vectors of the first rod and the second rod.

[0058] Step S5.7: Output the vectorized three-dimensional DXF format model of the single-pole tower.

[0059] Store the axis of the rod of the independent single-pole tower point cloud model, the cross-section of the rod of the independent single-pole tower point cloud model, and the node plate of the single-pole tower point cloud model as Line entities, LwpolyLine entities, and Polygon entities respectively, and establish a vectorized three-dimensional DXF format model of the single-pole tower.

[0060] Step S6: Use the projection dimensionality reduction comparison method to detect and analyze the differences between the vectorized three-dimensional DXF format model of the single-pole tower and the three-dimensional DXF format model of the manually designed tower.

[0061] The specific process of Step S6 is as follows:

[0062] Step S6.1: Convert the vectorized single-pole tower DXF model to the coordinate system consistent with the DXF model of the designed output tower through the transformation matrix.

[0063] Take the point coordinates in the three-dimensional DXF format model of the manually designed tower as , represents the coordinate vector of and

[0064] Take the point coordinates in the vectorized single-tower three-dimensional DXF format model as , represents the coordinate vector of

[0065] Transform the vectorized single-tower three-dimensional DXF format model through the rotation matrix and the translation vector so that the point coordinates of the transformed vectorized single-tower three-dimensional DXF format model are in the same coordinate system as the point coordinates of the manually designed tower three-dimensional DXF format model: (22).

[0066] Let be the point set of the manually designed tower three-dimensional DXF format model, represents the th point in the point set of the manually designed tower three-dimensional DXF format model, represents the number of points in the point set of the manually designed tower three-dimensional DXF format model.

[0067] Let be the point set of the vectorized single-tower three-dimensional DXF format model, represents the th point in the vectorized single-tower three-dimensional DXF format model, represents the number of points in the point set of the vectorized single-tower three-dimensional DXF format model.

[0068] For each , find such that is minimized, and the objective function to be minimized is: (23); In the formula, represents the square of the Euclidean distance between and the closest point

[0069] in the vectorized single-tower DXF model. and the translation vector are continuously iteratively updated until the objective function converges to a smaller value and meets the set threshold condition.

[0070] Output the point set of the vectorized single-tower three-dimensional DXF format model after transformation. At this time, the vectorized single-tower three-dimensional DXF format model and the manually designed tower three-dimensional DXF format model are in the same coordinate system.

[0071] Step S6.2: Project the points of the vectorized single-tower DXF model and the manually designed tower three-dimensional DXF format model onto three mutually perpendicular two-dimensional planes (xy plane, xz plane, and yz plane) to reflect the structural information of the tower from different angles.

[0072] Step S6.3: On the projected two-dimensional planes, respectively extract the two endpoints of the rod segments represented by the vectorized single-tower DXF model and the manually designed tower three-dimensional DXF format model, and respectively extract the polygons representing the gusset plates of the vectorized single-tower DXF model and the manually designed tower three-dimensional DXF format model.

[0073] Step S6.4: Perform two-dimensional geometric matching on the extracted rod segments.

[0074] Compare the rod segments after projection of the manually designed tower three-dimensional DXF format model and the vectorized single-tower three-dimensional DXF format model, and calculate the average value of the distances between the rod segments , and determine whether they match; (24); In the formula, , represent the two endpoints of; , represent the two endpoints of; represents the rod segment after projection of the manually designed tower three-dimensional DXF format model; represents the rod segment after projection of the vectorized single-tower three-dimensional DXF format model.

[0075] If is less than the set threshold , then these two rod segments are considered to match.

[0076] Step S6.5: Perform two-dimensional geometric matching on the extracted gusset plates.

[0077] Compare the polygons after projection of the manually designed tower three-dimensional DXF format model and the vectorized single-tower three-dimensional DXF format model, and calculate the average value of the distances between the corresponding vertices , and determine whether they match; (25); In the formula, , respectively represent and the th vertex; respectively represent the polygons after projection of the 3D DXF format model of the manually designed pole tower and the 3D DXF format model of the vectorized single pole tower; represent or the number of vertices of.

[0078] If is less than the set threshold , it is considered that these two polygons match.

[0079] Step S7: Output the spatial positions of the differences in the pole tower structure found during the matching process, and map and associate them with the mapping relationship between the pixel coordinates of the pixel block in step S3 and the spherical longitude and latitude coordinates to the pixel coordinates of the undistorted image, so as to realize the defect identification and positioning of the pole tower structure, facilitating relevant personnel to view.

[0080] The specific process of step S7 is as follows: If a certain geometric element (rod segment or polygon) in the 3D DXF format model of the manually designed pole tower has no matching geometric element in the projection of the 3D DXF format model of the vectorized single pole tower, record the information of these missing geometric elements and their positions in the 3D DXF format model of the manually designed pole tower, and associate and correspond them with the mapping relationship between the pixel coordinates of the pixel block in step S3.3 and the spherical longitude and latitude coordinates to the pixel coordinates of the undistorted image; for the geometric elements that match between the 3D DXF format model of the manually designed pole tower and the 3D DXF format model of the vectorized single pole tower, if the difference in their dimensional parameters (length, radius, area, etc.) exceeds the set threshold, record the information of these geometric elements with inconsistent dimensions and the specific dimensional difference values, and associate and correspond them with the mapping relationship between the pixel coordinates of the pixel block in step S3.3 and the spherical longitude and latitude coordinates to the pixel coordinates of the undistorted image, so as to realize the defect identification and positioning of the pole tower structure.

[0081] An electronic device, including a processor, a memory and a bus, the processor and the memory are connected through the bus, wherein, the memory is used to store a set of program codes, and the processor is used to call the program codes stored in the memory to execute the method for identifying defects in the pole tower structure and three-dimensional space positioning based on multi-source fusion.

[0082] A non-volatile computer storage medium, the computer storage medium stores computer-executable instructions, and the computer-executable instructions execute the method for identifying defects in the pole tower structure and three-dimensional space positioning based on multi-source fusion.

[0083] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying tower structure defects and three-dimensional spatial positioning based on multi-source fusion, characterized in that It includes the following steps: Step S1: Use a drone equipped with sensors to conduct a full - range scan of the pole tower, and construct multi - dimensional acquisition data, where the multi - dimensional acquisition data includes pole tower point cloud, pole tower visible - light image, and pole tower panoramic image; Step S2: Eliminate the hash points and isolated points in the pole tower point cloud, simplify the pole tower point cloud, and unify the simplified pole tower point cloud, pole tower visible - light image, and pole tower panoramic image into the same coordinate system; Step S3: Perform spherical projection de - distortion on the pole tower panoramic image in the unified coordinate system, calculate the mapping relationship from spherical longitude - latitude coordinates to pixel coordinates of the de - distorted image; Take panoramic images of the four sides of the pole tower, divide each panoramic image into pixel blocks, and store the mapping relationship between the pixel coordinates of the pixel blocks and the mapping relationship from spherical longitude - latitude coordinates to pixel coordinates of the de - distorted image; Step S4: Use an adaptive clustering method to extract features of the pole tower to form an independent single - body pole tower point cloud model; Step S5: Reverse - model the extracted independent single - body pole tower point cloud model to form a vectorized single - body pole tower three - dimensional DXF format model; Step S6: Use the projection dimensionality reduction comparison method to detect and analyze the differences between the vectorized single - body pole tower three - dimensional DXF format model and the artificial - designed pole tower three - dimensional DXF format model; Step S7: Output the spatial positions of the pole tower structure differences found during the matching process, and map - associate them with the mapping relationship between the pixel coordinates of the pixel blocks and the mapping relationship from spherical longitude - latitude coordinates to pixel coordinates of the de - distorted image in Step S3 to achieve defect identification and positioning of the pole tower structure.

2. The method for identifying tower structure defects and three-dimensional space positioning based on multi-source fusion according to claim 1, wherein: The specific process of Step S3 is as follows: Step S3.1: Based on the internal parameters of the panoramic camera for collecting the pole tower panoramic image, perform spherical projection transformation on the pole tower panoramic image in the unified coordinate system to eliminate lens distortion and generate a distortion - free pole tower equidistant cylindrical projection image; Among them, the internal parameters of the panoramic camera include: the focal length of the panoramic camera on the axis and the focal length of the axis , the principal point coordinates and the distortion coefficients , , , ; represents the quadratic term coefficient of the radial distortion, represents the quartic term coefficient of the radial distortion, , both represent the parameters of the tangential distortion; Step S3.2: Calculate the mapping relationship from spherical longitude - latitude coordinates to pixel coordinates of the pole tower equidistant cylindrical projection image, expressed as: ； ； In the formula, represents the pixel coordinates of the equally-spaced columnar projection image of the pole tower; represents the spherical longitude and latitude coordinates; represents the principal point coordinates; and respectively represent the focal lengths of the panoramic camera on the axis and axis ; Step S3.3: Set observation points on the four sides of the pole tower, take panoramic images, divide each panoramic image of each observation point into pixel blocks, and at the same time store the mapping relationship between the pixel coordinates of the pixel blocks and the mapping relationship from spherical longitude - latitude coordinates to pixel coordinates of the de - distorted image.

3. The method for identifying tower structure defects and three-dimensional space positioning based on multi-source fusion according to claim 2, characterized in that: The specific process of Step S4 is as follows: Step S4.1: Use a drone equipped with sensors to collect the point cloud of the single - body pole tower to be processed; Step S4.2: Separate the ground point cloud and non - ground point cloud of the single - body pole tower to be processed. Specifically: Generate an elevation constraint surface through a simulation filtering algorithm, combined with the elevation information of the single - body pole tower point cloud; ； In the formula, represents the elevation value of the elevation constraint surface at the position ; and respectively represent the second-order partial derivatives of the elevation in the , directions; represents the gradient of the elevation ; represents the preset slope threshold; Remove the ground point cloud with a distance less than the preset distance, and output the non - ground point cloud belonging to the pole tower structure, expressed as: ； In the formula, represents the th non-ground point cloud point obtained after screening; represents the elevation value of the th point of the single-tower pole point cloud; represents the elevation value at the horizontal position of the th point of the elevation constraint surface on the single-tower pole point cloud ; Step S4.3: Calculate the average distance from the non-ground point cloud point to other points within its own neighborhood: ； In the formula, represents the average distance to other points within its neighborhood; represents the neighborhood of , that is, the set of all non-ground point cloud points within the sphere centered at with a radius of ; represents a preset neighborhood radius; represents the number of non-ground point cloud points within the neighborhood of represents the th point in Step S4.4: Based on , calculate the average distance and standard deviation of each point of the non-ground point cloud: ； ； In the formula, represents the number of points in the non-ground point cloud; Step S4.5: Based on the average distance and the standard deviation calculate the adaptive threshold, and filter the non-ground point cloud through the adaptive threshold to obtain the candidate clustered point cloud; Adaptive threshold Expressed as: ； In the formula, represents the set coefficient used to adjust the size of the adaptive threshold ; Step S4.6: Calculate the covariance matrix of the candidate clustering point cloud And perform eigenvalue decomposition: ； ； In the formula, represents the center point of the candidate clustering point cloud; represents the transpose; represents the th eigenvector of the covariance matrix; represents the th eigenvalue of the candidate clustering point cloud; Step S4.7: When is less than the preset value, it is determined that the corresponding candidate clustered point cloud meets the eigenvalue and is retained; otherwise, it is marked as a candidate clustered point cloud that does not meet the eigenvalue. When there is a candidate clustered point cloud that does not meet the eigenvalue, adjust the value and repeat steps S4.3 - S4.5; Step S4.8: Merge all candidate clustering point clouds that meet the eigenvalue to form an independent single - body pole tower point cloud model.

4. The method for identifying tower structure defects and three-dimensional space positioning based on multi-source fusion according to claim 3, characterized in that: The specific process of Step S5 is as follows: Step S5.1: Identify the feature edges of the independent single - body pole tower point cloud model; Edges of the point cloud model of an independent single-tower pole Perform single adjacent edge screening, Respectively represent two vertices of the point cloud model of an independent single-tower pole. When The number of adjacent edges And is located at the end of the member, it is determined as the cross-section boundary edge; at the same time, determine the characteristic edges of the gusset plate of the point cloud model of the independent single-tower pole: ； Wherein, represents the angle between two normal vectors; , respectively represent the normal vectors of two adjacent sides of the point cloud model of an independent single-tower pole; represents the arccosine function; When , it is determined that is the characteristic edge of the gusset plate; Step S5.2: Construct the contour of the independent single - body pole tower point cloud model; For the point cloud of the end of the pole of the independent single-tower point cloud model perform plane fitting , and optimize the objective: ； In the formula, represents the fitted plane equation; are all coefficients of the plane equation; represents a point in, with the coordinates ; Step S5.3: Construct the axis of the pole of the independent single - body pole tower point cloud model; Extract the axis of the pole and tower point cloud model of an independent monomer according to the axis equation : , represents any point on the axis of the member; represents the midpoint of the axis of the member; represents the distance along the axis direction; represents the direction vector of the axis; Step S5.4: Construct the cross - section of the pole of the independent single - body pole tower point cloud model; Calculate the maximum distance between the rod profile vertices of the point cloud model of an independent single-tower pole in the cross-sectional plane: ； In the formula, represents the maximum spacing of the vertices of the rod profile within the cross-sectional plane; represents the th vertex on the rod profile, represents the th vertex on the rod profile, ; Calculate the minimum distance from the rod profile vertex to the opposite side: ； In the formula, represents the edge thickness of the rod, that is, the minimum distance from the vertex of the rod profile to the opposite side; represents the vertex of the rod profile; Step S5.5: Construct the gusset plate of the point cloud model of the independent single-tower pole; According to the characteristic contour point cloud of the gusset plate Determine the reference point , and calculate except for The polar angle of other points relative to is , expressed as: ； ； ； In the formula, represents the th point in ; represents the total number of points; represents the coordinate component of represents the standard two-dimensional arctangent function, which is used to calculate the angle of point relative to the reference point ; represents the coordinate difference of on the axis; represents the coordinate difference of on the axis; represents the coordinate difference of on the axis; According to the polar angle Sort in ascending order When The corresponding polar angles of two points in Are the same, then sort them in ascending order according to the distances between these two points and the reference point After traversing all points in Connect the remaining points to form the node plate of the single-tower point cloud model; Step S5.6: Perform vector modeling of the connection relationship of the tower components by the graph theory method; Establish a component connection diagram , the vertex set consists of bar vertices and gusset plate vertices , and the edge set consists of a coplanar connection constraint set and a coaxial connection constraint set ; ； ； In the formula, represents the direction vector of the rod member; represents the normal vector of the gusset plate; respectively represent the vertex of the first rod member and the vertex of the second rod member; respectively represent the direction vector of the first rod member and the direction vector of the second rod member; Step S5.7: Output the vectorized 3D DXF format model of the single-tower pole; Store the axis of the rod of the point cloud model of the independent single-tower pole, the cross-section of the rod of the point cloud model of the independent single-tower pole, and the gusset plate of the point cloud model of the single-tower pole as entities, and establish a vectorized 3D DXF format model of the single-tower pole.

5. The method for identifying tower structure defects and three-dimensional space positioning based on multi-source fusion according to claim 4, wherein: The specific process of Step S6 is as follows: Step S6.1: Convert the vectorized single-tower pole DXF model to the coordinate system consistent with the DXF model of the designed tower through the transformation matrix; Take the point coordinates in the 3D DXF format model of the manually designed pole tower as , denote the coordinate vector of denote transpose; Take the point coordinates in the vectorized 3D DXF format model of a single pole and tower as , which represents the coordinate vector of By means of a rotation matrix and a translation vector transform the vectorized 3D DXF format model of a single-tower pole so that the point coordinates of the transformed vectorized 3D DXF format model of the single-tower pole are in the same coordinate system as the point coordinates of the 3D DXF format model of the manually designed tower pole: ； Let be the point set of the 3D DXF format model of the manually designed pole tower, represent the th point in the point set of the 3D DXF format model of the manually designed pole tower, represent the number of points in the point set of the 3D DXF format model of the manually designed pole tower; Let be the point set of the vectorized single-tower three-dimensional DXF format model, represent the th point in the vectorized single-tower three-dimensional DXF format model, represent the number of points in the point set of the vectorized single-tower three-dimensional DXF format model; For each , find such that is minimized, minimizing the objective function is: ； In the formula, represents the square of the Euclidean distance between and the closest point in the vectorized single-tower DXF model; Continuously iterate and update the rotation matrix and the translation vector until the objective function converges to the set threshold condition; Output the point set of the transformed vectorized 3D DXF format model of the single-tower pole. At this time, the vectorized 3D DXF format model of the single-tower pole and the 3D DXF format model of the manually designed tower are in the same coordinate system; Step S6.2: Project the points of the vectorized single-tower pole DXF model and the 3D DXF format model of the manually designed tower onto three mutually perpendicular two-dimensional planes; Step S6.3: On the projected two-dimensional planes, respectively extract the two endpoints of the rod segments representing the vectorized single-tower pole DXF model and the 3D DXF format model of the manually designed tower, and respectively extract the polygons representing the gusset plates of the vectorized single-tower pole DXF model and the 3D DXF format model of the manually designed tower; Step S6.4: Perform two-dimensional geometric matching on the extracted rod segments; Compare the pole segments after projection of the manually designed pole tower 3D DXF format model and the vectorized single pole tower 3D DXF format model, and calculate the average value of the distances between the pole segments , and determine whether they match; ； In the formula, and represent the two end points of; and represent the two end points of; represents the rod segment after the projection of the 3D DXF format model of the manually designed pole tower; represents the rod segment after the projection of the 3D DXF format model of the vectorized single pole tower; When is less than the set threshold , it is determined that and match; Step S6.5: Perform two-dimensional geometric matching on the extracted gusset plates; Compare the polygons after projection of the 3D DXF format model of the manually designed pole tower and the 3D DXF format model of the vectorized single pole tower, and calculate the average value of the distances between the corresponding vertices , and determine whether they match; ； In the formula, , respectively represent and the th vertex; respectively represent the polygons after projection of the 3D DXF format model of the manually designed pole tower and the 3D DXF format model of the vectorized single pole tower; represents or the number of vertices; When is less than the set threshold , it is determined that and match.

6. The method for identifying tower structure defects and three-dimensional space positioning based on multi-source fusion according to claim 5, wherein: The specific process of Step S7 is as follows: When there are no matching geometric elements in the projection of the geometric elements in the 3D DXF format model of the manually designed tower in the vectorized 3D DXF format model of the single-tower pole, record the information of these missing geometric elements and their positions in the 3D DXF format model of the manually designed tower, and associate and correspond them with the mapping relationship between the pixel coordinates of the pixel block and the spherical longitude and latitude coordinates to the pixel coordinates of the undistorted image in Step S3.3; When the difference in the size parameters of the matching geometric elements between the 3D DXF format model of the manually designed tower and the vectorized 3D DXF format model of the single-tower pole exceeds the set threshold, record the information of these geometric elements with inconsistent sizes and the size difference values, and associate and correspond them with the mapping relationship between the pixel coordinates of the pixel block and the spherical longitude and latitude coordinates to the pixel coordinates of the undistorted image in Step S3.3 to realize the defect identification and positioning of the tower structure; The geometric elements are rod segments or polygons.

7. The method for identifying tower structure defects and three-dimensional space positioning based on multi-source fusion according to claim 1, wherein: The specific process of unifying the simplified tower point cloud, the tower visible light image, and the tower panoramic image into the same coordinate system is as follows: Make the world coordinate system of the simplified tower point cloud and the coordinate system of the high-definition camera in the external parameters of the tower visible light image satisfy orthogonality through the rotation matrix to realize the coordinate system unification; Perform spherical projection transformation on the tower panoramic image, establish the mapping relationship between the tower panoramic image and the simplified tower point cloud, and realize the coordinate system unification.

8. An electronic device, characterized in that, It includes a processor, a memory and a bus, and the processor and the memory are connected through the bus. Among them, the memory is used to store a set of program codes, and the processor is used to call the program codes stored in the memory to execute the method for identifying tower structure defects and three-dimensional space positioning based on multi-source fusion according to any one of claims 1-7.

9. A non-volatile computer storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions execute the method for identifying tower structure defects and three-dimensional space positioning based on multi-source fusion according to any one of claims 1-7.

Images