Automatic three-dimensional reconstruction method and system based on power transmission corridor point cloud

Abstract

The invention discloses an automatic three-dimensional reconstruction method and system based on power transmission corridor point cloud. The method comprises the following steps: acquiring data and exporting point cloud data; noise points, outliers and ground points are removed from the acquired point cloud data of the power transmission corridor, and down-sampling is performed on the point cloud with a large data volume; performing clustering segmentation on the preprocessed point cloud of the power transmission corridor, positioning the position of a tower, determining a point cloud range of a power line, and removing point clouds of other objects on the ground; aiming at the structural characteristics of a power line and an insulator in the power transmission line, extracting point clouds of the power line, the insulator and a tower by adopting a geometric constraint principal component analysis (PCA) algorithm; performing automatic geometric model reconstruction on the tower, insulator and power line point clouds according to different reconstruction rules to generate each module grid model; and transforming the grid model of each module, restoring the grid model to the original position of the point cloud, and generating a power transmission line digital twin model conforming to the real size. According to the method, automatic segmentation, classification and real-time reconstruction can be accurately carried out on the point cloud of the power transmission corridor.

Description

Technical Field

[0001] The present invention relates to an automated three-dimensional reconstruction method and system, and in particular to an automated three-dimensional reconstruction method and system based on a point cloud of a power transmission corridor. Background Art

[0002] Transmission equipment structures are becoming increasingly complex, and they are constantly exposed to factors such as strong winds, frost, and intense sunlight. Manual inspections are no longer sufficient to support high-quality development. With the advancement of the intelligent transformation of power systems, unmanned inspections have become the primary means of power operation and maintenance. Scanning transmission corridors with robots or drones equipped with lidar equipment generates massive amounts of high-precision point clouds, providing precise spatial information support for intelligent inspections. Digital twin technology is then used to construct virtual three-dimensional models of power grid equipment, achieving precise mapping between physical entities and virtual models. This approach not only integrates the transmission network into the digital space, but also enhances the real-time nature of operation and maintenance and the level of automation of detection.

[0003] In recent years, the application of digital twin models based on real-world transmission lines has gained significant attention. Currently, various approaches exist, including manual modeling, parametric modeling, and point cloud modeling. Traditional manual modeling is costly and inefficient; existing parametric modeling suffers from issues such as low parameter accuracy and incomplete data on some lines, leading to model distortion. Point cloud-based modeling, on the other hand, is derived from real-world point cloud data collected from transmission corridors, resulting in higher fidelity and broad prospects for practical application.

[0004] However, it remains a challenging problem to automatically segment the transmission corridor point cloud into point clouds of towers, conductors, insulator strings, ground and its ground objects, reconstruct the geometric model of each module in real time, and then synchronously load the transmission corridor model into the digital twin platform. Summary of the Invention

[0005] Purpose of the invention: The purpose of the present invention is to provide an automated three-dimensional reconstruction method and system based on transmission corridor point cloud, so as to realize automatic segmentation, classification and real-time reconstruction of transmission corridor point cloud.

[0006] Technical solution: The present invention comprises the following steps:

[0007] S1. Data collection: Use a drone equipped with a lidar to scan the transmission corridor in a certain area and export point cloud data;

[0008] S2. Data preprocessing: Remove noise points, outliers, and ground points from the collected transmission corridor point cloud data, and perform downsampling on point clouds with large data volumes;

[0009] S3. Transmission line point cloud positioning: Cluster and segment the pre-processed transmission corridor point cloud, locate the tower position, determine the power line point cloud range, and eliminate the point cloud of other ground objects;

[0010] S4. Transmission line point cloud segmentation: Based on the structural characteristics of power lines and insulators in the transmission line, the principal component analysis algorithm (PCA) with geometric constraints is used to extract the point clouds of power lines, insulators and towers;

[0011] S5. Automatic reconstruction of transmission line point clouds: Automatically reconstruct the geometric models of towers, insulators, and power lines using different reconstruction rules to generate mesh models of each module.

[0012] S6. Generation of digital twin model of transmission line: Transform the mesh model of each module and restore it to the original position of the point cloud to generate a digital twin model of the transmission line that conforms to the actual size.

[0013] The transmission line point cloud positioning includes:

[0014] Point cloud clustering and segmentation: The Euclidean clustering algorithm is used to cluster the transmission corridor point cloud based on the distance and density between point clouds;

[0015] Extracting the transmission line point cloud: By calculating the height mean of each point cloud cluster after clustering processing, the ground object point cloud and the transmission line point are separated, and the transmission line point cloud is screened out.

[0016] The transmission line point cloud segmentation specifically includes:

[0017] The principal component analysis algorithm PCA based on angle constraint is used to segment the insulator string and power line point cloud: starting from the z-axis perpendicular to the plane, the transmission line point cloud is traversed in sequence, and the angle between the line formed by the current point cloud and its local area is set as θ, and the angle threshold of the vertical structure insulator string line is set [θ i-min ,θ i-max ], the line angle threshold of the horizontal structure is [θ w-min ,θ w-max ], determine the category by judging the θ range value of the current point cloud;

[0018] Tower point cloud extraction: After being processed by the angle-constrained PCA algorithm, the remaining points after segmentation are the tower point cloud.

[0019] The automatic reconstruction of the transmission line point cloud specifically includes:

[0020] Insulator string reconstruction based on template matching: Using a template position matching reconstruction method, the transmission line point cloud is segmented and the positions of the two end points of each insulator string point cloud are calculated. The insulator string model in the template library is scaled and rotated, and the two ends are placed at the insulator string endpoint point cloud positions to achieve reconstruction of the insulator string model.

[0021] Power line point cloud reconstruction based on 3D point cloud fitting: Power line point cloud is obtained after rough segmentation using the angle-constrained PCA algorithm;

[0022] Automatic classification and model reconstruction of tower point clouds based on deep learning: A fully automatic classification and reconstruction algorithm of tower point clouds based on deep learning is used to achieve realistic model reconstruction of the overall structure of the tower point cloud.

[0023] The power line point cloud reconstruction based on three-dimensional point cloud fitting specifically includes:

[0024] The conductors and drainage lines are finely segmented using the point cloud Euclidean clustering algorithm. The search radius of the local point cloud range is adjusted, and each group of power lines is segmented with set density and connectivity to obtain point clouds of multiple power lines split from a power line. By finely extracting the power lines, the point clouds of all individual power lines are obtained.

[0025] For each power line, a three-dimensional fitting reconstruction method is used to fit the power line point cloud to a curve.

[0026] The automatic classification and model reconstruction of tower point clouds based on deep learning specifically includes:

[0027] Normalization of the tower point cloud coordinates: First, the segmented tower point cloud coordinates are normalized to the bounding box coordinate space of [0,1], and the point cloud scale transformation matrix is ​​S; the point cloud is sliced ​​based on the height value of the z-axis, and the coordinates of the four external vertices are calculated on the minimum slice of the z-axis. The plane formed by the coordinates of three of the vertices is used as the basis to transform it to the XOY plane to eliminate the tilt angle. The transformation matrix is ​​recorded as T1; a quadrilateral is constructed by the four vertices, and its two sides are aligned to the x-axis and y-axis to eliminate the deflection angle. The transformation matrix is ​​recorded as T2;

[0028] Tower point cloud projection: After the tower point cloud is standardized, the point cloud is projected onto the XOZ plane to generate a standard cross-sectional view of the tower;

[0029] Deep learning-based tower image classification and key point detection: Generate 2D projection images of various tower point clouds through the aforementioned steps, classify them, and annotate the key point pixel locations of each type as a dataset. After data enhancement, generate a tower type classification and key point detection dataset.

[0030] Mapping key points to 3D skeleton points: When the 2D key points are mapped back to 3D, the corresponding point cloud is a point cloud strip running through the Y axis at the (x,z) position. At both ends of the point cloud strip, the maximum and minimum point coordinates of the point cloud strip on the Y axis are calculated to obtain the skeleton point coordinates on both sides of the tower point cloud.

[0031] Automatic reconstruction of tower classification: After the external contour of the triangular pyramid structure is determined, its internal details are generated by regularization. For the most specialized structure of each type of tower, the connection method of its skeleton is customized through the external and internal key skeleton points. The connecting lines are filled with rectangular mesh patches of triangular steel to generate the final tower grid model.

[0032] When the key points are mapped to 3D skeleton points, for the internal frame, the internal tetrahedral structure is divided into cylinders with the midpoint position of the line connecting the external contour points of each face as the axis and the radius as the center point coordinates of each cluster of point clouds are extracted through Euclidean clustering, that is, the intersection of the internal frame lines on the tower body.

[0033] The ground points are removed using a cloth simulation algorithm.

[0034] The noise points and outliers are removed by statistical filtering algorithm.

[0035] An automated 3D reconstruction system based on transmission corridor point clouds, comprising:

[0036] Data acquisition module: Use a drone equipped with a lidar to scan a section of the transmission corridor, output and save the point cloud data;

[0037] Data preprocessing module: preprocesses the collected transmission corridor point cloud data;

[0038] Transmission Line Point Cloud Positioning Module: This module clusters and segments the pre-processed transmission corridor point cloud, clustering the ground object point cloud and the transmission line point cloud based on the distance and density between local point clouds. The module then locates the tower position and determines the power line point cloud range based on the difference in the average height of the point cloud clusters, eliminating other ground object point clouds.

[0039] Transmission line point cloud segmentation module: Based on the structural characteristics of power lines and insulators in transmission lines, the principal component analysis algorithm (PCA) with geometric constraints is used to extract point clouds of power lines, insulators, and towers;

[0040] Transmission line point cloud automatic reconstruction module: Automatically reconstructs the geometric models of towers, insulators, and power lines using different reconstruction rules to generate mesh models for each module;

[0041] Transmission line digital twin model generation module: transforms the mesh models of each module, restores them to the original position of the point cloud, and generates a digital twin model of the transmission line that conforms to the actual size.

[0042] Beneficial effects: The present invention has the following advantages:

[0043] (1) Geometrically constrained transmission corridor point cloud segmentation: Based on the existing PCA method for extracting linear point clouds, geometric constraints such as curvature, angle, and plane characteristics are added, enabling accurate extraction of horizontal power line point clouds and insulator string point clouds, as well as vertical suspended insulator string point clouds.

[0044] (2) Automatic classification and reconstruction of tower point clouds; the reconstruction process of the present invention does not require division. It only needs to use the structural symmetry relationship of the tower to obtain the key point positions of each type of tower. The three-dimensional skeleton points of the tower can be calculated through two-dimensional and three-dimensional mapping, and the skeleton connection method can be formulated according to the category to achieve reconstruction. It is also simple and fast to expand the tower category later. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flow chart showing the overall layout of the present invention;

[0046] Figure 2 This is a processing flow chart of the transmission line point cloud segmentation part of the present invention;

[0047] Figure 3 Schematic diagram of the tower point cloud coordinates before and after normalization processing of the present invention;

[0048] Figure 4 This is a processing flow chart of the tower point cloud automatic classification and model reconstruction algorithm based on deep learning of the present invention;

[0049] Figure 5 This is a schematic diagram of the 2D processing module results of the tower point cloud of the present invention;

[0050] Figure 6 Schematic diagram of extracting the internal frame point cloud coordinates of the tower point cloud according to the present invention;

[0051] Figure 7 This is a schematic diagram of the 3D processing module results of the tower point cloud of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below with reference to the accompanying drawings.

[0053] Example 1

[0054] like Figure 1As shown, the automatic three-dimensional reconstruction method based on the transmission corridor point cloud of the present invention relies on the real collected point cloud data to effectively improve the reconstruction speed and realism of the digital twin model of the transmission line. A drone equipped with a laser radar device is used to scan a section of the transmission corridor to obtain high-precision dense point cloud data of the scene; first, the point cloud data is preprocessed to remove noise points, outliers and ground points; the point cloud of other objects on the ground and the point cloud of the transmission line are segmented by point cloud clustering, and the position of the tower is located by the point cloud height to extract the point cloud of the transmission line; then the point cloud of the transmission line is segmented and divided into towers, insulator strings, and power lines (the power lines specifically include conductors, ground wires, and drainage lines), and the intersection coordinates of each module are calculated; finally, the point cloud of each module is reconstructed in three dimensions, and the grid model is connected into a whole through the intersection coordinates of each part to obtain a digital twin model of the transmission line, which can be used for real-time inspection, three-dimensional visualization and transmission project planning of the transmission line. Specifically comprising the following steps:

[0055] S1. Data collection: In this embodiment, a DJI M300 drone equipped with a laser radar is used to scan a 35KV distribution line corridor in a certain place, and the point cloud data is output and saved.

[0056] S2. Data preprocessing: For the collected transmission corridor point cloud data, there is a problem of excessive data volume leading to high computational complexity and low processing efficiency. In order to improve the efficiency and accuracy of subsequent process processing, the data is first preprocessed. Specifically, the following steps are included:

[0057] S2.1. Remove ground points. Since there are a large number of ground point clouds in the collected point cloud data, the cloth simulation algorithm is first used to remove ground points. This algorithm has a good removal effect on flat surfaces and relatively gentle slopes.

[0058] S2.2. Remove noise points and outliers. Remove outliers and noise points from the collected point cloud data using a statistical filtering algorithm.

[0059] S2.3. Point cloud downsampling: Use voxel grid filtering algorithm to downsample the point cloud data.

[0060] S3. Transmission line point cloud positioning: Cluster and segment the pre-processed transmission corridor point cloud, cluster the ground object point cloud and the transmission line point cloud based on the distance and density between local point clouds, and locate and extract the tower and power line point clouds based on the difference in the average height of the point cloud clusters. This specifically includes the following steps:

[0061] S3.1. Point cloud clustering and segmentation: The Euclidean clustering algorithm is used to cluster the transmission corridor point cloud based on the distance and density between point clouds.

[0062] S3.2. Extract the transmission line point cloud; calculate the average height value of all point clouds in each point cloud cluster after clustering processing, sort the point cloud clusters by their height values, and filter out the transmission line point cloud.

[0063] S4. Transmission line point cloud segmentation: Based on the distribution characteristics of power lines and insulator strings in the transmission line, the principal component analysis algorithm with geometric constraints is used to extract the power lines, insulator strings and tower point clouds in the transmission line. The transmission line point cloud segmentation process is as follows: Figure 2 The specific steps include:

[0064] S4.1. Principal Component Analysis (PCA) algorithm based on angle constraints to segment insulator strings and power line point clouds

[0065] After completing step S3, the generated transmission line point cloud exhibits linear features as a whole. This paper uses the angle-constrained PCA algorithm to segment the linearly distributed insulator string and conductor point clouds. The specific steps are as follows:

[0066] Taking the z-axis perpendicular to the plane as the reference direction, traverse the transmission line point cloud data point by point;

[0067] For the currently traversed point cloud, construct its local area and calculate the angle θ between the line formed by the point cloud and the local area and the z-axis;

[0068] Two types of angle thresholds are preset:

[0069] The vertical structure insulator string line angle threshold is [θ i-min ,θ i-max ];

[0070] The horizontal structure line angle threshold is [θ w-min ,θ w-max ];

[0071] By judging the threshold range of the current point cloud angle θ, its category is determined: if θ∈[θ i-min ,θ i-max ], then the point cloud is determined to belong to the vertical structure insulator string; if θ∈[θ w-min ,θ w-max ], then the point cloud is determined to belong to a horizontal structural line (conductor, etc.).

[0072] S4.2. Pole tower point cloud extraction: After being processed by the angle-constrained PCA algorithm, the remaining points after segmentation are the pole tower point cloud.

[0073] S5. Automatic reconstruction of the transmission line point cloud. The transmission line point cloud is divided into towers, insulator strings, and power lines. Because the reconstruction rules for curve point clouds are consistent, specific power lines such as conductors, ground wires, and drain wires can be reconstructed using the same algorithm. Towers, insulator strings, and power lines are each modeled using a different reconstruction algorithm. This includes the following steps:

[0074] S5.1. Insulator string reconstruction based on template matching: Using the template position matching reconstruction method, the end point positions of each insulator string point cloud can be calculated after the transmission line point cloud is segmented. The insulator string model in the template library is scaled and rotated, and its two ends are placed at the end point cloud positions of the insulator string to achieve reconstruction of the insulator string model.

[0075] S5.2. Power line point cloud reconstruction based on 3D point cloud fitting: A power line point cloud is obtained after rough segmentation using the angle-constrained PCA algorithm. Ground wires are single, while drain wires and conductors are typically multi-split. Therefore, a single power line point cloud must first be extracted from each set of power line point clouds.

[0076] The power line reconstruction in step S5.2 includes the following steps:

[0077] S5.2.1. Use the point cloud Euclidean clustering algorithm to finely segment the conductors and drainage lines. Adjust the search radius of the local point cloud range and segment each group of power lines at a certain density and connectivity. This yields point clouds of multiple power lines split from a single power line. By finely extracting the power lines, we obtain point clouds of all individual power lines.

[0078] S5.2.2. Due to the effect of gravity, power lines hang in the air in a curved shape. For each power line, a curve is fitted to the power line point cloud using a three-dimensional fitting reconstruction method.

[0079] S5.3. Automatic classification and model reconstruction of tower point clouds based on deep learning: This patent uses a fully automatic classification and reconstruction algorithm for tower point clouds based on deep learning to achieve realistic model reconstruction of the overall structure of the tower point cloud. The processing flow of the automatic classification and model reconstruction algorithm for tower point clouds based on deep learning is as follows: Figure 4 The specific process of the algorithm is as follows:

[0080] S5.3.1. Standardization of tower point cloud coordinates. Transmission lines are spread over various terrains, so towers have tilt angles relative to the ground during construction. Since transmission lines have directional trends, towers act as turning points of the lines. Therefore, the tower point cloud data scanned by drones have different lateral deflection angles, such as Figure 3As shown, after standardization, the coordinates of the tower point cloud can be unified, which facilitates the template processing of the same type of tower point cloud. In order to reconstruct the tower point cloud into a template, the tower point cloud coordinates must first be standardized. The actual scales of the towers are different. First, the segmented tower point cloud coordinates are normalized to the bounding box coordinate space of [0,1]. The point cloud scale transformation matrix is ​​S; by slicing the point cloud based on the height value of the z-axis, the coordinates of the four external vertices are calculated on the minimum slice of the z-axis. Based on the plane formed by the coordinates of three of the vertices, it is transformed to the XOY plane to eliminate the tilt angle. The transformation matrix is ​​recorded as T1; a quadrilateral is constructed using the four vertices, and its two sides are aligned to the x-axis and y-axis to eliminate the deflection angle. The transformation matrix is ​​recorded as T2. Through the above transformation process, tower point clouds of any size and deflection can be standardized to the same coordinate system.

[0081] S5.3.2, tower point cloud projection: After the input tower point cloud is standardized, the point cloud is projected onto the XOZ plane to generate a standard cross-sectional view of the tower, such as Figure 5 (a) and 5(b).

[0082] S5.3.3. Pole tower image classification and key point detection based on deep learning; Generate two-dimensional projection images of various types of pole tower point clouds through steps S5.3.1-S5.3.2, classify them, and label the key point pixel positions of each type as a dataset. After data enhancement, generate tower type classification and key point detection datasets. Use the ResNet convolutional neural network for image classification training, and use the Keypoint R-CNN network model to train the calibration box and key points. Generate a pre-trained model of the model under the pole tower dataset, input the pole tower two-dimensional image, use the model to classify the tower type, and infer the key point pixel coordinates on the image according to the training model specific to each type of pole tower, such as Figure 5 (c) shown.

[0083] S5.3.4, key points are mapped to 3D skeleton points; when a 3D point cloud is projected into 2D space, the Y-axis information is lost. Accordingly, when a 2D key point is mapped back to 3D space, the corresponding point cloud is a point cloud strip running through the Y-axis at the (x, z) position, such as Figure 7 As shown. At both ends of the point cloud strip, by calculating the maximum and minimum point coordinates of the point cloud strip on the Y axis, the skeleton point coordinates on both sides of the tower point cloud are obtained. In order to avoid the influence of outliers, the midpoint of 10% of the point cloud at the endpoints is taken as the skeleton point cloud coordinate of the tower. For the internal frame, the internal tetrahedral structure is divided into a cylinder with a radius of the midpoint position of the line connecting the external contour points of each face and the center point coordinates of each cluster of point clouds are extracted through Euclidean clustering, that is, the intersection of the internal frame lines on the tower body, as shown Figure 6 shown.

[0084] S5.3.5. Automatic reconstruction of tower classification. After determining the external contours of the triangular pyramid structure, its internal details are generated through regularization. The differences between tower types are mostly reflected in the tower head (the structure of the part that mounts power lines varies significantly). For each type of tower's most specialized structure, the skeleton connection method is customized by using key external and internal skeleton points. After determining the skeleton points and skeleton connection method for each type of tower, the connecting lines are filled with rectangular mesh patches of triangular steel to generate the final tower mesh model.

[0085] S6. Generate a digital twin model of the transmission line. In the previous steps, the tower point cloud is transformed multiple times to unify the reconstruction process. After the tower model is generated, an inverse transformation is performed to restore the 3D model to the original position of the tower point cloud, matching the position of the insulator string and power line model, and generating a complete digital twin model of the transmission line.

[0086] Example 2

[0087] An automated 3D reconstruction system based on transmission corridor point clouds, comprising:

[0088] Data acquisition module: Use a drone equipped with a lidar to scan a section of the transmission corridor, and output and save the point cloud data.

[0089] Data preprocessing module: For the collected transmission corridor point cloud data, there is a problem of high computational complexity and low processing efficiency due to the large amount of data. In order to improve the efficiency and accuracy of subsequent process processing, the data is first preprocessed. Specifically, it includes:

[0090] Remove ground points; Among the massive transmission corridor point clouds, the data volume of ground and aboveground non-transmission line point clouds accounts for a large proportion. However, this part of the point cloud is useless for transmission line reconstruction, so the cloth simulation algorithm is first used to remove the ground point cloud.

[0091] Remove noise points and outliers. Due to factors such as environmental dust and other impurities, slight vibrations of objects, and measurement errors of equipment, the collected point cloud data may contain outliers and noise points. Although the definitions of noise points and outliers are different, they are both point clouds that are separated from the object surface and are removed together through statistical filtering algorithms during processing.

[0092] Point cloud downsampling: Multi-line LiDAR equipment collects a large amount of dense point cloud data from the transmission corridor, which requires a large amount of computational processing. To reduce point cloud redundancy and improve processing efficiency, a voxel grid filtering algorithm is used to downsample the point cloud data.

[0093] The transmission line point cloud positioning module clusters and segments the pre-processed transmission corridor point cloud, clusters the ground object point cloud and the transmission line point cloud by the distance and density between local point clouds, locates the tower position and determines the power line point cloud range by the difference in the average height of the point cloud clusters, and eliminates other ground object point clouds. Specifically, it includes:

[0094] Point cloud clustering and segmentation: the Euclidean clustering algorithm is used to cluster the transmission corridor point cloud based on the distance and density between point clouds;

[0095] Extract the transmission line point cloud; by calculating the average height value of all point clouds in each point cloud cluster after clustering processing, the point cloud of objects on the ground and the transmission line points can be separated, and the transmission line point cloud can be screened out.

[0096] Transmission line point cloud segmentation module: Based on the distribution characteristics of power lines and insulator strings in transmission lines, this patent designs a geometrically constrained principal component analysis (PCA) algorithm to extract point clouds of power lines, insulator strings, and towers in transmission lines. After extracting the point clouds of each module, point clouds are reconstructed separately. Specifically, it includes:

[0097] The principal component analysis (PCA) algorithm based on angle constraints segments the insulator string and power line point clouds. Transmission line point clouds consist of conductors running through all transmission equipment, so the point cloud is generally linear. This characteristic allows the PCA algorithm to better extract the main direction of the point cloud data, that is, along the direction of the transmission line, making the algorithm more effective at extracting conductors than similar algorithms. Building on this, the present invention adds angle constraints to the PCA processing, which can divide the point cloud collection into horizontal and vertical structures when extracting linear points. Utilizing this characteristic, the improved PCA processing algorithm can not only extract horizontal conductors, ground wires, and drain wires, but also effectively extract vertical insulator string structural points.

[0098] The linear insulator strings and conductor points are segmented using the angle-constrained PCA algorithm. Starting from the z-axis perpendicular to the plane, the point cloud of the transmission line is traversed in sequence. The angle between the line formed by the current point cloud and its local area is set as θ, and the angle threshold of the vertical structure insulator string line is set [θ i-min ,θ i-max ], the line angle threshold of the horizontal structure is [θ w-min ,θ w-max ], and determine the category by judging the θ range value of the current point cloud.

[0099] Furthermore, insulator strings vary depending on the tower type. Linear towers all feature suspended insulator strings, and the insulator string point cloud can be directly extracted using an angle-constrained PCA algorithm. In tension towers, however, both suspended and tension insulator strings coexist. In a tension insulator string, the conductors are connected to the flow lines through the insulator strings. At these connection points, the curvature of the wires undergoes a sudden change due to the intersection of multiple lines. The curvature of the point cloud is calculated to determine whether the current point cloud belongs to an intersection. Above the intersection is the insulator string line, and below it is the flow line. Among all horizontal structure point clouds, the ground wire is located at the location with the highest height value and is a single line.

[0100] Pole tower point cloud extraction; after processing with the angle-constrained PCA algorithm, the remaining points after segmentation are the tower point cloud. This method can now segment the transmission corridor point cloud into ground points, above-ground non-transmission line points, tower points, insulator string points, and power line points.

[0101] Automatic point cloud reconstruction module for transmission lines; transmission line point clouds are divided into towers, insulator strings, and power lines. Because the reconstruction rules for curve point clouds are consistent, specific power lines such as conductors, ground wires, and drain wires can be reconstructed using the same algorithm. Towers, insulator strings, and power lines are each modeled using different reconstruction algorithms. Specifically, this includes:

[0102] Insulator string reconstruction based on template matching; using the template position matching reconstruction method, the end point positions of each insulator string point cloud can be calculated after the transmission line point cloud is segmented. The insulator string model in the template library is scaled and rotated, and its two ends are placed at the end point cloud positions of the insulator string to achieve reconstruction of the insulator string model.

[0103] Power line point cloud reconstruction is based on 3D point cloud fitting. A power line point cloud is obtained after rough segmentation using the angle-constrained PCA algorithm. Ground wires are single, while drain wires and conductors are typically multi-split. Therefore, a single power line point cloud must first be extracted from each set of power line point clouds.

[0104] The point cloud Euclidean clustering algorithm is used to finely segment the conductors and drainage lines. The search radius of the local point cloud is adjusted, and each group of power lines is segmented with a certain density and connectivity. This generates point clouds of multiple power lines split from a single power line. By finely extracting the power lines, the point clouds of all individual power lines are obtained.

[0105] Due to the effect of gravity, power lines hang in the air in a curved shape. For each power line, a three-dimensional fitting reconstruction method is used to fit the curve of the power line point cloud. Automatic classification and model reconstruction of tower point clouds based on deep learning; there are many types of transmission line towers, with different external frame shapes and complex internal details. General point cloud surface reconstruction algorithms such as Poisson reconstruction and triangulation are difficult to calculate and construct normal vectors of a common plane between point clouds due to the special "frame line" structure of the tower point cloud, resulting in a lack of surface features in the tower point cloud. Therefore, the use of general reconstruction algorithms has extremely poor results. This patent explores a fully automatic classification and reconstruction algorithm for tower point clouds based on deep learning. Utilizing the symmetry of the tower structure, the tower point cloud coordinates are corrected and projected to generate a two-dimensional image. The tower image is classified into tower types and the coordinates of key points are extracted. It is then mapped back to three-dimensional space to obtain the tower external skeleton point cloud. Skeleton points are extracted by category and the outer contour of the tower is constructed. At the same time, the coordinates of the internal structure points are extracted according to the skeleton points and internal structure rules to achieve realistic model reconstruction of the overall structure of the tower point cloud. The specific process of the algorithm is described as follows:

[0106] Standardizing the coordinates of tower point clouds. Transmission lines are located across a variety of terrains, so towers are constructed with tilt angles relative to the ground. Because transmission lines have directional curves, towers act as turning points for these lines, resulting in varying lateral deflection angles in the tower point cloud data captured by drones. To reconstruct tower point clouds into templates, the tower point cloud coordinates must first be standardized. Given the varying actual tower scales, the segmented tower point cloud coordinates are first normalized to the [0,1] bounding box coordinate space. The point cloud scaling matrix is ​​S. The point cloud is sliced ​​based on the z-axis height. The coordinates of the four outer vertices are calculated on the slice with the minimum z-axis value. The plane formed by three of these vertex coordinates is then transformed to the XOY plane to eliminate the tilt angle. This transformation matrix is ​​denoted as T1. A quadrilateral is constructed from the four vertices, with two of its sides aligned to the x- and y-axes to eliminate the deflection angle. This transformation matrix is ​​denoted as T2. Through this transformation process, tower point clouds of any size and deflection can be normalized to the same coordinate system.

[0107] Tower point cloud projection: After the input tower point cloud is standardized, the point cloud is projected onto the XOZ plane to generate a standard cross-sectional view of the tower.

[0108] Deep learning-based tower image classification and keypoint detection: The above steps generate 2D projection images of various tower point cloud types, which are then classified and labeled with keypoint pixel locations for each type. After data augmentation, a tower type classification and keypoint detection dataset is generated. Image classification is trained using a ResNet convolutional neural network, while bounding box and keypoint detection are trained using a Keypoint R-CNN network. A pre-trained model for the tower dataset is generated. This model is then used to classify a 2D tower image and infer the pixel coordinates of keypoints in the image using the trained model specific to each tower type.

[0109] The key points are mapped to 3D skeleton points; in process B, the three-dimensional point cloud is projected into two-dimensional space, and the Y-axis information is lost. Accordingly, when the two-dimensional key points are mapped back to three-dimensional space, the corresponding point cloud is a point cloud strip running through the Y-axis at the (x, z) position. At both ends of the point cloud strip, the maximum and minimum point coordinates of the point cloud strip on the Y-axis are calculated, which are the coordinates of the skeleton points on both sides of the tower point cloud. In order to avoid the influence of outliers, 10% of the point clouds at the endpoints are taken as the midpoint. For the internal frame, the internal tetrahedral structure is divided into cylinders with a radius of the midpoint position of the line connecting the outer contour points of each face as the axis, and the center point coordinates of each cluster of point clouds are extracted by Euclidean clustering.

[0110] Pole tower classification is automatically reconstructed. After the external contour of the triangular pyramid structure is determined, its internal details are generated through regularization. The differences between tower types are mostly reflected in the tower head (the structure of the part that mounts power lines varies greatly). For each type of tower's most specialized structure, the skeleton connection method is customized by using key external and internal skeleton points. After the skeleton points and skeleton connection method of each tower type are determined, the connecting lines are filled with rectangular mesh patches of triangular steel to generate the final tower mesh model.

[0111] Transmission line digital twin model generation module; in the aforementioned module, the tower point cloud is transformed multiple times to unify the reconstruction process, and then an inverse transformation is performed after the tower model is generated to restore the three-dimensional model to the original position of the tower point cloud, matching the position of the insulator string and power line model to generate a complete digital twin model of the transmission line.

Claims

1. An automated 3D reconstruction method based on transmission corridor point cloud, characterized in that: The following steps are involved: S1. Data collection: Use a drone equipped with a lidar to scan the transmission corridor in a certain area and export point cloud data; S2. Data preprocessing: Remove noise points, outliers, and ground points from the collected transmission corridor point cloud data, and perform downsampling on point clouds with large data volumes; S3. Transmission line point cloud positioning: Cluster and segment the pre-processed transmission corridor point cloud, locate the tower position, determine the power line point cloud range, and eliminate the point cloud of other ground objects; S4. Transmission line point cloud segmentation: Based on the structural characteristics of power lines and insulators in the transmission line, the principal component analysis algorithm (PCA) with geometric constraints is used to extract the point clouds of power lines, insulators and towers; S5. Automatic reconstruction of transmission line point clouds: Automatically reconstruct the geometric models of towers, insulators, and power lines using different reconstruction rules to generate mesh models of each module. S6. Generation of digital twin model of transmission line: Transform the mesh model of each module and restore it to the original position of the point cloud to generate a digital twin model of the transmission line that conforms to the actual size.

2. The automatic 3D reconstruction method based on transmission corridor point cloud according to claim 1, characterized in that: The transmission line point cloud positioning includes: Point cloud clustering and segmentation: The Euclidean clustering algorithm is used to cluster the transmission corridor point cloud based on the distance and density between point clouds; Extracting the transmission line point cloud: By calculating the height mean of each point cloud cluster after clustering processing, the ground object point cloud and the transmission line point are separated, and the transmission line point cloud is screened out.

3. The automatic 3D reconstruction method based on transmission corridor point cloud according to claim 1, characterized in that: The transmission line point cloud segmentation specifically includes: The principal component analysis algorithm PCA based on angle constraint is used to segment the insulator string and power line point cloud: starting from the z-axis perpendicular to the plane, the transmission line point cloud is traversed in sequence, and the angle between the line formed by the current point cloud and its local area is set as θ, and the angle threshold of the vertical structure insulator string line is set [θ i-min ,θ i-max ], the line angle threshold of the horizontal structure is [θ w-min ,θ w-max ], determine the category by judging the θ range value of the current point cloud; Tower point cloud extraction: After being processed by the angle-constrained PCA algorithm, the remaining points after segmentation are the tower point cloud.

4. The automatic 3D reconstruction method based on transmission corridor point cloud according to claim 1, characterized in that: The automatic reconstruction of the transmission line point cloud specifically includes: Insulator string reconstruction based on template matching: Using a template position matching reconstruction method, the transmission line point cloud is segmented and the positions of the two end points of each insulator string point cloud are calculated. The insulator string model in the template library is scaled and rotated, and the two ends are placed at the insulator string endpoint point cloud positions to achieve reconstruction of the insulator string model. Power line point cloud reconstruction based on 3D point cloud fitting: Power line point cloud is obtained after rough segmentation using the angle-constrained PCA algorithm; Automatic classification and model reconstruction of tower point clouds based on deep learning: A fully automatic classification and reconstruction algorithm of tower point clouds based on deep learning is used to achieve realistic model reconstruction of the overall structure of the tower point cloud.

5. The automatic 3D reconstruction method based on transmission corridor point cloud according to claim 4, characterized in that: The power line point cloud reconstruction based on three-dimensional point cloud fitting specifically includes: The conductors and drainage lines are finely segmented using the point cloud Euclidean clustering algorithm. The search radius of the local point cloud range is adjusted, and each group of power lines is segmented with set density and connectivity to obtain point clouds of multiple power lines split from a power line. By finely extracting the power lines, the point clouds of all individual power lines are obtained. For each power line, a three-dimensional fitting reconstruction method is used to fit the power line point cloud to a curve.

6. The automatic 3D reconstruction method based on transmission corridor point cloud according to claim 4, characterized in that: The automatic classification and model reconstruction of tower point clouds based on deep learning specifically includes: Normalization of the tower point cloud coordinates: First, the segmented tower point cloud coordinates are normalized to the bounding box coordinate space of [0,1], and the point cloud scale transformation matrix is ​​S; the point cloud is sliced ​​based on the height value of the z-axis, and the coordinates of the four external vertices are calculated on the minimum slice of the z-axis. The plane formed by the coordinates of three of the vertices is used as the basis to transform it to the XOY plane to eliminate the tilt angle. The transformation matrix is ​​recorded as T1; a quadrilateral is constructed by the four vertices, and its two sides are aligned to the x-axis and y-axis to eliminate the deflection angle. The transformation matrix is ​​recorded as T2; Tower point cloud projection: After the tower point cloud is standardized, the point cloud is projected onto the XOZ plane to generate a standard cross-sectional view of the tower; Deep learning-based tower image classification and key point detection: Generate 2D projection images of various tower point clouds through the aforementioned steps, classify them, and annotate the key point pixel locations of each type as a dataset. After data enhancement, generate a tower type classification and key point detection dataset. Mapping key points to 3D skeleton points: When the 2D key points are mapped back to 3D, the corresponding point cloud is a point cloud strip running through the Y axis at the (x,z) position. At both ends of the point cloud strip, the maximum and minimum point coordinates of the point cloud strip on the Y axis are calculated to obtain the skeleton point coordinates on both sides of the tower point cloud. Automatic reconstruction of tower classification: After the external contour of the triangular pyramid structure is determined, its internal details are generated by regularization. For the most specialized structure of each type of tower, the connection method of its skeleton is customized through the external and internal key skeleton points. The connecting lines are filled with rectangular mesh patches of triangular steel to generate the final tower grid model.

7. The automatic 3D reconstruction method based on transmission corridor point cloud according to claim 6, characterized in that: When the key points are mapped to 3D skeleton points, for the internal frame, the internal tetrahedral structure is divided into cylinders with the midpoint position of the line connecting the external contour points of each face as the axis and the radius as the center point coordinates of each cluster of point clouds are extracted through Euclidean clustering, that is, the intersection of the internal frame lines on the tower body.

8. The automatic 3D reconstruction method based on transmission corridor point cloud according to claim 1, characterized in that: The ground points are removed using a cloth simulation algorithm.

9. The automatic 3D reconstruction method based on transmission corridor point cloud according to claim 1, characterized in that: The noise points and outliers are removed by statistical filtering algorithm.

10. An automated 3D reconstruction system based on transmission corridor point cloud, characterized in that: include: Data acquisition module: Use a drone equipped with a lidar to scan a section of the transmission corridor, output and save the point cloud data; Data preprocessing module: preprocesses the collected transmission corridor point cloud data; Transmission Line Point Cloud Positioning Module: This module clusters and segments the pre-processed transmission corridor point cloud, clustering the ground object point cloud and the transmission line point cloud based on the distance and density between local point clouds. The module then locates the tower position and determines the power line point cloud range based on the difference in the average height of the point cloud clusters, eliminating other ground object point clouds. Transmission line point cloud segmentation module: Based on the structural characteristics of power lines and insulators in transmission lines, the principal component analysis algorithm (PCA) with geometric constraints is used to extract point clouds of power lines, insulators, and towers; Transmission line point cloud automatic reconstruction module: Automatically reconstructs the geometric models of towers, insulators, and power lines using different reconstruction rules to generate mesh models for each module; Transmission line digital twin model generation module: transforms the mesh models of each module, restores them to the original position of the point cloud, and generates a digital twin model of the transmission line that conforms to the actual size.

Images