Power transmission corridor point cloud semantic segmentation method based on improved data enhancement strategy and multi-scale global feature aggregation

Abstract

The invention discloses a power transmission corridor point cloud semantic segmentation method based on an improved data enhancement strategy and multi-scale global feature aggregation. The method comprises the following steps: constructing a power transmission corridor point cloud semantic segmentation data set; enhancing the constructed semantic segmentation data set through hierarchical down-sampling and coordinate system reconstruction; constructing a power transmission corridor point cloud semantic segmentation model; training the constructed semantic segmentation model based on the enhanced data set; and the category of each point in the power transmission corridor point cloud is analyzed by the trained power transmission corridor point cloud semantic segmentation model, so that semantic segmentation is realized. According to the segmentation method, an original data set is uniformly enhanced through hierarchical down-sampling and a coordinate system, and the problems of unbalanced segmentation categories and inconsistent main direction vectors are relieved; meanwhile, a multi-scale global feature aggregation module is introduced between an encoding layer and a decoding layer, the module aggregates global features output by all encoders and inputs the global features into the decoding layer, and the receptive field of the model is enriched.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent inspection of high-voltage transmission lines, and in particular to a power transmission corridor point cloud semantic segmentation method based on an improved data enhancement strategy and multi-scale global feature aggregation. BACKGROUND

[0002] The fine inspection of high-voltage transmission lines by drones overcomes the shortcomings of traditional manual inspection, such as low efficiency and high safety risks, and has been widely applied. LiDAR (Light Detection and Ranging) is one of the common sensors carried by drones for inspection, and can obtain laser point cloud data of the power transmission corridor. Point cloud data reflects the fine 3D spatial structure of the target, and can effectively assist the digital and intelligent inspection of the power transmission corridor. However, the original point cloud data obtained by the airborne LiDAR system does not contain the semantic information of each component (ground, power transmission tower, power transmission conductor, overhead ground wire, insulator string, and jumper), and cannot be directly used for automatic detection tasks such as tree barrier analysis, sag monitoring, and tower inclination angle calculation.

[0003] Therefore, before performing automatic detection tasks using the power transmission corridor point cloud, the original point cloud needs to be semantically segmented, i.e., the class attribute of each point is obtained. The Chinese patent "A Power Laser Point Cloud Semantic Segmentation Method Based on Improved RandLA-Net" (Application No. 202411541491.0) proposes a semantic segmentation method based on voxel convolution. This method removes noise points and outliers in the original point cloud through straight-through filtering and statistical filtering, improving the data quality. However, this data enhancement strategy fails to address the class imbalance problem of the original point cloud and does not fully utilize the spatial characteristics of the power transmission corridor point cloud. In addition, in the segmentation model, the decoding layer directly inputs the output of the last encoder of the encoding layer, lacking aggregation of global features from all encoders, resulting in insufficient ability of the model to learn global features of large-scale point clouds.

[0004] In addition, the power transmission corridor slice point cloud has the following characteristics:

[0005] 1) Class imbalance: Non-power element points (ground, vegetation, and buildings) account for a very high proportion (more than 95%), while power element points (tower, conductor, ground wire, insulator string, and jumper) account for a small proportion, among which the proportion of insulator string points and jumper points is the lowest. The class imbalance of the data set can cause the model to fail to adequately focus on small sample classes (power element points) during training, resulting in poor segmentation results of the above fine structures.

[0006] 2) Main direction direction vector inconsistency: the main direction of the power transmission corridor point cloud in the XY plane is the trend of the power transmission conductor and overhead ground wire within the span, however, the direction vectors of the main directions of different slice point clouds are inconsistent. The inconsistency of the direction vectors will cause the data set to be insufficiently adaptable to the point cloud deep learning network without rotation invariance. SUMMARY

[0007] In view of the problem of insufficient accuracy of the current power transmission corridor point cloud semantic segmentation, the present application provides a power transmission corridor point cloud semantic segmentation method based on an improved data enhancement strategy and multi-scale global feature aggregation, which enhances the original data set through hierarchical downsampling and coordinate system unification, thereby alleviating the problems of unbalanced segmentation categories and inconsistent main direction direction vectors; meanwhile, a multi-scale global feature aggregation module is introduced between the encoding layer and the decoding layer, which aggregates all the global features output by the encoder and inputs the decoding layer, thereby enriching the receptive field of the model.

[0008] The technical scheme adopted by the present application is as follows:

[0009] The power transmission corridor point cloud semantic segmentation method based on the improved data enhancement strategy and multi-scale global feature aggregation comprises the following steps:

[0010] Step 1: constructing a power transmission corridor point cloud semantic segmentation data set;

[0011] Step 2: enhancing the semantic segmentation data set constructed in step 1 through hierarchical downsampling and coordinate system reconstruction;

[0012] Step 3: constructing a power transmission corridor point cloud semantic segmentation model;

[0013] Step 4: training the semantic segmentation model constructed in step 3 based on the data set enhanced in step 2;

[0014] Step 5: analyzing the categories of each point in the power transmission corridor point cloud by using the power transmission corridor point cloud semantic segmentation model trained in step 4, thereby realizing semantic segmentation.

[0015] The step 1 comprises the following steps:

[0016] S1.1: flying along the power transmission corridor by using a UAV carrying a laser radar system to scan and acquire power line point cloud data of different terrains, different voltage levels, different line numbers and different tower types.

[0017] S1.2: slicing the original point cloud by using an artificial cutting method in units of span;

[0018] The power transmission corridor point cloud data collected by the laser radar system usually contains multiple line spans. In order to perform semantic recognition on the point clouds of each span, the original point cloud is sliced by using an artificial cutting method, and the slicing is specifically performed as follows:Figure 1 as shown.

[0019] S1.3: For each slice point cloud, the point cloud is labeled as ground (including vegetation, houses, and low-voltage distribution lines), power transmission towers, power transmission lines, overhead ground wires, insulator strings, and drainage lines 6 classes in a manual manner in the CloudCompare software; as shown in the specific Figure 2

[0020] S1.4: The slice point cloud is divided into a training set and a test set in a ratio of 8:2, the training set is used to adjust and optimize the parameters of the semantic segmentation model; the test set is used to evaluate the performance of the model.

[0021] The step 2 proposes a semantic segmentation dataset enhancement method based on hierarchical downsampling and coordinate system unity; the semantic segmentation dataset constructed in step 1 is enhanced by hierarchical downsampling and coordinate system unity, which is used to alleviate the problems of class imbalance and inconsistent main direction vectors.

[0022] In step 2, the purpose of hierarchical downsampling is to eliminate the imbalance between the number of power element categories and non-power element categories in the dataset, which includes the following steps:

[0023] First, the power element points and non-power element points are roughly separated by normalized elevation filtering and region growing algorithm;

[0024] The power element points are the power transmission tower, power transmission line, overhead ground wire, insulator string and drainage line point cloud in the original point cloud. The non-power element points are the ground, vegetation, house and other point clouds in the original point cloud.

[0025] Next, for the non-power element points which account for a large proportion, voxel downsampling processing is performed with a larger voxel edge length; for the power element points which account for a small proportion, voxel downsampling processing is performed with a smaller voxel edge length.

[0026] The normalized elevation filtering is as follows:

[0027] The slice point cloud is divided into a grid with a side length of g in the XY plane, and the normalized elevation z of the points in the grid is calculated by formula (1) n :

[0028] z n =z-z min (1);

[0029] In formula (1), z is the Z coordinate of the point, and z min is the minimum value of the Z coordinate of the points in the grid. After obtaining z n of all points, the points with z n less than z threshold are regarded as ground initial separation points P​g , let z n be the maximum value of z threshold . o .

[0030] z threshold is a key parameter of the normalized height filtering algorithm, which is used to coarsely separate ground points and non-ground points. Specifically, points with normalized height z n greater than z threshold are regarded as non-ground points; points with normalized height z n less than z threshold are regarded as ground points. In the present application, z threshold is 4m.

[0031] The region growing algorithm is specifically as follows:

[0032] ①. Take the point with the maximum Z coordinate in P o as the seed point P s , and regard the seed point P s as one point in the power element coarsely separated point P l ;

[0033] ②. In P o , points with a distance less than d threshold from all points in P l are added to P l ; d threshold is a key parameter of the region growing algorithm, which is used to define the query range of neighborhood points. In the present application, d threshold is 1m.

[0034] ③. Repeat step ② until the number of points in P l no longer increases.

[0035] The voxel downsampling is specifically as follows: the point cloud is divided into spatial voxels with v as the edge length, the centroid of all points in the voxel is replaced with the rest of the points, and the label of the point is the label that appears most frequently in the voxel. Details are shown in Figure 3 .

[0036] In step 2, the purpose of coordinate system reconstruction is to solve the problem that the principal direction vectors of slice point clouds in the dataset are inconsistent in the XY plane; details are as follows:

[0037] The slice point cloud of the power transmission corridor in the dataset constructed in step 1 has obvious directional characteristics, i.e., the elevation direction of the power transmission corridor is the Z axis direction of the point cloud, and in the XY plane, the principal direction of the point cloud is the line direction of the span. Details are shown in Figure 4 . Based on the above characteristics, the coordinate system of all slice point clouds in the dataset is unified by coordinate system reconstruction.

[0038] The coordinate system reconstruction performs coordinate transformation on the slice point cloud, so that the X-axis direction of all point cloud coordinate systems is consistent with the line trend, and in addition, the origin of the point cloud coordinate system is the centroid of the slice point cloud.

[0039] Firstly, the spatial coordinate set P(n x 3) of the slice point cloud is projected to the XY plane to obtain the plane coordinate set P'(n x 2), and the covariance matrix M thereof is calculated by formula (2):

[0040]

[0041] In formula (2), p c and p i are the centroid coordinate vector and the coordinate vector of the i-th point of P' respectively; n is the number of points in the slice point cloud; and T represents an operation on a matrix.

[0042] Then, the M is subjected to eigenvalue decomposition to obtain eigenvalues λ1, λ2(λ1≥λ2) and unit eigenvectors e1, e2 corresponding thereto. In order to make the X-axis direction of the point cloud coordinate system consistent with the line trend, the spatial coordinate set P of the slice point cloud is subjected to a rotation transformation by formula (3) to obtain the coordinate set P r (n x 3) after rotation:

[0043]

[0044] In formula (3), x is the x component of e1, is the y component of e1, is the x component of e2, is the y component of e2.

[0045] In order to make the origin of the point cloud coordinate system the centroid of the slice point cloud, the P r is subjected to a translation transformation by formula (4) to obtain P t (4 x n):

[0046]

[0047] In formula (4), x i , y i , and z i are the x, y, and z coordinates of the i-th point in P r .

[0048] Finally, the fourth row of P t is removed and transposed to obtain the spatial coordinate set P c of the slice point cloud of the power transmission corridor in the reconstructed point cloud coordinate system. Specifically as follows:

[0049] The matrix P cEach row represents the coordinates of a point, where columns 1, 2, and 3 represent the x, y, and z coordinate values; P c The resulting point cloud is a point cloud reconstructed from a coordinate system, and its principal directions on the XY plane are consistent, all being the x-axis direction.

[0050] After all point clouds are reconstructed using this method, the coordinate system is unified.

[0051] In step 3, the constructed point cloud semantic segmentation model for the power transmission corridor includes an encoding layer, a global feature aggregation module, a decoding layer, and a recognition layer. A multi-scale global feature aggregation module is introduced between the encoding layer and the decoding layer. This module fuses the global features output by all encoders in the encoding layer to obtain a global feature descriptor, which is then used as the input to the decoding layer. The introduction of this module can enrich the receptive field of the model and improve the model's ability to extract global features at different scales.

[0052] The encoding layer consists of at least one encoder, which is composed of a random downsampling, neighborhood information encoding, and local feature pooling module based on an attention mechanism.

[0053] The random downsampling module performs random downsampling on the input point cloud at a fixed magnification, thereby reducing the number of points in the point cloud and lowering the point cloud resolution; the random downsampling module also performs thinning on the input point cloud at a certain magnification to reduce the number of points.

[0054] The neighborhood information encoding module aggregates any point b in the downsampled point cloud. i The local spatial and semantic information between it and its K neighboring points is shown in Equation (5):

[0055]

[0056] In equation (5): For any point b in the point cloud i The local information encoding, where K is the number of neighborhood points and C is the feature dimension; Represents a K×C dimensional tensor filled with real numbers; c i f i b i Spatial information and semantic information; b i Spatial and semantic information of neighboring points; MLP stands for Multilayer Perceptron.

[0057] The attention-based local feature pooling module adaptively acquires the weights of each K-neighbor point relative to its center point, thereby obtaining local features with stronger neighborhood representation through weighted summation. The specific implementation is as follows:

[0058] First, the adaptive weight W is calculated using equation (6):

[0059] W = softmax (2) (M△l) (6);

[0060] In equation (6): Encoding local information of the overall point cloud; M is a linear mapping; softmax (2) (·) represents the softmax normalization of the second dimension of the tensor.

[0061] The local information encoding Δl of the overall point cloud is multiplied by the adaptive weight W, and then summed in the neighborhood dimension.

[0062] Finally, enhanced local features are obtained through a multilayer perceptron. See formula (7) for details:

[0063]

[0064] The global feature aggregation module converts all local features output by the encoder into global features and aggregates them, thereby enriching the receptive field of the features. The specific implementation is as follows:

[0065] Let the local enhancement features output by each encoder be:

[0066]

[0067] In equation (8): F1, F2, ..., F L These represent the local enhancement features output by each encoder layer; L is the number of encoders; F l N represents the local enhancement features of the output of the l-th layer encoder; l C l These represent the number of feature points and the feature dimension output by the l-th layer encoder, respectively.

[0068] For the number of clusters Q l The l-th encoder and the q-th cluster center c are calculated using equation (9). l,q Corresponding sub-feature descriptor v l,q :

[0069]

[0070] In equation (9): w l,q b l,q It is a linear transformation matrix;

[0071] The global features of each encoder are v l,1 ,v l,2 …,v l,QlQ represents the sub-feature description vector corresponding to the cluster centers of each type in each encoder layer; l Indicates the number of clusters;

[0072] Equation (10) concatenates the global features of all encoders to obtain the multi-scale global feature description vector V.

[0073] V = concat(v1, v2, ..., v L (10);

[0074] In equation (10): v1, v2, ..., v L These represent the global features of each encoder layer, and concat(i) means concatenating tensors to obtain a new three-dimensional tensor from multiple two-dimensional tensors.

[0075] The decoding layer adopts a structure symmetrical to the encoding layer, and recovers the number of points and feature dimensions in the point cloud through upsampling and a multilayer perceptron, specifically as follows: Figure 5 As shown.

[0076] The recognition layer uses multiple multilayer perceptrons to convert the feature dimension of the point cloud to be consistent with the number of classification categories, as shown in Equation (11):

[0077] F cls =mlp(mlp(F d ))(11);

[0078] In equation (11), C represents the point cloud features output by the last encoder. d F represents the number of categories; cls It is a point cloud feature tensor with the same feature dimension as the number of classification categories.

[0079] Then, the predicted probabilities of all points for each category are obtained by the softmax activation function, as shown in equation (12):

[0080] F pre =softmax(F cls (12);

[0081] In equation (12), F cls The eigenvalues ​​are directly point cloud feature tensors representing the predicted probabilities of each category.

[0082] The category with the highest probability is taken as the predicted category for that point, as shown in equation (13):

[0083] C p =max{C1, C2, ..., C M} (13);

[0084] In equation (13), C1, C2, ..., C M The probability that a point is predicted to be in any category, C p The predicted category for this point;

[0085] In step 4, the training uses a weighted cross-entropy loss function and the Adam optimizer, and updates the model parameters through gradient descent to minimize the loss.

[0086] The weighted cross-entropy loss function is defined as follows:

[0087]

[0088] In equation (14), N1 is the number of sample points, M is the number of categories, and y i c is the true label of the point (0 or 1), p i c is the probability that a point is predicted to be of class c, and w is the probability that a point is predicted to be of class c. c The category weights are calculated using equation (15):

[0089]

[0090] In equation (15), N c This represents the number of points of category c in the sample.

[0091] After training for a preset number of rounds, a semantic segmentation model of the power transmission corridor point cloud is obtained.

[0092] Using weighted cross-entropy as the loss function during model training can improve the model's segmentation accuracy for small sample categories (insulator strings, jumpers).

[0093] In step 5, the semantic segmentation model of the power transmission corridor point cloud trained in step 4 is used to infer the test set, thereby obtaining the category attribute of each point in the test set. Specifically, as follows... Figure 6 As shown.

[0094] This invention discloses a semantic segmentation method for point clouds in power transmission corridors based on improved data augmentation strategies and multi-scale global feature aggregation. The technical effects are as follows:

[0095] 1) The semantic segmentation dataset constructed in step 1 of this invention covers point cloud data with multiple voltage levels, multiple tower types, multiple line cycles, and multiple ground types. The semantic segmentation model trained on this dataset is more universal in practical applications. This invention divides the original point cloud into six categories: ground, tower, conductor, ground wire, insulator string, and jumper wire, which is suitable for the fine segmentation task of point cloud in transmission corridors.

[0096] 2) Step 2 of this invention enhances the semantic segmentation dataset through a hierarchical downsampling method, solving the class imbalance problem between power element points and non-power element points, and improving the segmentation accuracy of key components of transmission lines (insulator strings, current-carrying wires). This invention unifies the local cloud coordinate system of each slice point cloud through coordinate system reconstruction, making similar point clouds have more similar local coordinate relationships. Therefore, the model can more easily learn point cloud features during training.

[0097] 3) In step 3 of this invention, a multi-scale global feature aggregation module is introduced between the encoder and decoder of the semantic segmentation model. This module can convert the local features output by each encoder into global features and aggregate them to obtain a multi-scale global feature description vector, which is used as the input of the decoder. This can enrich the receptive field of the model and avoid misclassifying irrelevant points in the local area whose feature shapes are similar to those of towers and wires.

[0098] 4) Step 4 of this invention trains the semantic segmentation model of the transmission line point cloud using the Adam optimizer and gradient descent algorithm. The learning rate can be adaptively adjusted to accelerate convergence and reduce the cost of manual parameter tuning. In addition, this method is conducive to fully exploring the complex geometric features of insulator strings and diverter wires, which can improve segmentation accuracy and model generalization ability.

[0099] 5) The model trained in step 5 of this invention can achieve end-to-end fast segmentation of point clouds in power transmission corridors; in addition, the model has lightweight characteristics, can be deployed on mobile devices, and has practical engineering application value. Attached Figure Description

[0100] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0101] Figure 1 This is a schematic diagram of a point cloud slice.

[0102] Figure 2 Diagram illustrating the classification of the original point cloud.

[0103] Figure 3 A schematic diagram of the voxel downsampling algorithm

[0104] Figure 4 Schematic diagram of the directional characteristics of point cloud slices for power transmission corridors

[0105] Figure 5 Schematic diagram of encoder and decoder structure

[0106] Figure 6 A schematic diagram of semantic segmentation of point cloud for testing.

[0107] Figure 7 This is a flowchart of the point cloud semantic segmentation method for power transmission corridors according to the present invention.

[0108] Figure 8This is a schematic diagram showing the coarse separation of power element points and non-power element points.

[0109] Figure 9 This is a graph showing the proportion of power element points and non-power element points before and after stratified downsampling of the dataset.

[0110] Figure 10(a) shows the point cloud coordinate system of the power transmission corridor (before reconstruction);

[0111] Figure 10(b) shows the point cloud coordinate system of the power transmission corridor (after reconstruction).

[0112] Figure 11 This is a diagram of a semantic segmentation model.

[0113] Figure 12 This describes the segmentation effect of the multi-scale global feature aggregation module on the test set. Detailed Implementation

[0114] A semantic segmentation method for power transmission corridor point clouds based on improved data augmentation strategies and multi-scale global feature aggregation is proposed. First, the original point cloud dataset is augmented through hierarchical downsampling and a coordinate system. Second, a semantic segmentation model for power transmission corridor point clouds with a global feature aggregation module is constructed and trained using the Adam optimizer. Finally, the trained network infers the category of each point in the original point cloud of the power transmission corridor, thereby achieving semantic segmentation. This method can provide a reference for refined semantic segmentation of large-scale power transmission corridor point clouds.

[0115] A semantic segmentation method for power transmission corridor point clouds based on improved data augmentation strategies and multi-scale global feature aggregation is described in the following process: Figure 7 As shown, it includes the following steps (1) to (5):

[0116] Step (1): Construction of the semantic segmentation dataset for point clouds of power transmission corridors;

[0117] A drone equipped with a lidar system flew along the power transmission corridor to scan and acquire point cloud data of transmission lines with different terrains, voltage levels, number of lines, and tower types. First, the original point cloud was sliced ​​manually, segmented by span. Then, for each slice, the point cloud was manually labeled using CloudCompare software into six categories: ground, transmission tower, transmission conductor, overhead ground wire, insulator string, and drop wire. Finally, the sliced ​​point cloud was divided into training and testing sets at an 8:2 ratio. The training set was used to adjust and optimize the parameters of the semantic segmentation model; the testing set was used to evaluate the model's performance.

[0118] Step (2): Dataset Augmentation:

[0119] The transmission corridor slice point clouds constructed in step (1) have the following characteristics:

[0120] 1) Class Imbalance: Non-electrical element points (ground, vegetation, buildings) account for an extremely high proportion (over 95%), while electrical element points (towers, conductors, ground wires, insulator strings, and jumpers) account for a relatively small proportion, with insulator string points and jumper points having the lowest proportions. This class imbalance in the dataset prevents the model from adequately focusing on smaller sample classes (electrical element points) during training, resulting in poor segmentation performance for the aforementioned refined structure.

[0121] 2) Inconsistent principal direction vectors: The principal directions of the power transmission corridor point cloud on the XY plane are the orientations of the transmission lines and overhead ground wires within the span. However, the principal direction vectors of different slice point clouds are inconsistent. This inconsistency in the direction vectors leads to insufficient adaptability of the dataset to point cloud deep learning networks that do not possess rotational invariance.

[0122] In view of the above characteristics of the point cloud of the power transmission corridor slice, in order to improve the model training effect in step (4), this method proposes a semantic segmentation dataset enhancement method based on hierarchical downsampling and coordinate system one.

[0123] The purpose of the hierarchical downsampling is to eliminate the imbalance between the number of power element categories and non-power element categories in the dataset. The specific implementation steps are as follows:

[0124] First, normalized elevation filtering and a region growing algorithm are used to coarsely separate power and non-power feature points. Then, for the non-power feature points that account for a larger proportion, voxel downsampling is performed with a larger voxel edge length; for the power feature points that account for a smaller proportion, voxel downsampling is performed with a smaller voxel edge length.

[0125] The specific implementation steps of the normalized elevation filtering are as follows:

[0126] The point cloud slice is divided into grids in the XY plane with side length g, and the normalized elevation z of the points within the grid is calculated by the following formula. n :

[0127] z n =zz min ;

[0128] In the formula, z is the Z-coordinate of the point, z min This is the minimum Z-coordinate of a point within the grid. This yields the z-coordinate of all points. n Then, z n Less than z threshold The point is considered as the initial separation point P on the ground. g , let z n Greater than z threshold Point P o .

[0129] The specific implementation steps of the region growing algorithm are as follows:

[0130] ①. Take P o The point with the largest Z-coordinate is the seed point P. s , will P s Considered as the coarse separation point P of the power element l One point in the middle;

[0131] ②. In P o In the middle, will be with P l The distance between all points is less than d threshold Add point P l ;

[0132] ③. Repeat step ② until P. l The number of midpoints no longer increases.

[0133] The specific implementation steps of the voxel downsampling are as follows:

[0134] Divide the point cloud into spatial voxels with side length v. Replace the other points with the centroid of a point in the voxel. The label of the point is the label of the point that appears most frequently in the voxel.

[0135] The purpose of the first coordinate system is to solve the problem of inconsistent principal direction vectors in the XY plane of the sliced ​​point cloud in the dataset. The specific implementation steps are as follows:

[0136] The transmission corridor slice point cloud in the dataset constructed in step (1) has obvious directional characteristics, namely, the elevation direction of the transmission corridor is the Z-axis direction of the point cloud, and in the XY plane, the main direction of the point cloud is the line direction within the span. Based on the above characteristics, the coordinate system of all slice point clouds in the dataset is unified by coordinate system reconstruction.

[0137] The coordinate system reconstruction refers to performing coordinate transformation on the slice point cloud so that the X-axis direction of all point cloud coordinate systems is consistent with the direction of the line. In addition, the origin of the point cloud coordinate system is the centroid of the slice point cloud.

[0138] First, project the spatial coordinate set P(n×3) of the sliced ​​point cloud onto the XY plane to obtain the planar coordinate set P'(n×2), and calculate its covariance matrix M using the following formula:

[0139]

[0140] In the formula, p c p i Let P' be the centroid coordinate vector and the i-th point coordinate vector, respectively, and n be the number of points in the slice point cloud.

[0141] Next, eigenvalue decomposition is performed on M to obtain eigenvalues ​​λ1 and λ2 (λ1 ≥ λ2) and their corresponding unit eigenvectors e1 and e2. To align the X-axis of the point cloud coordinate system with the route, a rotation transformation is performed on P using the following formula to obtain the rotated coordinate set P. r (n×3):

[0142] Let x be the x component of e1. Let e1 be the y-component. For the x-component of e2, Let y be the component of e2.

[0143] To make the origin of the coordinate system the centroid of the slice point cloud, P is calculated using the following formula. r Perform a translation transformation to obtain P t (4×n):

[0144]

[0145] In the formula: x i y i z i For P r The x, y, and z coordinates of the i-th point.

[0146] Finally, remove the P. t The fourth line is transposed to obtain the spatial coordinate set of the power transmission corridor slice point cloud in the reconstructed coordinate system.

[0147] Step (3): Semantic segmentation model construction:

[0148] The constructed point cloud semantic segmentation model for power transmission corridors includes an encoding layer, a global feature aggregation module, a decoding layer, and a recognition layer.

[0149] The encoding layer consists of at least one encoder, which is composed of random downsampling, neighborhood information encoding, and local feature pooling modules based on an attention mechanism.

[0150] The random downsampling module refers to thinning the input point cloud by a certain factor to reduce the number of points;

[0151] The neighborhood information encoding module refers to aggregating any point b in the downsampled point cloud. i The local spatial and semantic information between it and its K neighboring points is obtained as shown in the following formula:

[0152]

[0153] In the formula, For any point b in the point cloud iThe local information encoding, where K is the number of neighborhood points, C is the feature dimension, and c i f i For b i Spatial information and semantic information For b i Spatial and semantic information of neighboring points; MLP stands for Multilayer Perceptron.

[0154] The attention-based local feature pooling module adaptively obtains the weights of each K-neighbor point relative to its center point, thereby obtaining local features with stronger neighborhood representation through weighted summation. The specific implementation is as follows:

[0155] First, the adaptive weight W is calculated using the following formula:

[0156] W = softmax (2) (M△l);

[0157] In the formula: Encoding local information of the overall point cloud, M is a linear mapping, and softmax is used. (2) (·) represents the softmax normalization of the second dimension of the tensor.

[0158] The local information encoding Δl of the overall point cloud is multiplied by the adaptive weight W, then summed along the neighborhood dimension, and finally, the enhanced local features are obtained through a multilayer perceptron. See the following formula for details:

[0159]

[0160] In the formula, K represents the number of neighboring points.

[0161] The global feature aggregation module refers to converting and aggregating the local features output by all encoders into global features, thereby enriching the receptive field of the features. The specific implementation is as follows:

[0162] Let the local enhancement features of each encoder output be F1,F2…,F L These represent the local enhancement features output by each encoder layer; L is the number of encoders; F l N represents the local enhancement features of the output of the l-th layer encoder; l C l These represent the number of feature points and the feature dimension output by the l-th layer encoder, respectively.

[0163] For the number of clusters Q l The l-th encoder and the q-th cluster center c are calculated by the following formula. l,q Corresponding sub-feature descriptor v l,q :

[0164]

[0165] In the formula, w l,q b l,q Let be a linear transformation matrix.

[0166] The global feature of each encoder is v l ={v l,1 ,v l,2 …,v l,Ql}, v l,1 ,v l,2 …,v l,Ql Let V represent the sub-feature description vectors corresponding to the cluster centers of each type in the encoder layer. The global features of all encoders are concatenated by the following formula to obtain the multi-scale global feature description vector V.

[0167] V = concat(v1, v2, ..., v L );

[0168] Where: v1, v2, ..., v L These represent the global features of each encoder layer, and concat(i) means concatenating tensors to obtain a new three-dimensional tensor from multiple two-dimensional tensors.

[0169] The decoding layer adopts a structure symmetrical to the encoding layer, and recovers the number of points and feature dimensions in the point cloud through upsampling and multilayer perceptron.

[0170] The recognition layer uses multiple multilayer perceptrons to convert the feature dimension of the point cloud to be consistent with the number of classification categories. Then, the softmax activation function is used to obtain the predicted probability of each point for each category, and the category with the highest probability is taken as the predicted category of that point.

[0171] Step (4): Semantic segmentation model training:

[0172] The semantic segmentation model constructed in step (3) is trained using the dataset enhanced in step (2). Training employs a weighted cross-entropy loss function and the Adam optimizer, updating model parameters via gradient descent to minimize the loss. After training reaches a preset number of rounds, a semantic segmentation model of the power transmission corridor point cloud is obtained.

[0173] Step (5): Semantic segmentation of point cloud in power transmission corridor:

[0174] The semantic segmentation model of the power transmission corridor point cloud trained in step (4) is used to infer the test set, thereby obtaining the category attribute of each point in the test set.

[0175] Step (5) uses the model trained in step (4) to segment the original point cloud, then compares the model inference class with the real class of the power transmission corridor point cloud, and then evaluates the model performance using the average intersection-union ratio.

[0176] Specific calculation example:

[0177] The constructed semantic segmentation dataset contains point clouds of 200 spans, exhibiting differentiation in tower types, lines, terrain types, and voltage levels. Voltage levels include 110kV, 330kV, and 500kV; transmission lines include single-circuit and double-circuit lines; and tower types include typical tower types such as cat-type towers, dry-type towers, top-type towers, and goblet towers. Furthermore, the terrain of the data collection area is complex and varied, encompassing typical landforms such as valleys, hillsides, and plains. The dataset was manually labeled using CloudCompare software, categorized into six types: ground (including vegetation, buildings, and low-voltage distribution lines), transmission towers, transmission lines, overhead ground wires, insulator strings, and drain wires. The dataset was then divided into training and testing sets in an 8:2 ratio.

[0178] Next, based on the improved data augmentation strategy proposed in this invention, the dataset is augmented. First, based on normalized elevation filtering and region growing algorithms, power element points and non-power element points are coarsely separated. The results are shown in [the diagram]. Figure 8 Where the side length g of the planar grid is 1m, z thredshold d thredshold Set to 4m and 1m respectively. Figure 8 As can be seen, normalized elevation filtering and the region growing algorithm achieve coarse separation of power element points and non-power element points. Next, the coarsely separated power element points and non-power element points are thinned out using voxel side lengths of 0.1m and 1m respectively, thus achieving stratified downsampling. The proportions of the two types of points before and after stratified downsampling are shown in the figure. Figure 9 .Depend on Figure 9 As can be seen, after stratified downsampling, the proportion of power element points increased from 4% to 42.9%, which alleviated the class imbalance in the dataset. This allowed the model to focus more on the finer structures such as towers, lines, insulator strings, and drain wires during training. Subsequently, the coordinate system of each point cloud in the dataset was reconstructed so that the direction of the conductors was parallel to the X-axis, thus unifying the coordinate system, as shown in Figures 10(a) and 10(b).

[0179] Subsequently, a semantic segmentation model is constructed; the specific architecture can be found in [link to architecture]. Figure 11The model comprises an encoding layer, a decoding layer, and a recognition layer. The encoding layer consists of five encoders, each of which performs random downsampling of the point cloud at ratios of 6, 4, 4, 3, and 2. Simultaneously, a local feature pooling module (local feature enhancement module) based on an attention mechanism gradually increases the feature dimension from 8 to 512. A multi-scale global feature aggregation module extracts and aggregates the global features output by each encoder to obtain a global feature descriptor, which serves as the input to the decoding layer. The decoding layer adopts a structure symmetrical to the encoding layer, progressively performing upsampling operations to restore the resolution and feature dimension of the point cloud. Finally, a multilayer perceptron in the recognition layer converts the feature dimension of the point cloud into the number of classification categories, and then a softmax activation function is used to obtain the predicted probability of each point for each category. The category with the highest probability is taken as the predicted category of that point.

[0180] After the model is built, it is trained using the enhanced semantic segmentation dataset. The training hyperparameter settings are shown in Table 1.

[0181] Table 1 Model Training Hyperparameters

[0182]

[0183]

[0184] The trained model was used to segment the point cloud of the power transmission corridor. Table 2 shows the impact of different data augmentation strategies on the segmentation performance of the test set. It is evident that without hierarchical downsampling of the dataset, the model's segmentation performance is poor, with an average cross-union ratio (CUN) of 79.2%, and CUNs for insulator strings and jumpers being only 44.7% and 54.8%, respectively. This is because the proportion of power element points in the dataset is relatively low at this stage, and the model focuses more on fitting non-power element points during training, resulting in lower segmentation accuracy for fine structures. After hierarchical downsampling of the dataset, the CUNs for insulators and jumpers are significantly improved, reaching 73.6% and 95.1%, respectively. Finally, after adding coordinate system reconstruction in the dataset augmentation step, the average CUN of the test set is further improved.

[0185] Table 2 Comparison of Data Augmentation Strategies

[0186]

[0187] Note: "√" indicates that the corresponding data augmentation method was used.

[0188] Figure 12The impact of the multi-scale global feature aggregation module on the segmentation effect of the test set is demonstrated. It can be seen that when the model does not include this module, some low-voltage distribution lines around the transmission lines have similar features to the transmission towers and transmission conductors in a local range, which leads to misclassification of the towers and conductors. After introducing the multi-scale global feature aggregation module, the receptive field of the model is enriched, and it can learn the features of the distribution lines in a larger range, thus enabling the point cloud to be correctly segmented.

Claims

1. A semantic segmentation method for point clouds in power transmission corridors based on improved data augmentation strategies and multi-scale global feature aggregation; characterized in that... Includes the following steps: Step 1: Construct a semantic segmentation dataset of point clouds for power transmission corridors; Step 2: Enhance the semantic segmentation dataset constructed in Step 1 by hierarchical downsampling and coordinate system reconstruction; Step 3: Construct a semantic segmentation model of the point cloud of the power transmission corridor; Step 4: Train the semantic segmentation model constructed in Step 3 based on the dataset enhanced in Step 2; Step 5: Analyze the category of each point in the power transmission corridor point cloud using the semantic segmentation model trained in Step 4, thereby achieving semantic segmentation.

2. The semantic segmentation method for point clouds of power transmission corridors based on improved data augmentation strategies and multi-scale global feature aggregation as described in claim 1, characterized in that: Step 1 includes the following steps: S1.1: By using a drone equipped with a lidar system to fly and scan along the power transmission corridor, point cloud data of power transmission lines with different terrains, voltage levels, number of lines, and tower types can be obtained. S1.2: The original point cloud is sliced ​​manually, using the distance as the unit; S1.3: For each slice point cloud, the point cloud is manually labeled into 6 categories in CloudCompare software: ground, transmission tower, transmission line, overhead ground wire, insulator string and drain line. S1.4: Divide the sliced ​​point cloud into a training set and a test set in an 8:2 ratio. The training set is used to adjust and optimize the parameters of the semantic segmentation model; the test set is used to evaluate the performance of the model.

3. The semantic segmentation method for point clouds of power transmission corridors based on improved data augmentation strategies and multi-scale global feature aggregation as described in claim 2, characterized in that: In step 2, the purpose of stratified downsampling is to eliminate the imbalance between the number of power element categories and non-power element categories in the dataset. This specifically includes the following steps: First, normalized elevation filtering and region growing algorithms are used to coarsely separate power element points from non-power element points. Power element points are the point clouds of transmission towers, transmission lines, overhead ground wires, insulator strings and diversion lines in the original point cloud; non-power element points are the point clouds of ground, vegetation and buildings in the original point cloud. Next, for non-power element points that account for a larger proportion, voxel downsampling is performed with a larger voxel edge length; for power element points that account for a smaller proportion, voxel downsampling is performed with a smaller voxel edge length.

4. The semantic segmentation method for power transmission corridor point clouds based on improved data augmentation strategies and multi-scale global feature aggregation as described in claim 3, characterized in that: The normalized elevation filtering is specifically as follows: The slice point cloud is divided into grids in the XY plane with side length g, and the normalized elevation z of the points in the grid is calculated by equation (1). n : With n =zz min (1); In equation (1): z is the Z-coordinate of the point, z min This represents the minimum Z-coordinate of a point within the grid. Get the z-axis of all points n Then, z n Less than z threshold The point is considered as the initial separation point P on the ground. g , let z n Greater than z threshold Point P o ; z threshold The normalized elevation filtering algorithm is a key parameter used to coarsely separate ground points from non-ground points. Specifically, it involves setting the normalized elevation z... n Greater than z threshold Points are considered non-ground points; normalized elevation z n Less than z threshold The point is considered a ground point.

5. The semantic segmentation method for point clouds of power transmission corridors based on improved data augmentation strategies and multi-scale global feature aggregation as described in claim 3, characterized in that: The specific region growing algorithm is as follows: ①. Take P o The point with the largest Z-coordinate is the seed point P. s Seed point P s Considered as the coarse separation point P of the power element l One point in the middle; ②. In P o In the middle, will be with P l The distance between all points is less than d threshold Add point P l ;d threshold These are key parameters for the region growing algorithm, used to define the query range of neighboring points; ③. Repeat step ② until P. l The number of midpoints no longer increases.

6. The semantic segmentation method for power transmission corridor point clouds based on improved data augmentation strategies and multi-scale global feature aggregation as described in claim 3, characterized in that: The voxel downsampling specifically involves dividing the point cloud into spatial voxels with side length v, replacing the other points with the centroid of all points in the voxel, and labeling the point with the label that appears most frequently among all points in the voxel.

7. The semantic segmentation method for point clouds of power transmission corridors based on improved data augmentation strategies and multi-scale global feature aggregation as described in claim 6, characterized in that: Step 2 is as follows: The transmission corridor slice point cloud in the dataset has obvious directional characteristics, namely, the elevation direction of the transmission corridor is the Z-axis direction of the point cloud, and in the XY plane, the main direction of the point cloud is the line direction within the span; the coordinate system of all slice point clouds in the dataset is unified by coordinate system reconstruction. Coordinate system reconstruction transforms the sliced ​​point cloud so that the X-axis direction of all point cloud coordinate systems is consistent with the direction of the line. In addition, the origin of the point cloud coordinate system is the centroid of the sliced ​​point cloud. First, project the spatial coordinate set P(n×3) of the sliced ​​point cloud onto the XY plane to obtain the planar coordinate set P'(n×2), and calculate its covariance matrix M using equation (2): In equation (2): p c p i Let P' be the centroid coordinate vector and the i-th point coordinate vector, respectively; n is the number of points in the slice point cloud; T represents the operation performed on the matrix. Next, eigenvalues ​​λ1 and λ2 (λ1 ≥ λ2) and their corresponding unit eigenvectors e1 and e2 are obtained from M. In order to make the X-axis direction of the point cloud coordinate system consistent with the direction of the line, the spatial coordinate set P of the sliced ​​point cloud is rotated according to equation (3) to obtain the rotated coordinate set P. r (n×3): In formula (3): Let x be the x component of e1. Let e1 be the y-component. For the x-component of e2, Let e2 be the y-component; To make the origin of the point cloud coordinate system the centroid of the sliced ​​point cloud, P is adjusted using equation (4). r Perform a translation transformation to obtain P t (4×n): In equation (4): x i y i z i P respectively r The x, y, and z coordinates of the i-th point in the array; Finally, remove the P. t The fourth line is transposed to obtain the spatial coordinate set P of the power transmission corridor slice point cloud in the reconstructed point cloud coordinate system. c .

8. The semantic segmentation method for point clouds of power transmission corridors based on improved data augmentation strategies and multi-scale global feature aggregation as described in claim 7, characterized in that: In step 3, the constructed point cloud semantic segmentation model of the power transmission corridor includes an encoding layer, a global feature aggregation module, a decoding layer, and a recognition layer; the encoding layer consists of at least one encoder, which is composed of a random downsampling, neighborhood information encoding, and local feature pooling module based on an attention mechanism; The random downsampling module performs random downsampling on the input point cloud at a fixed magnification, thereby reducing the number of points in the point cloud and lowering the point cloud resolution; The neighborhood information encoding module aggregates any point b in the downsampled point cloud. i The local spatial and semantic information between it and its K neighboring points is shown in Equation (5): In equation (5): For any point b in the point cloud i The local information encoding, where K is the number of neighborhood points and C is the feature dimension; Represents a K×C dimensional tensor filled with real numbers; c i f i b i Spatial information and semantic information; b i Spatial and semantic information of neighboring points; MLP stands for Multilayer Perceptron; The local feature pooling module based on the attention mechanism adaptively obtains the weights of each K-neighbor point with respect to its center point, thereby obtaining local features with stronger neighborhood representation through weighted summation. The specific implementation is as follows: First, the adaptive weight W is calculated using equation (6): W=softmax (2) (M△l) (6); In equation (6): Encoding local information of the overall point cloud; M is a linear mapping; softmax (2) (·) represents the softmax normalization of the second dimension of the tensor; The local information encoding Δl of the overall point cloud is multiplied by the adaptive weight W, and then summed in the neighborhood dimension. Finally, enhanced local features are obtained through a multilayer perceptron. See formula (7) for details:

9. The semantic segmentation method for point clouds of power transmission corridors based on improved data augmentation strategies and multi-scale global feature aggregation as described in claim 8, characterized in that: The global feature aggregation module converts all local features output by the encoder into global features and aggregates them, thereby enriching the receptive field of the features. The specific implementation is as follows: Let the local enhancement features output by each encoder be: In equation (8): F1, F2, ..., F L These represent the local enhancement features output by each encoder layer; L is the number of encoders; F l N represents the local enhancement features of the output of the l-th layer encoder; l C l These represent the number of feature points and the feature dimension output by the l-th layer encoder, respectively. For the number of clusters Q l The l-th encoder and the q-th cluster center c are calculated using equation (9). l,q Corresponding sub-feature descriptor v l,q : In equation (9): w l,q b l,q It is a linear transformation matrix; The global features of each encoder are Q represents the sub-feature description vector corresponding to the cluster centers of each type in each encoder layer; l Indicates the number of clusters; Equation (10) concatenates the global features of all encoders to obtain the multi-scale global feature description vector V. V=concat(v1,v2,...,v L ) (10); In equation (10): v1, v2, ..., v L These represent the global features of each encoder layer, and concat(i) means concatenating tensors to obtain a new three-dimensional tensor from multiple two-dimensional tensors.

10. The semantic segmentation method for point clouds of power transmission corridors based on improved data augmentation strategies and multi-scale global feature aggregation as described in claim 9, characterized in that: The recognition layer uses multiple multilayer perceptrons to convert the feature dimension of the point cloud to be consistent with the number of classification categories, as shown in Equation (11): F cls =mlp(mlp(F d )) (11); In equation (11), C represents the point cloud features output by the last encoder. d F represents the number of categories; cls A point cloud feature tensor with the same feature dimension as the number of classification categories; Then, the predicted probabilities of all points for each category are obtained by the softmax activation function, as shown in equation (12): F pre =softmax(F cls ) (12); In equation (12), F cls The eigenvalues ​​are directly the point cloud feature tensors representing the predicted probabilities of each category; The category with the highest probability is taken as the predicted category for that point, as shown in equation (13): C p =max{C1,C2,...,C M } (13); In equation (13), C1, C2, ..., C M The probability that a point is predicted to be in any category, C p The predicted category for this point.

Images