A power tower point cloud segmentation method and system based on a hierarchical neural network

Abstract

The application discloses a power tower point cloud segmentation method and system based on a layered neural network, and the method comprises the following steps: acquiring point cloud data of a power tower to be processed and a surrounding environment; inputting the point cloud data into a layered neural network; sampling the point cloud data by using a multi-stage set sampling layer; performing up-sampling on a local feature vector of each centroid obtained by sampling by using a multi-stage feature propagation layer, thereby completing feature learning of the point cloud data, obtaining a global feature vector of the entire point cloud, mapping the global feature vector of the entire point cloud into a category score vector by using a full connection layer, and converting the category score vector into a probability distribution by using a Softmax activation layer, so as to obtain the probability of each point in the point cloud data belonging to different categories. The application aims to better adapt to the point cloud data features of the power tower, and realizes fast, accurate and automatic segmentation of the power tower point cloud data by using the layered neural network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic processing of power facilities, and in particular to a power tower point cloud segmentation method and system based on a hierarchical neural network. BACKGROUND

[0002] The safe transmission of electric energy relies on the normal operation of power lines, and power towers are an important part of power facilities and are of great significance to the normal operation of the power system. Traditional inspection, maintenance and monitoring of power towers require a large amount of manpower and material resources, are low in efficiency, and have safety hazards. With the rapid development of artificial intelligence, using drones as image acquisition and transmission devices and applying deep learning technology to classify, segment and identify power lines and their components to achieve automatic inspection of the power system has become a popular trend in today's smart grid.

[0003] Point cloud segmentation of power towers is an image processing technology based on point cloud data, aiming to separate power towers from the surrounding environment in point cloud data, usually including information such as the shape and position of the power tower, for further analysis and processing. The key to point cloud segmentation is how to process point cloud data. Traditional methods convert point clouds into structured data such as voxels or images and then use convolutional neural networks for processing, but these methods lose point cloud information and increase computational complexity. In recent years, researchers have proposed some methods that directly process point clouds, such as PointNet, PointNet++, etc., which can better preserve point cloud information and have higher computational efficiency. In the field of power tower detection and segmentation, the PointNet network has been widely used and can effectively process point cloud data and extract feature information. However, current tower segmentation methods still have some problems, such as difficulty in accurately segmenting the details of the power tower, or dependence on specific parameters or network structures for segmentation results. SUMMARY

[0004] The technical problem to be solved by the present application: In view of the above problems of the prior art, the present application provides a power tower point cloud segmentation method and system based on a hierarchical neural network, which aims to better adapt to the characteristics of power tower point cloud data and use a hierarchical neural network to achieve fast and accurate automatic segmentation of power tower point cloud data.

[0005] To solve the above technical problems, the technical scheme adopted by the present application is:

[0006] A power tower point cloud segmentation method based on a hierarchical neural network, comprising:

[0007] S101, acquiring point cloud data of a power tower to be processed and the surrounding environment;

[0008] S102, input the point cloud data into the hierarchical neural network: sample the point cloud data by using a multi-level set sampling layer, including: learning local feature vectors of each centroid by performing local feature learning on the point cloud, selecting neighborhood points around each centroid to construct feature vectors of the neighborhood points to realize local region extraction, and performing local feature fusion on the local feature vectors of each centroid and the feature vectors of the neighborhood points of the centroid to obtain a local feature vector of each centroid; performing up-sampling on the local feature vector of each centroid obtained by sampling by using a multi-level feature propagation layer to complete feature learning on the point cloud data and obtain a global feature vector of the entire point cloud, mapping the global feature vector of the entire point cloud into a category score vector by using a fully connected layer, and then converting the category score vector into a probability distribution by using a Softmax activation layer to obtain probabilities of each point in the point cloud data belonging to different categories.

[0009] Optionally, the set sampling layer includes:

[0010] The sampling layer is configured to select a series of points as centroids of local regions from the input point cloud by using a maximum distance sampling method (MDS), including: randomly selecting two initial points as representative points according to a set initial distance value; calculating distances between each point and the representative points and selecting a point with the largest sum of distances from the representative points as the next representative point, adding the point to a set of selected representative points, and gradually expanding the set of representative points by iteratively calculating distances between each point and the representative points and selecting a point with the largest sum of distances from the representative points as the next representative point until the entire point cloud data is covered; and selecting K points around each centroid as neighborhood points of the centroid by using a cross-level aggregation method (CLA) to realize local region extraction, including: calculating a neighborhood point set of each point according to a pre-set radius list and a number of neighborhood points, selecting a certain number of neighborhood points from the neighborhood point set to splice with the center point to generate an enhanced feature representation, and finally selecting K points around each centroid as neighborhood points of the centroid by gradually increasing the size of the radius to expand the feature receptive field through iteration;

[0011] The group layer is configured to construct a local region set group by finding neighborhood points around the centroids extracted by the sampling layer by using the cross-level aggregation method (CLA), taking a point set with a size of N×(d+C) and a centroid coordinate with a size of N'×d as inputs, where the point set represents N points with d-dimensional coordinates and C-dimensional features, and the output is a point set group with a size of N'×K×(d+C'), where each group corresponds to a local region, K is a number of points near the centroid point, and the point set is composed of N' subsampled points with d-dimensional coordinates and new C' feature vectors summarizing local context;

[0012] Point net layer, used for output processing of the group layer, outputs each local region as a sampling result of the set sampling layer by a centroid and a local feature abstraction of a size of N'x(d+C') of its neighborhood.

[0013] Optionally, the step S102 of upsampling the sampling result by using the multi-level feature propagation layer refers to upsampling the sampling result by using distance-weighted linear interpolation, and a function expression thereof is as follows:

[0014]

[0015] In the above formula, f (j) (x) is the j-th derivative of the feature after j times of upsampling of the feature dimension at the position x, w i (x) is the distance weight corresponding to the i-th sampling result at the position x, f i () is the i-th sampling result feature of the feature dimension j times, d(x, x i ) is the distance between the position x and the sampling result x i , p is the exponential parameter of the distance, j is the feature dimension, k is the number of sampling results, and C is the total dimension of the feature.

[0016] Optionally, the step S102 further includes the step of training the hierarchical neural network before the step S102:

[0017] S201, acquiring point cloud data samples of a power tower to be processed and a surrounding environment of the power tower;

[0018] S202, labeling the point cloud data samples to add labels to obtain an original power tower data set;

[0019] S203, preprocessing the original power tower data set, and dividing the preprocessed power tower data set into a training set, a validation set and a test set according to a specified proportion;

[0020] S204, training the hierarchical neural network by using the training set, the validation set and the test set to obtain the trained hierarchical neural network.

[0021] Optionally, the preprocessing in step S203 on the original power tower dataset comprises: segmentation processing on the original point cloud data sample, obtaining point cloud slices of the power tower by using different angle slices, the point cloud slices of different angles of the same tower containing all tower points and natural environment points with different ranges, setting a sparse coefficient according to the point cloud data density, inputting the point cloud slices of the power tower, calculating the distance between two points in space, discarding the points within the sparse coefficient range, and the distance between the remaining points being equal to the sparse coefficient, so as to finally obtain multiple point cloud slices with different sparse degrees of the same power tower from one point cloud slice to realize data expansion, and then extracting effective information columns X, Y, Z, R, G, B and label from the data expanded point cloud data sample, and performing homogeneity normalization on each column of X, Y, Z, R, G, B, wherein (X, Y, Z) is the three-dimensional coordinates of the point, (R, G, B) is the color value of the three channels of the point, and label is the label.

[0022] Optionally, when the sparse coefficient is set, the value range of the sparse coefficient is 0.08-0.15.

[0023] Optionally, the homogeneity normalization comprises:

[0024] S301, calculate the average value (X m ,Y m ,Z m ) of the maximum value and the minimum value in the three directions of the coordinate axis according to the following formula:

[0025]

[0026] In the above formula, (X, Y, Z) are the three-dimensional coordinates of the point, X min , Y min and Z min are the minimum values of X, Y and Z in the three directions, respectively, and X max , Y max and Z max are the maximum values of X, Y and Z in the three directions, respectively.

[0027] S302, calculate the scaling factor δ according to the average value (X m ,Y m ,Z m ) of the maximum value and the minimum value in the three directions of the coordinate axis:

[0028] δ = max (X-X m ,Y-Y m ,Z-Z m ),

[0029] In the above formula, max represents the maximum value.

[0030] S303, the point cloud data of the power tower and the surrounding environment is scaled by using a scaling factor δ to the interval (-1, 1) to obtain the isotropic normalized three-dimensional coordinates.

[0031] Optionally, the function expression for scaling the point cloud data of the power tower and the surrounding environment by using a scaling factor δ to the interval (-1, 1) in step S303 is as follows:

[0032]

[0033] In the above formula, (X n ,Y n ,Z n ) is the isotropic normalized three-dimensional coordinates of the three-dimensional coordinates (X, Y, Z) of the point.

[0034] In addition, the application also provides a power tower point cloud segmentation system based on a hierarchical neural network, which comprises a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the power tower point cloud segmentation method based on the hierarchical neural network.

[0035] In addition, the application also provides a computer readable storage medium, which stores a computer program for programming or configuring a microprocessor to execute the power tower point cloud segmentation method based on the hierarchical neural network.

[0036] Compared with the prior art, the application mainly has the following advantages: the power tower point cloud segmentation method based on the hierarchical neural network comprises inputting the point cloud data into the hierarchical neural network: using a multi-level set sampling layer to sample the point cloud data, including learning the local feature vector of each centroid to obtain the local feature vector of each centroid, selecting the neighborhood points around each centroid to construct the feature vector of the neighborhood points to realize local region extraction, and performing local feature fusion on the local feature vector of each centroid and the feature vector of its neighborhood points to obtain the local feature vector of each centroid; using a multi-level feature propagation layer to up-sample the local feature vector of each centroid obtained by sampling to complete feature learning of the point cloud data and obtain the global feature vector of the entire point cloud, using a fully connected layer to map the global feature vector of the entire point cloud into a category score vector, and then using a Softmax activation layer to convert the category score vector into a probability distribution to obtain the probability of each point in the point cloud data belonging to different categories, which can greatly reduce the labor cost and workload and improve the work efficiency; the method can realize efficient and accurate segmentation of the power tower, improve the safety and reliability of the power facilities, and can be widely applied to the safety inspection, maintenance and monitoring of the power facilities, and has wide application prospect and market value. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The structural schematic diagram of the hierarchical neural network for the embodiment of the present application.

[0038] Figure 2 The training flowchart of the hierarchical neural network for the embodiment of the present application.

[0039] Figure 3 The comparison diagram of the output result, the real label and the actual point cloud scene in the embodiment of the present application. DETAILED DESCRIPTION

[0040] As shown in the figure, the power tower point cloud segmentation method based on the hierarchical neural network in the embodiment includes: Figure 1

[0041] S101, acquiring the point cloud data of the power tower to be processed and the surrounding environment;

[0042] S102, inputting the point cloud data into the hierarchical neural network: using a multi-level set sampling layer (the number can be selected according to the needs, for example, the three levels of SA1, SA2 and SA3 are included in the embodiment) to sample the point cloud data, including: learning the local feature vector of each centroid by learning the local features of the point cloud, selecting the neighborhood points around each centroid to construct the feature vector of the neighborhood points to realize local region extraction, and fusing the local feature vector of each centroid and the feature vector of its neighborhood points to obtain the local feature vector of each centroid; using a multi-level feature propagation layer (the number can be selected according to the needs, for example, the three levels of FP3, FP2 and FP1 are included in the embodiment) to up-sample the local feature vector of each centroid obtained by sampling, thereby completing the feature learning of the point cloud data to obtain the global feature vector of the entire point cloud, using a fully connected layer to map the global feature vector of the entire point cloud into a class score vector, and then using a Softmax activation layer to convert the class score vector into a probability distribution to obtain the probability of each point in the point cloud data belonging to different categories.

[0043] Referring to Figure 2 , the method in the embodiment uses the hierarchical neural network to learn the local features of the point cloud, uses the maximum interval sampling method MDS to select the centroid, thereby obtaining the local feature vector of each centroid. The cross-level aggregation method CLA is used to select K points around each centroid as its neighborhood points to realize local region extraction. The local feature vector of each centroid and the feature vector of its neighborhood points are fused to obtain its global feature vector. The hierarchical neural network is used to learn the global feature vector, thereby obtaining the global feature vector of the entire point cloud. The global feature vector is mapped to a class score vector, and then the Softmax function is used to convert it into a probability distribution to obtain the probability of each point belonging to different categories.

[0044] ​In this embodiment, the set abstraction layer (SA) includes:

[0045] The sampling layer is configured to select a series of points as the centroids of local regions from the input point cloud by using a maximum distance sampling method (MDS), including: randomly selecting two initial points as representative points according to a set initial distance value; calculating the distance between each point and the representative points:

[0046]

[0047] where d is the distance, x i is the i-th point, x a , x b is the representative point, and n is the number of all points;

[0048] and selecting the point with the largest sum of distances from the representative points as the next representative point, adding it to the set of selected representative points, and gradually expanding the set of representative points by iteratively calculating the distance between each point and the representative points and selecting the point with the largest sum of distances from the representative points as the next representative point until the entire point cloud data is covered; and using a cross-level aggregation method (CLA) to select K points around each centroid as its neighborhood points to achieve local region extraction, including: calculating the neighborhood point set of each point according to a pre-set radius list and the number of neighborhood points, selecting a certain number of neighborhood points from the neighborhood point set to splice with the center point to generate an enhanced feature representation, and gradually increasing the size of the radius to expand the feature receptive field through iteration, and finally selecting K points around each centroid as its neighborhood points.

[0049] The grouping layer is configured to use the cross-level aggregation method (CLA) to construct a local region set group by finding neighborhood points around the centroids extracted by the sampling layer, taking a point set with a size of N x (d + C) and a centroid coordinate with a size of N' x d as input, where the point set represents N points with d-dimensional coordinates and C-dimensional features, and the output is a point set group with a size of N' x K x (d + C'), where each group corresponds to a local region, K is the number of points near the centroid point, and the point set consists of N' subsampled points with d-dimensional coordinates and new C' feature vectors summarizing the local context.

[0050] The PointNet layer is configured to process the output of the grouping layer, and output each local region as a set abstraction layer output sampling result with a size of N' x (d + C') of local feature abstraction of the centroid and its neighborhood.

[0051] In this embodiment, a series of points are selected as the centroids of local regions from the input point cloud using a maximum distance sampling method MDS. Compared with other sampling methods, in the case of the same number of representative points, the maximum distance sampling method MDS can uniformly select representative points by the sampling mode of the region center points, thereby more comprehensively covering the entire point cloud, helping to extract key shape features and structural information, so that the selected points can better represent the local region and the global structure, thereby improving the accuracy of point cloud segmentation.

[0052] In this embodiment, the cross-level aggregating method CLA is used to calculate the neighborhood points of each point according to the preset radius list and the number of neighborhood points, and a certain number of neighborhood points are selected to splice with the center point to form an enhanced feature representation, and the size of the radius is gradually increased through iteration to expand the receptive field of different sizes and extract more global and abstract feature information, which helps to accurately segment the target result. Compared with other aggregation methods, the CLA method can capture features of different scales by fusing feature information of different levels and gradually increasing the size of the receptive field, so as to better understand the semantics and structure of the point cloud data and improve the accuracy and robustness of the segmentation result. The Grouping layer uses the cross-level aggregation method CLA to construct a local region set group by finding neighborhood points around the centroid extracted by the Sampling layer. The Grouping layer takes a point set with a size of N×(d+C) and a centroid coordinate with a size of N'×d as input, where the point set represents N points with d-dimensional coordinates and C-dimensional features, and the output is a point set group with a size of N'×K×(d+C'), where each group corresponds to a local region, K is the number of points near the centroid, and the point set consists of N' sub-sampled points with d-dimensional coordinates and new C' feature vectors that summarize the local context. Then, the PointNet layer is processed, and the output of each local region is a local feature abstraction with a size of N'×(d+C') from the centroid and its neighborhood. The feature extraction layer is the feature obtained by convolving and pooling the K points through a small PointNet network as the feature of the center point, which is then sent to the next hierarchical layer. In this way, the center points obtained by each layer are a subset of the center points of the previous layer, and as the number of layers increases, the number of center points decreases, but the information contained in each center point increases. In addition, the cross-level aggregation method CLA includes: first, calculating the neighborhood point set of each point according to the preset radius list and the number of neighborhood points; then, selecting a certain number of neighborhood points from the neighborhood point set to splice with the center point to generate an enhanced feature representation, which can fuse feature information of different levels; next, gradually increasing the size of the radius through iteration to expand the receptive field of the feature, the purpose is to extract more global and abstract feature information. With the increase of the radius, the range of the aggregated features gradually expands, which can capture more global context information, helps to accurately segment the target result, and better understand the shape and structure of the target.

[0053] Referring to Figure 1The local feature vector of each centroid and the feature vectors of its neighborhood points are fused to obtain a global feature vector. After the local region feature extraction is completed in the set sampling layer, the global coordinates and local coordinates output by the set sampling layer SA2 layer are sent to the feature propagation module (FP, Feature Propagation) together with the global coordinates and local coordinates output by the set sampling layer SA3 layer. An input point set with a size of N is received, and an up-sampling set with a size of N' is output, while the feature dimension remains unchanged. The hierarchical neural network is used to learn the global feature vector, thereby obtaining the global feature vector of the entire point cloud. The point cloud extracts a global feature, and then the feature propagation module uses distance-weighted linear interpolation to gradually up-sample the point cloud. The up-sampling process takes the discarded points in the down-sampling process, and assigns features according to the features of the K nearest points that are not discarded in the three-dimensional space. The farther the distance, the smaller the weight. Finally, the weight of each point is globally normalized. In step S102 of the embodiment, the multi-level feature propagation layer is used to up-sample the sampling result, which means that the sampling result is up-sampled using distance-weighted linear interpolation, and the function expression is as follows:

[0054]

[0055] In the above formula, f (j) (x) is the j-th derivative of the feature after j times of up-sampling at position x, w i (x) is the distance weight corresponding to the i-th sampling result at position x, f i () is the contribution of the i-th sampling result feature (denoting the i-th sample point x i of the j-th derivative of the function f with respect to the variable x) to the j-th derivative of the feature, d(x, x i ) is the distance between position x and the sampling result x i , p is the exponential parameter of the distance, j is the feature dimension, k is the number of sampling results, and C is the total feature dimension.

[0056] The global feature vector is mapped to a class score vector, and then the score vector is converted to a class probability distribution using the Softmax function to obtain the probability of each point belonging to different classes. The global feature vector output by the FP module is passed as input to the fully connected layer, which performs a weight linear combination on the global feature and outputs a score vector equal to the number of classes. Then, the Softmax function is used to convert it to a probability distribution to output the class to which each point belongs. Finally, the rod tower point cloud is segmented according to the class.

[0057] As shown in Figure 2 , the embodiment step S102 further includes the step of training the hierarchical neural network:

[0058] S201, acquire point cloud data samples of the power tower to be processed and the surrounding environment;

[0059] S202, label the point cloud data samples to add labels, to obtain an original power tower dataset;

[0060] S203, preprocess the original power tower dataset, and divide the preprocessed power tower dataset into a training set, a validation set and a test set according to a specified ratio. The ratio can be configured according to actual needs. For example, in this embodiment, the preprocessed power tower dataset is divided into a training set, a validation set and a test set according to a ratio of 8:1:1.

[0061] S204, train the hierarchical neural network using the training set, the validation set and the test set, to obtain a trained hierarchical neural network.

[0062] In this embodiment, the labeling of the point cloud data samples in step S202 is to use the CloudCompare software to label the point cloud data of the power tower and the surrounding environment collected by the unmanned aerial vehicle carrying the laser radar. A certain number of power towers and the environment within a certain range are collected as a unit, and the unmanned aerial vehicle carrying the laser radar is used to collect and shoot, to obtain the original point cloud data in las format. The CloudCompare software is used to process the original point cloud data, and the tower is obtained by multiple segmentation at multiple angles until the environment background points are not contained. After obtaining all the towers of a line, the tower part and the environment background part are fused respectively. The tower part is labeled as 1 and the natural environment part is labeled as 0, and then the tower part and the environment background part labeled are fused into a whole. Because the point cloud data is a three-dimensional model composed of points, the tower points and the background trees are mixed together, so different angles are rotated to segment the relatively pure tower by multiple segmentation at multiple angles.

[0063] In this embodiment, the preprocessing of the original power tower data set in step S203 includes segmentation processing for the original point cloud data sample (size 2-4 GB), using different angle slices to divide the point cloud slices of the power tower, and the point cloud slices of different angles of the same tower contain all tower points and natural environment points with different ranges. Then, according to the point cloud data density, a sparse coefficient is set, the point cloud slices of the power tower are input, the distance between two points in space is calculated, and if the points within the sparse coefficient range are discarded, the distance between the remaining points is equal to the sparse coefficient, so that multiple point cloud slices of different sparse degrees of the same power tower are obtained from one point cloud slice to realize data expansion. Then, according to the data expanded point cloud data sample, the effective information columns X, Y, Z, R, G, B and label are extracted, and the X, Y, Z, R, G, B columns are normalized. Among them, (X, Y, Z) is the three-dimensional coordinates of the point, (R, G, B) is the color value of the three channels of the point, and label is the label. Through the above data expansion method, the problems of slow segmentation network processing caused by too large point cloud slices and insufficient training data caused by too small number of tower point cloud data samples can be solved without affecting the segmentation accuracy.

[0064] Because the original line point cloud data is too large, it needs to be sparsified. In this embodiment, when the sparse coefficient is set, the sparse coefficient value range is 0.08-0.15, representing the distance between adjacent points, with the unit of meters. In addition, to solve the problem of too few actual power tower samples, this embodiment adopts a solution of slicing the part containing the tower, that is, using multiple different angle slices or slices containing environment background with inconsistent ranges to realize data expansion.

[0065] The position information (X, Y, Z), color information (R, G, B) and label information label in the point cloud slice are extracted, and the position information and color information are normalized. First, the average value of the maximum and minimum values in the three directions of the coordinate axis is calculated. In this embodiment, the normalization includes:

[0066] S301, the average value of the maximum and minimum values in the three directions of the coordinate axis is calculated according to the following formula: m m m

[0067]

[0068] In the above formula, (X, Y, Z) are the three-dimensional coordinates of the point, X min , Y min and Z min are the minimum values of X, Y and Z in three directions, respectively, and X max , Y​​​max and Z max are the maximum values in the three directions of X, Y and Z respectively;

[0069] S302, calculate the scaling factor δ according to the average value of the maximum value and the minimum value in the three directions of the coordinate axis (X m , Y m , Z m ):

[0070] δ = max (X-X m , Y-Y m , Z-Z m ),

[0071] In the above formula, max represents taking the maximum value;

[0072] S303, use the scaling factor δ to scale the point cloud data of the power tower and the surrounding environment to the interval (-1, 1) as a whole, and obtain the three-dimensional coordinates after isotropic normalization.

[0073] In this embodiment, the function expression for scaling the point cloud data of the power tower and the surrounding environment to the interval (-1, 1) as a whole by using the scaling factor δ in step S303 is:

[0074]

[0075] In the above formula, (X n , Y n , Z n ) is the three-dimensional coordinate (X, Y, Z) of the point after isotropic normalization. This normalization method scales the point cloud in the three coordinate dimensions at the same scale, without changing the original shape of the point cloud in the scene. Similarly, the color information R, G and B are also processed in the same way; finally, all the tower point cloud data slices after normalization are numbered. Figure 3 is the output result of the layered neural network in this embodiment, the real label and the actual point cloud scene comparison diagram.

[0076] In the training of this embodiment, the Adam optimizer is used to train the hierarchical neural model, the learning rate is 0.001, the momentum parameter is 0.9, and the weight decay is 0.0001. The size of each mini-batch is 4, and the number of training iterations is 200 epochs. In order to reduce overfitting, random inactivation method (dropout) and early stopping method are used in each epoch of training. In the test, the input point cloud is converted into a feature representation, and then the feature representation is input into the trained classifier for prediction. The cross-entropy loss function is used to calculate the loss of the classifier, and the average cross-entropy loss function mIoU and the accuracy indicator Acc are used for training and evaluation. The hierarchical neural network model has good performance and availability in the power tower point cloud segmentation task, and can provide support for the automatic analysis and intelligent diagnosis of power towers.

[0077] In addition, the embodiment also provides a power tower point cloud segmentation system based on a hierarchical neural network, which comprises a microprocessor and a memory connected with each other, and the microprocessor is programmed or configured to execute the power tower point cloud segmentation method based on the hierarchical neural network. In addition, the embodiment also provides a computer readable storage medium, wherein a computer program for programming or configuring the microprocessor to execute the power tower point cloud segmentation method based on the hierarchical neural network is stored in the computer readable storage medium.

[0078] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code. The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a machine that implements the functions described in the flowcharts and / or block diagrams. Figure 1 one flow or multiple flows and / or blocks Figure 1apparatuses that implement the functions specified in one or more flowcharts and / or blocks. These computer program instructions can also be loaded onto a computer or other programmable data processing devices to cause a series of operational steps to be performed on the computer or other programmable devices to produce a computer implemented process such that the instructions that are executed on the computer or other programmable devices provide steps for implementing the functions specified in the flowcharts and / or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks These computer program instructions can also be loaded onto a computer or other programmable data processing devices to cause a series of operational steps to be performed on the computer or other programmable devices to produce a computer implemented process such that the instructions that are executed on the computer or other programmable devices provide steps for implementing the functions specified in the flowcharts and / or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0079] The above description is merely preferred embodiments of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solutions falling within the concept of the present application shall be considered as falling within the protection scope of the present application. It should be noted that, for those of ordinary skill in the art, some improvements and refinements can be made without departing from the principles of the present application, and these improvements and refinements shall also be considered as falling within the protection scope of the present application.

Claims

1. A method for segmenting point clouds of power poles based on hierarchical neural networks, characterized in that, include: S101, acquire point cloud data of the power poles and surrounding environment to be processed; S102, input the point cloud data into the hierarchical neural network: use a multi-level ensemble sampling layer to sample the point cloud data, including: learning local features of the point cloud to obtain the local feature vector of each centroid; selecting the neighboring points around each centroid to construct the feature vector of the neighboring points to achieve local region extraction; fusing the local feature vector of each centroid and the feature vector of its neighboring points to obtain the local feature vector of each centroid; using a multi-level feature propagation layer to upsample the local feature vector of each centroid obtained by sampling to complete the feature learning of the point cloud data to obtain the global feature vector of the entire point cloud; using a fully connected layer to map the global feature vector of the entire point cloud into a class score vector; and then using a Softmax activation layer to convert the class score vector into a probability distribution to obtain the probability that each point in the point cloud data belongs to a different class. The ensemble sampling layer includes: The sampling layer is used to select a series of points from the input point cloud as centroids of local regions using the maximum margin sampling method (MDS). This includes: randomly selecting two initial points as representative points based on a set initial distance value; calculating the distance between each point and the representative points and selecting the point with the furthest sum of distances to the representative points as the next representative point, adding it to the selected representative point set, and iteratively expanding the representative point set by calculating the distance between each point and the representative points and selecting the point with the furthest sum of distances to the representative points as the next representative point until the entire point cloud data is covered; and using the cross-level aggregation method (CLA) to select K points around each centroid as its neighborhood points to achieve local region extraction. This includes: calculating the neighborhood point set for each point based on a preset radius list and the number of neighborhood points; selecting a certain number of neighborhood points from the neighborhood point set and concatenating them with the center point to generate an enhanced feature representation; iteratively increasing the radius to expand the receptive field of the feature; and finally selecting K points around each centroid as its neighborhood points. Group layers are used to construct local region sets (groups) by utilizing the cross-level aggregation method CLA to find neighborhood points around the centroids extracted from the sampling layer, with a size of [missing information]. The sum and size of the point set are The centroid coordinates are used as input, where the point set represents the point with... 3D coordinates and dimensional features There are 1 point, and the output is of size 1. A set of points, where each set corresponds to a local region, and K is the number of points near the centroid. This set of points is composed of... A new one with d-dimensional coordinates and a summary of local context The feature vector is composed of sub-sampling points; The dot layer is used for processing the output of the group layer. Each local region output consists of its centroid and its neighborhood of a size of [missing information]. The local feature abstraction is used as the sampling result output by the set sampling layer.

2. The point cloud segmentation method for power poles based on hierarchical neural networks according to claim 1, characterized in that, In step S102, upsampling the sampling results using a multi-level feature propagation layer refers to upsampling the sampling results using distance-weighted linear interpolation, the function expression of which is: ， In the above formula, For in position Dimension of features The features after the second upsampling First derivative, For in position First The distance weight corresponding to each sampling result For the first Each feature dimension Characteristics of the second sampling result For position Compared with sampling results The distance between them For the exponential parameter of distance, The feature dimension, The number of samples. denoted as the total dimension of the features.

3. The point cloud segmentation method for power poles based on hierarchical neural networks according to claim 1 or 2, characterized in that, Step S102 is preceded by a step of training a hierarchical neural network: S201, Obtain point cloud data samples of the power poles and surrounding environment to be processed; S202, label the point cloud data samples to obtain the original power pole tower dataset; S203, preprocessing the original power pole dataset, dividing the preprocessed power pole dataset into training set, validation set and test set according to a specified ratio; S204 uses the training set, validation set, and test set to train a hierarchical neural network, resulting in a well-trained hierarchical neural network.

4. The point cloud segmentation method for power poles based on hierarchical neural networks according to claim 3, characterized in that, Step S203 involves preprocessing the original power pole dataset, including: segmenting the original point cloud data samples by using slices at different angles to obtain point cloud slices for the power poles. Point cloud slices from different angles for the same pole contain all pole points and natural environment points with varying ranges. A sparsity coefficient is then set based on the point cloud data density. The point cloud slices are input, and the distance between two points is calculated. Points within the sparsity coefficient range are discarded, and the distances between the remaining points are equal to the sparsity coefficient. This process ultimately yields multiple point cloud slices with different sparsity levels for the same power pole from a single point cloud slice, thus augmenting the data. Then, effective information columns X, Y, Z, R, G, B, and label are extracted from the augmented point cloud data samples. Columns X, Y, Z, R, G, and B are normalized to the same attribute. Let the coordinates of the point be 3D coordinates. Here are the color values ​​for the three channels of the point, and label is the label.

5. The point cloud segmentation method for power poles based on hierarchical neural networks according to claim 4, characterized in that, When setting the sparsity coefficient, the value of the sparsity coefficient is in the range of 0.08-0.

15.

6. The point cloud segmentation method for power poles based on hierarchical neural networks according to claim 5, characterized in that, The process of performing homogeneous normalization includes: S301, calculate the average of the maximum and minimum values ​​in the three directions of the coordinate axis according to the following formula. : ， In the above formula, These are the three-dimensional coordinates of the point. , and These are the minimum values ​​in the X, Y, and Z directions, respectively. , and These are the maximum values ​​in the X, Y, and Z directions, respectively. S302, based on the average of the maximum and minimum values ​​in the three directions of the coordinate axes. Calculate scaling factor : ， In the above formula, max represents taking the maximum value; S303, using scaling factor The point cloud data of the power pole and its surrounding environment are scaled to the interval (-1,1) to obtain the three-dimensional coordinates after isotropic normalization.

7. The point cloud segmentation method for power poles based on hierarchical neural networks according to claim 6, characterized in that, In step S303, the scaling factor is used. The function expression for scaling the point cloud data of power poles and their surrounding environment to the interval (-1, 1) is as follows: ， In the above formula, Three-dimensional coordinates of a point The three-dimensional coordinates after being normalized to the same polarity.

8. A point cloud segmentation system for power poles based on a hierarchical neural network, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to execute the power pole point cloud segmentation method based on any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, The computer program is used to be programmed or configured by a microprocessor to execute the power pole point cloud segmentation method based on any one of claims 1 to 7.

Images