Power gallery point cloud semantic segmentation method and system based on RandLA-Net

Abstract

The invention relates to the technical field of point cloud processing, in particular to an electric power gallery point cloud semantic segmentation method and system based on RandLA-Net, and the method comprises the steps: constructing an electric power gallery point cloud data set; randomly sampling the power corridor point cloud data set to generate target data, extracting local features through the target data based on a training model, transmitting the local features to an encoder for aggregation, transmitting the aggregated local features to a decoder, and obtaining an output probability that the target data belongs to each label category; the output probability is adjusted, the adjusted output probability is smoothed, a screening threshold value is set, the label category of the target data with the confidence coefficient smaller than the screening threshold value is determined again, training is completed, and a RandLA-Net model is obtained; inputting to-be-tested original point cloud data into the RandLA-Net model to obtain a segmentation result, displaying the segmentation result and generating an analysis report; according to the method, an existing RandLA-Net algorithm is improved, and the processing precision and efficiency of the point cloud data are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud processing, and in particular to a method and system for semantic segmentation of power corridor point clouds based on RandLA-Net. Background Art

[0002] With the rapid development of lidar technology, the operation and inspection work of the power system is gradually shifting from traditional manual operations to the direction of intelligence and automation; by obtaining the three-dimensional point cloud data of the power corridor through airborne or vehicle-mounted lidar equipment, the spatial distribution and status of power facilities, namely towers, conductors, insulators, etc., can be comprehensively grasped, providing accurate information support for fault diagnosis and hidden danger investigation. However, power corridor point cloud data usually has characteristics such as a large amount of data, many target types, and complex structures, and there are often significant density differences or occlusion relationships between power facilities. Therefore, semantic segmentation and category recognition are required to accurately analyze the point cloud data to ensure intelligent inspection and safety monitoring in the power industry.

[0003] Existing point cloud data segmentation techniques usually include point cloud segmentation based on traditional methods such as region growing algorithms, RANSAC (Random Sample Consensus), etc., which rely on geometric features for clustering and segmentation, with relatively high computational efficiency. However, it is difficult to fully extract the deep semantic information of the point cloud, is easily interfered by noise and complex backgrounds, and is difficult to adapt to the accurate recognition of multi-category targets in various scenarios; or point cloud segmentation based on deep learning, which includes three schemes based on voxels, projections, or direct processing, but all have problems such as detail loss, missed and false detections, and large computational overhead, affecting the segmentation efficiency; for this, the RandLA-Net (Randomized Local Attention Network) method is proposed, which can better retain the local geometric features of the point cloud while reducing the computational complexity by introducing random sampling and local feature aggregation modules; however, directly applying RandLA-Net to the power corridor scenario still has defects such as an overly large amount of point cloud data, target density differences and small target recognition, and complex background interference. Among them, since the power corridor usually covers dozens of kilometers or even a larger range, a single lidar scan can generate tens of millions to hundreds of millions of points, and facilities such as conductors and insulators account for a very small proportion in the overall scenario and have uneven densities. Random sampling may lead to the loss of key feature points, affecting the recognition accuracy of small targets; in addition, the power corridor often contains complex environments such as dense vegetation and mountainous terrains, and in order to accurately analyze, it is necessary to accurately distinguish power facilities from the background. Summary of the Invention

[0004] To solve the technical problems existing in the existing RandLA-Net method for point cloud data segmentation, such as large amount of point cloud data, different target density distributions, low recognition accuracy for key features, and strong interference from complex environmental backgrounds, the purpose of the present invention is to provide a method for semantic segmentation of power corridor point clouds based on RandLA-Net. The specific technical solutions adopted are as follows:

[0005] Collect the original point cloud data of the power corridor, perform preprocessing, and construct a power corridor point cloud dataset;

[0006] Obtain a training model, randomly sample the power corridor point cloud dataset to generate target data, extract local features from the target data based on the training model, and transmit the local features to the encoder for aggregation. The aggregated local features are transmitted to the decoder to obtain the output probability of the target data belonging to each label category;

[0007] Adjust the output probability through class weights, smooth the adjusted output probability based on spatial continuity to obtain confidence, set a screening threshold, re-determine the label category of the target data with a confidence less than the screening threshold, and complete the training to obtain the RandLA-Net model;

[0008] Input the original point cloud data to be measured into the RandLA-Net model to obtain the segmentation result, display the segmentation result, and generate an analysis report.

[0009] Preferably, the label categories of the power corridor point cloud dataset include power towers, conductors, and vegetation.

[0010] Preferably, collecting the original point cloud data of the power corridor, performing preprocessing, and constructing a power corridor point cloud dataset includes:

[0011] Use airborne lidar and camera equipment to scan the power corridor to obtain the original point cloud data. The original point cloud data includes position information and color information, and fuse the position information and color information to generate true-color point cloud data;

[0012] According to the power layout in the power corridor, cut the true-color point cloud data into several power segments, and sequentially process the original point cloud data of each power segment using the cloth simulation filtering algorithm and sampling method to construct a power corridor point cloud dataset.

[0013] Preferably, randomly sampling the power corridor point cloud dataset to generate target data, extracting local features from the target data based on the training model, and transmitting the local features to the encoder for aggregation. The aggregated local features are transmitted to the decoder to obtain the output probability of the target data belonging to each label category, including:

[0014] Construct input data based on the power corridor point cloud dataset, and randomly sample the input data to generate target data;

[0015] Based on the trained model, use the K-nearest neighbor algorithm to obtain each data point in the target data and its corresponding K nearest neighbors, perform local spatial encoding and attention pooling, extract local features, and aggregate and transmit them to the decoder;

[0016] The decoder performs upsampling through deconvolution to obtain the aggregated local features of each layer of the corresponding encoder, and performs class prediction on each target data through a fully connected layer to obtain the output probability that the target data belongs to each label category.

[0017] Preferably, based on the trained model, use the K-nearest neighbor algorithm to obtain each data point in the target data and its corresponding K nearest neighbors, perform local spatial encoding and attention pooling, extract local features, and aggregate and transmit them to the decoder, including:

[0018] Extract the local features of the target data based on the trained model and transmit them to the encoder. The encoder uses the K-nearest neighbor algorithm to obtain K neighboring points according to the center point of each local feature, calculates the Euclidean distance between each neighboring point and the center point, and connects the Euclidean distance with the center point of the local feature, the neighboring points, and the differences between the center point and the neighboring points to generate neighborhood features;

[0019] Obtain attention weights, determine weighted features in combination with neighborhood features, sum the weighted features and the neighborhood features to obtain the aggregated local features, and transmit them to the decoder.

[0020] Preferably, the corresponding calculation formula for generating neighborhood features is:

[0021]

[0022] Among them, represents the neighborhood feature of the neighboring point corresponding to the center point of the i-th local feature; MLP represents a multi-layer perceptron; p i represents the position information of the center point of the i-th local feature; represents the position information of the neighboring point corresponding to the center point of the i-th local feature; represents the concatenation operation; ||·|| represents the Euclidean distance calculation.

[0023] Preferably, the corresponding calculation formula for obtaining the aggregated local features is:

[0024]

[0025]

[0026] Among them, f idenote the aggregated local features corresponding to the central point of the $i$-th local feature and its neighboring points; $K$ denotes $K$ neighboring points; denote the neighborhood features of the neighboring points corresponding to the central point of the $i$-th local feature; denote the weighted features corresponding to the central point of the $i$-th local feature and its neighboring points; $g$ denotes the attention function; $W$ denotes the attention weight.

[0027] Preferably, adjust the output probability through class weights, smooth the adjusted output probability based on spatial continuity to obtain the confidence, set a screening threshold, re-determine the label category for the target data with a confidence less than the screening threshold, and complete the training to obtain the RandLA-Net model, including:

[0028] Determine the label category based on the power corridor point cloud dataset, calculate the class weights, and adjust the output probability through the class weights;

[0029] Statistically analyze the occurrence frequencies of the predicted label categories and neighborhood label categories of the target data corresponding to each output probability to obtain the confidence;

[0030] If the predicted label category and the neighborhood label category of the target data with a confidence less than the screening threshold are inconsistent, correct the predicted label category to the neighborhood label category and re-determine the label category according to the neighborhood information.

[0031] Preferably, input the original point cloud data to be measured into the RandLA-Net model to obtain the segmentation result, display the segmentation result and generate an analysis report, including:

[0032] Input the original point cloud data to be measured into the RandLA-Net model to obtain the segmentation result, display the label category determined by the segmentation result using color information, and statistically analyze any one of the data information such as the quantity, density, feature distribution, and distribution range of each point cloud in the label category to generate an analysis report.

[0033] To solve the above problems, the present application also provides: a power corridor point cloud semantic segmentation system based on RandLA-Net, the system includes:

[0034] A data acquisition and preprocessing module, used for: collecting the original point cloud data of the power corridor, performing preprocessing, and constructing a power corridor point cloud dataset;

[0035] A RandLA-Net semantic segmentation module, used for: obtaining a training model, randomly sampling the power corridor point cloud dataset to generate target data, extracting local features from the target data based on the training model and transmitting the local features to the encoder for aggregation, and transmitting the aggregated local features to the decoder to obtain the output probability of the target data belonging to each label category;

[0036] The classification result optimization module is used to: adjust the output probability through category weights, smooth the adjusted output probability based on spatial continuity to obtain confidence, set a screening threshold, re-determine the label category for the target data with confidence less than the screening threshold, and complete the training to obtain the RandLA-Net model;

[0037] The result visualization module is used to: input the original point cloud data to be measured into the RandLA-Net model to obtain the segmentation result, display the segmentation result and generate an analysis report.

[0038] The present invention has the following beneficial effects:

[0039] 1. In the present invention, the existing RandLA-Net is improved, and a random sampling strategy based on RandLA-Net is proposed, which significantly improves the point cloud processing efficiency, enabling it to directly process large-scale power corridor point cloud data without more complex preprocessing operations compared to this solution. It has high processing efficiency, can process point cloud data in the millions, and is suitable for real-time analysis of power corridors, that is, regional processing of point cloud data is achieved through power section segmentation; for the layout of power corridors, the improved RandLA-Net effectively captures the local features of power facilities and performs fusion processing to obtain aggregated local features, enabling it to still maintain a high segmentation accuracy even after random sampling, that is, by combining RandLA-Net and local feature aggregation, targeted optimization is carried out according to the distribution characteristics of power facilities to ensure accurate segmentation of the areas between different power towers; in addition, the overall design adapts to the characteristics of the power corridor scene, can effectively identify key power facilities such as power transmission towers and conductors, that is, the robustness and segmentation accuracy of the RandLA-Net model are further improved through post-processing means, with good scalability and versatility, and can adapt to power corridor scenes of different scales and complexities, and both the classification accuracy and the average intersection over union are improved.

[0040] 2. The present invention also provides a power corridor point cloud semantic segmentation system based on RandLA-Net for implementing the above-mentioned power corridor point cloud semantic segmentation method based on RandLA-Net. This system has the same beneficial effects as the above-mentioned power corridor point cloud semantic segmentation method based on RandLA-Net, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0042] Figure 1 The flowchart of the steps of a method for semantic segmentation of power corridor point clouds based on RandLA-Net provided by an embodiment of the present invention. Detailed implementation manners

[0043] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following, in combination with the accompanying drawings and preferred embodiments, details the specific implementation manners, structures, features and effects of a method and system for semantic segmentation of power corridor point clouds based on RandLA-Net proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0045] The following specifically describes the specific solutions of a method and system for semantic segmentation of power corridor point clouds based on RandLA-Net provided by the present invention in combination with the accompanying drawings.

[0046] The existing RandLA-Net reduces the computational complexity through random sampling and local feature aggregation while retaining local geometric features; however, in the power corridor scenario, problems such as large amounts of point cloud data, uneven target density, and small target recognition still exist. That is, due to the large number of points in the power corridor, the proportion of facilities is small and the density is uneven, random sampling may lead to the loss of key feature points, affecting the recognition accuracy of small targets; moreover, the power corridor environment is complex and the distinction from the background is insufficient. The first embodiment of the present invention provides a method for semantic segmentation of power corridor point clouds based on RandLA-Net. After sequentially performing power section segmentation, ground point filtering, and point cloud downsampling preprocessing on the original point cloud data, it uses random sampling and local feature aggregation to achieve efficient processing of the point cloud and optimize the segmentation result, and then visualizes the segmentation result to generate an analysis report. To solve a method for semantic segmentation of power corridor point clouds based on RandLA-Net, the second embodiment of the present invention provides a system for semantic segmentation of power corridor point clouds based on RandLA-Net. This system is essentially a software system composed of units that implement corresponding functions. Now, the specific steps in this method are introduced in detail.

[0047] Please refer to Figure 1 , which shows the flowchart of the steps of a method for semantic segmentation of power corridor point clouds based on RandLA-Net provided by the first embodiment of the present invention. The method includes:

[0048] Step S1: Collect the original point cloud data of the power corridor, preprocess it, and construct a point cloud dataset of the power corridor;

[0049] Step S2: Obtain a training model, randomly sample the point cloud dataset of the power corridor to generate target data, extract local features based on the training model through the target data, and transmit the local features to the encoder for aggregation. The aggregated local features are transmitted to the decoder to obtain the output probability of the target data belonging to each label category;

[0050] Step S3: Adjust the output probability through class weights, perform smoothing processing on the adjusted output probability based on spatial continuity to obtain the confidence, set a screening threshold, re-determine the label category for the target data with a confidence less than the screening threshold, and complete the training to obtain the RandLA-Net model;

[0051] Step S4: Input the original point cloud data to be measured into the RandLA-Net model to obtain the segmentation result, display the segmentation result, and generate an analysis report.

[0052] For better illustration, RandLA-Net is a deep learning model for processing point cloud data; it has a local attention mechanism (LocalAttention), enabling the model to effectively focus on the local features of the point cloud and reduce the computational amount; a random sampling strategy (Random Sampling), maintaining the representative features of the point cloud data while reducing the computational complexity; point cloud classification and segmentation tasks, that is, identifying different objects in the point cloud and dividing the point cloud into multiple different regions or categories; and it can also automatically extract and optimize features from the point cloud data through an adaptive learning method to adapt to the task requirements in different scenarios.

[0053] The point cloud data of the power corridor refers to a set of points in three-dimensional space, that is, composed of a series of irregularly distributed three-dimensional points, which can represent the precise position information of the power corridor and its surrounding environment, and detailedly record the geometric shapes and spatial distributions of power facilities such as power lines, towers, and insulators, providing important data support for the planning, construction, and maintenance of the power system.

[0054] Furthermore, in step S1, the label categories of the point cloud dataset of the power corridor include power towers, conductors, and vegetation; that is, the point cloud dataset is identified and divided by label categories. Among them, the power tower is an important structure for supporting the transmission line; the conductor is the main medium for power transmission; and the vegetation is a part of the surrounding environment of the power corridor. In practical applications, the point cloud data in each category varies greatly. The amount of point cloud data of vegetation is much larger than that of the other two categories, while the amount of point cloud data of the conductor is the least, forming an unbalanced data distribution, providing a training basis for the subsequent training model.

[0055] Furthermore, in step S1, it includes:

[0056] Step S11: Use airborne lidar and camera equipment to scan the power corridor to obtain the original point cloud data. The original point cloud data includes position information and color information. The position information and color information are fused to generate true-color point cloud data.

[0057] It should be noted that the position information refers to the three-dimensional coordinates of each point cloud in the original point cloud data, and the color information refers to the RGB information of the point cloud; the position information and color information provide more intuitive visual support for the monitoring and maintenance of the power corridor. By fusing the two, true-color point cloud data is generated, making the 3D model of the power facilities more vivid and real, and improving the accuracy of the point cloud data for analysis.

[0058] Step S12: According to the power layout in the power corridor, the true-color point cloud data is segmented into several power segments, and the original point cloud data of each power segment is processed in turn using the cloth simulation filtering algorithm and sampling method to construct a point cloud dataset of the power corridor.

[0059] It can be explained that in this embodiment, each power segment contains all the point cloud data of two power towers and the area between them.

[0060] Specifically, the true-color point cloud data is segmented mainly based on the positions of the power towers and the spatial layout of the power facilities. In the power corridor, the power towers are usually evenly distributed, so the area between every two power towers is regarded as a power segment. The three-dimensional coordinates of the power towers and the surrounding spatial area are obtained by scanning with lidar equipment, and based on these coordinates, the true-color point cloud data is segmented into several segments, which usually include two power towers and key power facilities such as the conductors between them, the support structures and auxiliary facilities around the power towers; after segmentation, the power towers and the power facilities such as the connected conductors are regarded as important areas, laying a foundation for subsequent semantic segmentation; the parts that are not power facilities, namely vegetation, ground, and areas far from the power towers, are regarded as non-key areas and the attention can be reduced subsequently.

[0061] The cloth simulation filtering algorithm is used to process the true-color point cloud data to identify and remove the ground points, that is, the true-color point cloud data is regarded as rigid particles and the ground is regarded as flexible cloth. By simulating the process of the cloth sinking, the ground points and non-ground points are distinguished, which not only improves the accuracy of the true-color point cloud data processing, but also is closer to the physical behavior of the cloth in the real world in terms of visual effect.

[0062] The true-color point cloud data is downsampled using an equidistant sampling method to reduce the point cloud density and eliminate the characteristic of denser near and sparser far caused by airborne lidar scanning. That is, the grid voxel downsampling method is used to achieve the downsampling of the true-color point cloud data by setting the voxel size. In this embodiment, the voxel size is set to 0.04, which can reduce the true-color point cloud data volume to about 12% of the original, while maintaining the integrity of the geometric structure.

[0063] It should be noted that to provide data support for the subsequent training, validation, and testing of the model, in this embodiment, the power corridor point cloud dataset is divided into a training set, a validation set, and a test set according to the ratio of 7:1.5:1.5; among them, the training set data is used during the training of the training model, that is, the output data; the validation set is used to verify the reliability of the obtained RandLA-Net model; the test set refers to the original point cloud data to be measured.

[0064] Furthermore, in step S2, it includes:

[0065] Step S21: Construct input data based on the power corridor point cloud dataset, and randomly sample the input data to generate target data.

[0066] It should be noted that using a random sampling strategy to downsample the input data has higher computational efficiency and is suitable for processing large-scale point cloud data; that is, random sampling is used to select representative point clouds in the large-scale point cloud data, so as to efficiently extract effective point cloud data from each power section and retain the key structural information of the power tower and its surrounding facilities, which can quickly narrow the processing range and avoid unnecessary waste of computing resources; in this embodiment, the downsampling rate for each layer is set to 25%, that is, the number of point clouds after each sampling is one-fourth of the original, so as to effectively simplify the scale of the input data.

[0067] Step S22: Based on the training model, use the K-nearest neighbor algorithm to obtain each data point and its corresponding K nearest neighbors in the target data, perform local spatial encoding and attention pooling, extract local features, and aggregate and transmit them to the decoder.

[0068] As an optional implementation manner, the training model in this embodiment is the RandLA-Net model before training.

[0069] It should be noted that by performing local feature aggregation on the target data obtained after random sampling, not only can the local geometric and semantic features of power facilities in each power section be accurately extracted, but also multi-scale information from different regions can be effectively aggregated. For example, between a power tower and a wire, the geometric information of the power tower is combined with the semantic features of the surrounding area through a local feature aggregation module to accurately identify and segment key facilities such as the power tower and the wire connected to it. The layout information can be retained during the aggregation process to ensure the accuracy of the semantic segmentation results for different power sections.

[0070] Further, in step S22, it includes:

[0071] Step S221: Extract the local features of the target data based on the training model and transmit them to the encoder. The encoder uses the K-nearest neighbor algorithm to obtain K nearest neighbor points according to the center point of each local feature, calculates the Euclidean distance between each nearest neighbor point and the center point, and connects the Euclidean distance with the differences between the center point of the local feature, the nearest neighbor point, and the center point and the nearest neighbor point to generate neighborhood features.

[0072] Specifically, for the center point of each local feature, that is, the corresponding point cloud data, the K-nearest neighbor algorithm is used to find its K nearest neighbor points. Optionally, in this embodiment, K = 16. Then, calculate the relative position of each nearest neighbor point with respect to the center point, that is, the Euclidean distance, connect the Euclidean distance with the point features between the point cloud data, and process the connected features through a shared MLP (Multi-Layer Perceptron) to generate neighborhood features.

[0073] Further, in step S221, when generating neighborhood features, the corresponding calculation formula is:

[0074]

[0075] Wherein, represents the neighborhood feature of the nearest neighbor point corresponding to the center point of the i-th local feature; MLP represents a multi-layer perceptron; p i represents the position information of the center point of the i-th local feature; represents the position information of the nearest neighbor point corresponding to the center point of the i-th local feature; represents the connection operation; ||·|| represents the Euclidean distance calculation.

[0076] Step S222: Obtain the attention weights, determine the weighted features in combination with the neighborhood features, sum the weighted features and the neighborhood features to obtain the aggregated local features, and transmit them to the decoder.

[0077] Specifically, the attention mechanism is used to perform weighted aggregation on local features. That is, for the center point of each local feature and its corresponding neighborhood features, the attention weights are calculated to highlight the contributions of important features and reduce the influence of unimportant features. Then, the weighted features are summed with the neighborhood features to obtain the aggregated local features, so as to strengthen the important point cloud data while retaining the overall key information.

[0078] Furthermore, in step S222, the aggregated local features are obtained, and the corresponding calculation formula is:

[0079]

[0080] where, f i represents the aggregated local feature corresponding to the center point of the i-th local feature and its neighboring points; K represents K neighboring points; represents the neighborhood feature of the neighboring points corresponding to the center point of the i-th local feature; represents the weighted feature corresponding to the center point of the i-th local feature and its neighboring points; g represents the attention function; W represents the attention weight.

[0081] It should be noted that in this embodiment, a residual connection is introduced. That is, multi-scale feature information is retained through skip connection, and the input neighborhood features are directly connected to the aggregated local features to form a residual connection, which is helpful for network training and feature transmission.

[0082] It can be understood that due to the use of random sampling, some key features in the target data may be lost, resulting in the loss of some local features, that is, the processing of small targets or edges such as wires will be affected. Therefore, by encoding and aggregating each point cloud data in the target data and its surrounding neighborhoods, the influence of feature loss can be reduced. That is, through local spatial encoding and attention pooling operations, the neighborhood features can be effectively aggregated, and the surrounding point cloud data and neighborhood features can still make up for these missing features. And through the way of residual connection and aggregation, multi-scale information is retained, so that even if some key data in the target data obtained by random sampling is lost, the extraction and aggregation of local features can still retain the overall feature information.

[0083] Step S23: The decoder performs upsampling through deconvolution to obtain the aggregated local features corresponding to each layer of the encoder, and performs class prediction on each target data through a fully connected layer to obtain the output probability that the target data belongs to each label category.

[0084] Specifically, through upsampling and feature fusion operations, the density of the point cloud data is restored and the class prediction of each point cloud data is output. That is, upsampling is performed through deconvolution, the features of the corresponding encoder layer are fused using skip connections, and then the output probability of each point cloud data belonging to each label class is output at the last layer of the decoder.

[0085] It can be understood that in this embodiment, the encoder includes 5 layers, and each layer of the encoder includes random sampling, local feature aggregation, and feature extraction operations; the decoder correspondingly includes 5 layers; specifically, before the local features enter the encoder, the local features are lifted from 3 dimensions to 8 dimensions, and the downsampling rate is 0.25 for each layer. The output dimensions of the encoder are [32, 64, 128, 256, 512] in sequence. Assuming the input point cloud data volume is 40960, after entering the encoder, the first layer is represented as (40960, 8) -> (10240, 32), and the aggregated local features are obtained, and the dimension is lifted to 32 dimensions to enhance the expression ability of the local features. At this time, each subsequent layer is represented as (10240, 32) -> (2560, 64); (2560, 64) -> (640, 128); (640, 128) -> (160, 256) and (160, 256) -> (40, 512) in sequence; then the obtained data representation is transmitted to the decoder, where the first layer is correspondingly represented as (40, 512) -> (160, 512), and it is concatenated with the local features output by the fourth layer of the encoder to obtain (160, 512 + 256) -> (160, 256). Similarly, the representations of each layer corresponding to the decoder are obtained as (160, 256) -> (640, 128); (640, 128) -> (2560, 64); (2560, 64) -> (10240, 32) and (10240, 32) -> (40960, 32) in sequence, and the local features of the first layer are concatenated twice during decoding; finally, at the fifth layer of the decoder, the training model performs class prediction on the point cloud data corresponding to each local feature through a fully connected layer, that is, the label class of each point cloud data is calculated through the softmax function, and then the output probability of each point cloud data belonging to different label classes is obtained; it should be noted that the shape of the decoder output is (B, N, num_classes), that is, the output of each point cloud data is a vector of num_classes dimensions, and each element represents the output probability of the point belonging to the corresponding label class; the sum of the output probabilities of all label classes is 1.

[0086] Furthermore, in step S3, it includes:

[0087] Step S31: Determine the label classes based on the power corridor point cloud dataset, calculate the class weights, and adjust the output probabilities through the class weights.

[0088] It should be noted that there is a problem of unbalanced sample data in each label category of the point cloud data in the power corridor. That is, in the power corridor scenario, vegetation and ground points usually occupy most of the point cloud data, while the number of points of power facilities such as insulators and jumpers is small. Therefore, the category weights are adjusted to improve the recognition accuracy of small categories in power facilities, that is, a higher weight is assigned to small categories, and a smaller weight is assigned to unimportant point cloud data such as vegetation. Specifically, the cross-entropy loss function is used, and the weights are set according to the category occurrence frequency. The corresponding calculation formula is:

[0089]

[0090] where w j represents the category weight of the j-th label category; f j represents the frequency of occurrence of the j-th label category; 0.02 is used to prevent the denominator from being zero.

[0091] It should be noted that during the training process, the batch size is set to 4, and the maximum number of training epochs is 100. That is, while ensuring the training efficiency, the limited computing resources are utilized as much as possible. To avoid overfitting and affecting the generalization ability of the model on unknown data, when the performance on the validation set has not improved for 6 consecutive epochs, the training is stopped in advance to ensure that the trained model can better adapt to new and unseen data.

[0092] Step S32: Statistically calculate the occurrence frequencies of the predicted label category and the neighborhood label category of the target data corresponding to each output probability to obtain the confidence level.

[0093] It should be noted that by using the spatial continuity characteristic of the point cloud data, it is smoothed; that is, for the target data corresponding to each output probability, the occurrence frequencies of each label category in its neighborhood and its predicted label category are statistically calculated to obtain the confidence level, where the confidence level refers to the prediction confidence of the trained model for the data point.

[0094] Step S33: If the predicted label category of the target data with a confidence level less than the screening threshold is inconsistent with the neighborhood label category, the predicted label category is corrected to the neighborhood label category, and the label category is re-determined according to the neighborhood information.

[0095] As an optional implementation manner, in this embodiment, the screening threshold is 0.5.

[0096] Specifically, if the predicted label category of the currently analyzed point cloud data is inconsistent with the neighborhood label category and the confidence level is less than the screening threshold, that is, the training model is not confident enough in the predicted label category of this point cloud data, it will be corrected to the neighborhood label category and marked as unclassified, or the label category of the corresponding point cloud data will be re-determined according to the neighborhood information; on the contrary, if the confidence level is not high but the predicted label category is consistent with the neighborhood label category, no correction is required; it should be noted that to ensure the consistency of each point cloud data in the point cloud data, the label category distribution within the neighborhood of each point cloud data is statistically analyzed, and the label category with the highest frequency of occurrence within the neighborhood is used as the currently analyzed neighborhood label category; thus, a RandLA-Net model trained based on the improved method is obtained.

[0097] Further, in step S4, it includes:

[0098] Input the original point cloud data to be measured into the RandLA-Net model to obtain the segmentation result, display the label category determined by the segmentation result using color information, and statistically analyze any one of the data information such as the number, density, feature distribution, and distribution range of each point cloud in the label category to generate an analysis report.

[0099] Specifically, input the original point cloud data to be measured into the RandLA-Net model to obtain the segmentation result, that is, obtain the label category distribution corresponding to the point cloud data, perform three-dimensional visualization display on the segmentation result, display the segmentation result in different colors, and intuitively display the distribution of various facilities and environmental elements in the power corridor; at the same time, statistically analyze any one of the data information such as the number, density, feature distribution, and distribution range of each point cloud in the label category to generate an analysis report, that is, distinguish the point cloud data of different label categories with different colors or markings, and clearly present the spatial distribution relationship between the power facilities and the surrounding environment.

[0100] Preferably, in this embodiment, the mean intersection over union (mIoU) and classification accuracy (Accuracy) are used as evaluation indicators, where OA (Overall Accuracy, that is, the overall classification accuracy) represents the overall effect of classification, to compare the experimental results of different methods on the power corridor point cloud data set, and ensure that the RandLA-Net model of this application is the best, as shown in Table 1, the result comparison of different methods on the power corridor point cloud data set:

[0101] Table 1 Result comparison of different methods on the power corridor point cloud data set

[0102]

[0103] It is explained that the test set achieves a classification accuracy of 94.4% and an average intersection over union of 76.0%. For the label categories with a large proportion in the power corridor point cloud data, such as vegetation or power towers, etc., the recognition accuracy is relatively high, and the accuracy rate can reach over 90%; while for the label categories with a small proportion and slender shapes, such as conductors, insulators or jumpers, etc., the recognition accuracy has also been significantly improved; that is to say, the whole method can quickly and accurately identify various facilities and environmental elements in the power corridor, providing an efficient and reliable technical means for the intelligent inspection, equipment monitoring and potential hazard investigation of the power corridor, and having important engineering application value.

[0104] Optionally, in terms of processing efficiency, this method can directly process the power corridor point cloud data containing tens of millions of points. Compared with the traditional method using the farthest point sampling (FPS) strategy, the processing speed is increased by 2 - 5 times; for example, for the point cloud data with a scale of about 10 million points, based on the RandLA-Net model proposed in this application, the overall duration can be controlled within about 62 seconds during a single processing, while using the FPS-based scheme usually requires more than 200 seconds; and the improved RandLA-Net model in this application can be compatible with power corridor scenarios of different scales and complexities by appropriately adjusting the network structure and training strategy, which can not only meet the requirements of high accuracy but also streamline the parameters in cases where faster inspection is needed, providing real-time or quasi-real-time analysis capabilities that meet the requirements of engineering applications.

[0105] In the second embodiment of the present invention, a power corridor point cloud semantic segmentation system based on RandLA-Net is also proposed, including:

[0106] A data acquisition and preprocessing module, which is used to: collect the original point cloud data of the power corridor, perform preprocessing, and construct a power corridor point cloud data set;

[0107] A RandLA-Net semantic segmentation module, which is used to: obtain a training model, randomly sample the power corridor point cloud data set to generate target data, extract local features based on the training model through the target data and transmit the local features to the encoder for aggregation, and transmit the aggregated local features to the decoder to obtain the output probability of the target data belonging to each label category;

[0108] A classification result optimization module, which is used to: adjust the output probability through category weights, perform smoothing processing on the adjusted output probability based on spatial continuity, obtain the confidence, set a screening threshold, re-determine the label category for the target data with a confidence less than the screening threshold, and complete the training to obtain the RandLA-Net model;

[0109] A result visualization module is used to: input the original point cloud data to be measured into the RandLA-Net model to obtain the segmentation result, display the segmentation result, and generate an analysis report.

[0110] Understandably, in this embodiment, the data acquisition and preprocessing module includes a data acquisition unit, a power section segmentation unit, a ground point filtering unit, and a point cloud downsampling unit. These units cooperate with each other to obtain and preprocess the original point cloud data to obtain the constructed power corridor point cloud dataset. The RandLA-Net semantic segmentation module includes a random sampling unit, a local feature aggregation unit, an encoder unit, and a decoder unit, and realizes the semantic segmentation of the power corridor point cloud dataset based on the RandLA-Net semantic segmentation module. The classification result optimization module includes a class balance processing unit, a spatial consistency optimization unit, and a confidence threshold filtering unit, which are used to post-process and optimize the original segmentation result output by the RandLA-Net semantic segmentation module, that is, the output probability. The result visualization module visually displays the segmentation result of the optimized RandLA-Net model in three dimensions and generates an analysis report.

[0111] It can be illustrated that through the cooperation of several modules, the power department can realize the automated monitoring and analysis of various facilities and environmental elements within the corridor range, including but not limited to any application scenarios such as power equipment status monitoring, vegetation intrusion analysis, and foreign object detection, thus providing an efficient and reliable technical means for the intelligent inspection and safe operation and maintenance of power corridors.

[0112] It should be noted that a power corridor point cloud semantic segmentation system based on RandLA-Net has the same beneficial effects as the power corridor point cloud semantic segmentation method provided in the foregoing first embodiment, and will not be elaborated here.

[0113] It should be noted that the above sequence of the embodiments of the present invention is only for description and does not represent the advantages or disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0114] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other, and the key points of each embodiment are the differences from other embodiments.

Claims

1. A semantic segmentation method for power corridor point clouds based on RandLA-Net, characterized in that The method includes: Collecting the original point cloud data of the power corridor, preprocessing it, and constructing a point cloud dataset of the power corridor; Obtaining a training model, randomly sampling the point cloud dataset of the power corridor to generate target data, extracting local features based on the training model through the target data, and transmitting the local features to the encoder for aggregation. The aggregated local features are transmitted to the decoder to obtain the output probability of the target data belonging to each label category; Adjusting the output probability through class weights, smoothing the adjusted output probability based on spatial continuity to obtain confidence, setting a screening threshold, re-determining the label category of the target data with a confidence less than the screening threshold, and completing the training to obtain the RandLA-Net model; Inputting the original point cloud data to be measured into the RandLA-Net model to obtain a segmentation result, displaying the segmentation result, and generating an analysis report.

2. The method for semantic segmentation of power corridor point cloud based on RandLA-Net according to claim 1, wherein The label categories of the point cloud dataset of the power corridor include power towers, conductors, and vegetation.

3. The method for semantic segmentation of power corridor point cloud based on RandLA-Net according to claim 2, characterized in that, Collecting the original point cloud data of the power corridor, preprocessing it, and constructing a point cloud dataset of the power corridor, including: Scanning the power corridor using airborne lidar and camera equipment to obtain the original point cloud data, where the original point cloud data includes position information and color information, and fusing the position information and color information to generate true-color point cloud data; According to the power layout in the power corridor, cutting the true-color point cloud data into several power segments, and sequentially processing the original point cloud data of each power segment using the cloth simulation filtering algorithm and sampling method to construct a point cloud dataset of the power corridor.

4. The method for semantic segmentation of power corridor point cloud based on RandLA-Net according to claim 3, wherein, Randomly sampling the point cloud dataset of the power corridor to generate target data, extracting local features based on the training model through the target data, and transmitting the local features to the encoder for aggregation. The aggregated local features are transmitted to the decoder to obtain the output probability of the target data belonging to each label category, including: Constructing input data based on the point cloud dataset of the power corridor and randomly sampling the input data to generate target data; Based on the training model, using the K-nearest neighbor algorithm to obtain each data point in the target data and its corresponding K nearest neighbors, performing local spatial encoding and attention pooling, extracting local features, and aggregating and transmitting them to the decoder; The decoder performs upsampling through deconvolution to obtain the aggregated local features corresponding to each layer of the encoder, and performs class prediction on each target data through a fully connected layer to obtain the output probability of the target data belonging to each label category.

5. The method for semantic segmentation of power corridor point cloud based on RandLA-Net according to claim 4, wherein Based on the training model, using the K-nearest neighbor algorithm to obtain each data point in the target data and its corresponding K nearest neighbors, performing local spatial encoding and attention pooling, extracting local features, and aggregating and transmitting them to the decoder, including: Extracting the local features of the target data based on the training model and transmitting them to the encoder. The encoder uses the K-nearest neighbor algorithm to obtain K neighboring points according to the center point of each local feature, calculates the Euclidean distance between each neighboring point and the center point, and connects the Euclidean distance with the center point of the local feature, the neighboring points, and the difference between the center point and the neighboring points to generate neighborhood features; Obtain attention weights, determine weighted features by combining neighborhood features, sum the weighted features with neighborhood features to obtain aggregated local features, and transmit them to the decoder.

6. The method for semantic segmentation of power corridor point cloud based on RandLA-Net according to claim 5, wherein Generate neighborhood features, and the corresponding calculation formula is: Among them, represents the neighborhood feature of the neighbor points corresponding to the center point of the i-th local feature; MLP represents a multi-layer perceptron; p i represents the position information of the center point of the i-th local feature; represents the position information of the neighbor points corresponding to the center point of the i-th local feature; represents the connection operation; ||·|| represents the calculation of the Euclidean distance.

7. The method for semantic segmentation of power corridor point cloud based on RandLA-Net according to claim 5, wherein Obtain the aggregated local features, and the corresponding calculation formula is: where, f i represents the aggregated local feature corresponding to the central point of the i-th local feature and its neighboring points; K represents K neighboring points; represents the neighborhood feature of the neighboring points corresponding to the central point of the i-th local feature; represents the weighted feature corresponding to the central point of the i-th local feature and its neighboring points; g represents the attention function; W represents the attention weight.

8. The method for semantic segmentation of power corridor point cloud based on RandLA-Net according to claim 1, characterized in that, Adjust the output probability through class weights, smooth the adjusted output probability based on spatial continuity to obtain confidence, set a screening threshold, re-determine the label class for the target data with a confidence less than the screening threshold, and complete the training to obtain the RandLA-Net model, including: Determine the label class based on the power corridor point cloud dataset, calculate the class weights, and adjust the output probability through the class weights; Statistically calculate the occurrence frequencies of the predicted label classes and neighborhood label classes of the target data corresponding to each output probability to obtain the confidence; If the predicted label class and the neighborhood label class of the target data with a confidence less than the screening threshold are inconsistent, correct the predicted label class to the neighborhood label class and re-determine the label class according to the neighborhood information.

9. The method for semantic segmentation of power corridor point cloud based on RandLA-Net according to claim 1, characterized in that Input the original point cloud data to be measured into the RandLA-Net model to obtain the segmentation result, display the segmentation result and generate an analysis report, including: Input the original point cloud data to be measured into the RandLA-Net model to obtain the segmentation result, display the label class determined by the segmentation result using color information, and statistically calculate any one of the data information such as the number, density, feature distribution, and distribution range of each point cloud in the label class to generate an analysis report.

10. The semantic segmentation system of power corridor point cloud based on RandLA-Net is characterized in that, The system includes: A data acquisition and preprocessing module, which is used to: collect the original point cloud data of the power corridor, perform preprocessing, and construct a power corridor point cloud dataset; A RandLA-Net semantic segmentation module, which is used to: obtain a training model, randomly sample the power corridor point cloud dataset to generate target data, extract local features from the target data based on the training model and transmit the local features to the encoder for aggregation, and transmit the aggregated local features to the decoder to obtain the output probability of the target data belonging to each label class; A classification result optimization module, which is used to: adjust the output probability through class weights, smooth the adjusted output probability based on spatial continuity to obtain confidence, set a screening threshold, re-determine the label class for the target data with a confidence less than the screening threshold, and complete the training to obtain the RandLA-Net model; A result visualization module, which is used to: input the original point cloud data to be measured into the RandLA-Net model to obtain the segmentation result, display the segmentation result and generate an analysis report.

Images