Distribution line point cloud panoramic segmentation method and system based on improved PTv3 model

Abstract

The invention relates to a distribution line point cloud panoramic segmentation method based on an improved PTv3 model, and the method comprises the steps: collecting and obtaining distribution network line point cloud data, and dividing the data into a training set, a verification set and a test set; semantic annotation of segmentation categories is carried out on the point cloud data set; performing instance labeling on instances belonging to the same semantic category in the point cloud data set; the method comprises the following steps: carrying out improvement on the basis of a Point Transform V3 model decoder, and introducing a double-branch decoder comprising semantics and instance branches; designing a branch interaction module to realize feature interaction of semantics and instance branches; designing a panoramic fusion module, and fusing a semantic result output by the semantic branch and an instance mask output by the instance branch; constructing an improved PTv3 model suitable for point cloud panoramic segmentation; importing the marked training and verification set data into the improved PTv3 model for training; and finally, inputting point cloud test set data into the trained model to realize distribution network overhead line point cloud panoramic segmentation.

Description

Technical Field

[0001] This invention relates to the fields of point cloud panoramic segmentation and power distribution line operation and maintenance, specifically to a method and system for power distribution line point cloud panoramic segmentation based on an improved PTv3 model. Background Technology

[0002] Electricity is a fundamental energy source in modern society, vital to national welfare and energy security. With economic and social development and rising living standards, society as a whole has placed higher demands on power supply reliability, power quality, and service levels. In recent years, driven by both energy transition and the construction of a digital China, the power distribution network is rapidly evolving from its traditional single power supply function towards active, intelligent, and flexible operation. Through the in-depth application of big data, artificial intelligence, and other next-generation information technologies, technological innovations have been achieved in operation and maintenance aspects such as distribution line condition monitoring, fault early warning, and adaptive protection, significantly improving the level of lean equipment management. The "Action Plan for High-Quality Development of Power Distribution Networks" issued by the State Council clearly proposes to promote the construction of an intelligent inspection and operation and maintenance system for distribution lines, enhancing the resilience and self-healing capabilities of the power distribution network. Against this backdrop, the power distribution system is accelerating its digital transformation, providing crucial support for building a new type of power system.

[0003] Point cloud segmentation is a fundamental task for realizing point cloud technology applications. By dividing the 3D point cloud of overhead power lines into different categories, various functions can be achieved. For example, by accurately segmenting conductors, trees, and buildings, applications such as tree hazard identification and determining safe distances for overhead line crossings can be implemented; by accurately segmenting conductors, towers, and insulators in a panoramic manner, intelligent construction and management of digital ledgers for overhead power lines can be achieved. In recent years, scholars have conducted research on many deep learning models for point cloud segmentation. Among them, the PTv3 model performs excellently in semantic segmentation tasks on public datasets, but it still cannot achieve panoramic segmentation.

[0004] Existing research on point cloud segmentation methods for power distribution lines is relatively limited, and the limited research is also limited to the extraction of partial elements from the point cloud of power distribution lines, or semantic segmentation. When dealing with large-scale power distribution line point clouds, panoramic segmentation research has more significant practical implications, but related research is lacking.

[0005] Therefore, conducting research on the panoramic segmentation of three-dimensional point clouds of overhead power distribution lines is an urgent problem to be solved by researchers in this field. Summary of the Invention

[0006] In view of this, it is necessary to provide a panoramic segmentation method for distribution line point clouds based on the improved PTv3 model, so as to realize the digital management application of large-scale distribution lines.

[0007] This invention provides a panoramic segmentation method for distribution line point clouds based on an improved PTv3 model, comprising:

[0008] Point cloud data of distribution network lines is collected and divided into training set, validation set and test set;

[0009] Semantic labeling of segmentation categories is performed on the point cloud dataset of overhead power distribution lines;

[0010] Instance annotation is performed on instances belonging to the same semantic category in the point cloud dataset of overhead distribution lines;

[0011] Improvements were made to the original decoder of the Point Transformer V3 model by introducing a dual-branch decoder that includes semantic branches and instance branches;

[0012] Design a branch interaction module to enable feature interaction between semantic branches and instance branches;

[0013] Design a panoramic fusion module to fuse the semantic results output by the semantic branch and the instance mask output by the instance branch;

[0014] An improved PTv3 model suitable for point cloud panoramic segmentation is constructed.

[0015] Import the labeled training and validation sets into the improved PTv3 model for training.

[0016] Finally, the test set data of the distribution network overhead line point cloud is input into the trained model to achieve panoramic segmentation of the distribution network overhead line point cloud.

[0017] Furthermore, the ratio of the training set, validation set, and test set is 8:1:1.

[0018] Furthermore, the semantic annotation method for the point cloud dataset of overhead power distribution lines involves using point cloud editing software with annotation capabilities to assign semantic category information to each point in the point cloud dataset. The categories include ten types: overhead lines, ground wires, insulators, towers, vegetation, shrubs, houses, vehicles, obstacles, and ground.

[0019] Furthermore, the instance annotation method includes:

[0020] First, the ten semantic categories are divided into the stuff category and the thing category, with the ground category being the stuff category and the other categories being the thing category.

[0021] Secondly, different instances of the same semantics within the `thing` class are marked as instances. For example, single-span three-phase overhead power distribution conductors are marked as 'conductor-1', 'conductor-2', and 'conductor-3', respectively.

[0022] Furthermore, the dual-branch decoder includes:

[0023] Semantic segmentation branch: Starting from the low-level features output by the backbone network encoder, the layer-by-layer decoder module in PTv3 is used to aggregate features and output the semantic category probability vector of each point. The category corresponding to the vector index with the highest probability is the predicted semantic category of that point. In the semantic segmentation branch, the cross-entropy loss function and the Lovasz loss function are added together as the loss function of this branch.

[0024] Instance segmentation branch: Instance queries are generated using the instance query generation method. The queries are then connected to the multi-scale features in the backbone network encoder to output instance features. These instance features are mapped to instance categories. Dot product attention is then used to generate K instance masks, where K is the maximum number of things instances defined in the initialization. In the instance segmentation branch, the Hungarian matching loss function is used as the loss function for this branch.

[0025] Furthermore, the branch interaction module allows the instance query generated by the instance segmentation branch to be connected with the semantic features in the semantic segmentation branch through cross-attention in each layer of the decoder module, thereby generating instance features and semantic features respectively. The cross-attention consists of two parts: one part is semantic guidance instance query initialization attention, and the other part is instance-aware semantic feature enhancement attention.

[0026] Furthermore, the panoramic fusion module is a post-processing module that fuses the instance branch prediction results and semantic branch prediction results of each point to obtain the fusion output of that point. The core part is a conflict resolution logic. For an instance proposal, if a sufficient number of points in the points contained in the proposal mask are predicted as the "thing" category in the semantic segmentation branch, then the instance proposal is considered valid.

[0027] Furthermore, the improved PTv3 model for point cloud panorama segmentation includes:

[0028] Topology layer: By adding linear and circular features, it enhances the feature extraction capability for point clouds of key power equipment such as insulators in power distribution line point cloud scenarios;

[0029] Serialization layer: Neighborhood search of 3D point clouds is achieved using Z-space filling curves and Hilbert space filling curves;

[0030] Encoder layer: Contains several encoders, each encoder consists of a pooling layer and several feature extraction blocks, each feature extraction block contains conditional position encoding, self-attention mechanism and multilayer perceptron;

[0031] Decoder layer: Contains several decoders, each consisting of depooling and several feature extraction blocks, and is also the main component of the semantic segmentation branch;

[0032] Instance segmentation branch: consists of a learnable query vector generation module, a category mapping module, and a mask generation module;

[0033] Branch interaction module: uses a cross-attention mechanism;

[0034] Panoramic fusion module: This is a post-processing module;

[0035] Loss function: Includes instance segmentation loss and semantic segmentation loss, and combines the two types of losses during training.

[0036] Compared with the prior art, the beneficial effects of the present invention include:

[0037] 1. The proposed panoramic segmentation method for distribution line point clouds based on the improved PTv3 model extends the original PTv3 model's limitation of only being applicable to single semantic segmentation or instance segmentation, and achieves panoramic segmentation. On this basis, the topological characteristics of distribution overhead line point clouds are considered, a topological layer is added, the feature extraction capability of key power element point clouds is improved, and the accuracy of panoramic point cloud segmentation is improved.

[0038] 2. The provided method for panoramic segmentation of distribution line point clouds based on the improved PTv3 model divides the distribution overhead line point cloud scene into ten types of point clouds, including overhead lines, ground wires, insulators, towers, vegetation, shrubs, houses, vehicles, obstacles, and ground. This method can meet the needs of intelligent large-scale tree obstacle hazard identification and large-scale digital ledger management in the process of digital transformation of distribution networks. Attached Figure Description

[0039] Figure 1 This is a flowchart of a panoramic segmentation method for power distribution line point clouds based on an improved PTv3 model.

[0040] Figure 2 This is a pseudocode image for the panoramic fusion module.

[0041] Figure 3 An improved PTv3 model framework diagram for point cloud panoramic segmentation. Detailed Implementation

[0042] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0043] Example 1: This embodiment of the invention provides a panoramic segmentation method for distribution line point clouds based on an improved PTv3 model, combined with... Figure 1 Let's take a look. Figure 1 The flowchart of the distribution line point cloud panoramic segmentation method based on the improved PTv3 model provided by the present invention includes steps S1 to S9, wherein:

[0044] This invention provides a panoramic segmentation method for distribution line point clouds based on an improved PTv3 model, comprising:

[0045] In step S1, point cloud data of distribution network lines is collected and divided into training set, validation set and test set;

[0046] In step S2, semantic annotation of the segmentation categories is performed on the point cloud dataset of the overhead distribution network lines;

[0047] In step S3, instances belonging to the same semantic category in the point cloud dataset of overhead distribution lines are labeled.

[0048] In step S4, an improvement is made to the original decoder of the Point Transformer V3 model by introducing a dual-branch decoder that includes semantic branches and instance branches.

[0049] In step S5, a branch interaction module is designed to realize feature interaction between semantic branches and instance branches;

[0050] In step S6, a panoramic fusion module is designed to fuse the semantic results output by the semantic branch and the instance mask output by the instance branch.

[0051] In step S7, an improved PTv3 model suitable for point cloud panoramic segmentation is constructed.

[0052] In step S8, the labeled training and validation set data are imported into the improved PTv3 model for training.

[0053] In step S9, the test set data of the distribution network overhead line point cloud is finally input into the trained model to achieve panoramic segmentation of the distribution network overhead line point cloud.

[0054] Preferably, the ratio of the training set, validation set, and test set is 8:1:1.

[0055] Preferably, the semantic annotation method for the point cloud dataset of overhead power distribution lines is to use point cloud editing software with annotation function to assign semantic category information to each point in the point cloud dataset. The categories include ten categories such as overhead lines, ground wires, insulators, towers, vegetation, shrubs, houses, vehicles, obstacles and ground. As a specific embodiment, in this embodiment, CloudCompare point cloud editing software is selected for point cloud semantic annotation.

[0056] Preferably, the instance annotation method includes:

[0057] First, the ten semantic categories are divided into the stuff category and the thing category, with the ground category being the stuff category and the other categories being the thing category.

[0058] Secondly, for different instances of the same semantic meaning in the `thing` class, instance labeling is performed. For example, for a single-span three-phase overhead power distribution conductor, it is labeled as 'conductor-1', 'conductor-2', and 'conductor-3' respectively. As a specific embodiment, in this embodiment, CloudCompare point cloud editing software is also selected for point cloud semantic annotation.

[0059] Preferably, the dual-branch decoder includes:

[0060] Semantic segmentation branch: Starting from the low-level features output by the backbone network encoder, feature aggregation is performed using the layer-by-layer decoder module in PTv3, outputting a semantic category probability vector for each point. The category corresponding to the vector index with the highest probability is the predicted semantic category for that point. In the semantic segmentation branch, the cross-entropy loss function and the Lovasz loss function are added together as the loss function for this branch. The calculation method of this branch loss function is as follows:

[0061]

[0062]

[0063]

[0064]

[0065] Among them, L cross Let L be the cross-entropy loss function. lovasz Let y be the Lovasz loss function. ti y is the real semantic label vector. pi To predict the semantic category vector, y ti (x c ) and y pi (x c ) represent vectors y and y respectively. ti and y pi The c-th value in.

[0066] Instance segmentation branch: Instance queries are generated using an instance query generation method. These queries are then connected to multi-scale features in the backbone network encoder to output instance features. These features are mapped to instance categories. Dot product attention is then used to generate K instance masks, where K is the maximum number of `thing` instances defined initially. In this specific implementation, K is set to 1000. In the instance segmentation branch, the Hungarian matching loss function is used as the loss function for this branch. The calculation method for this branch loss function is as follows:

[0067]

[0068]

[0069]

[0070] Among them, y i =(c i ,m i ) is the i-th real instance, c i =Φ represents an empty instance; This represents the σ(i)th prediction instance; This indicates that the predicted instance corresponds to the true class c. i The predicted probability; L mask It is a mask similarity metric, L match It is the optimal matching metric, L inst It is the instance splitting branch loss function. It is the optimal match, where m is the true binary mask. The mask probability is the predicted probability, α is the positive sample weight coefficient, which is set to 0.25 in this embodiment, and γ is the adjustment factor used to reduce the weight of easily classified samples, which is set to 2 in this embodiment.

[0071] Preferably, the branch interaction module, in each layer of the decoder module, allows the instance query generated by the instance segmentation branch to be cross-attention connected with the semantic features in the semantic segmentation branch, generating instance features and semantic features respectively. This cross-attention consists of two parts: one part is the semantic-guided instance query initialization attention, which consists of a trainable attention matrix and feature dot products, and then superimposed with max pooling and a linear MLP, resulting in the instance query initialization as follows:

[0072]

[0073] Among them, W sem2inst The first part is the trainable attention matrix, where F represents the semantic features; the second part is the instance-aware semantic feature enhancement attention, which uses the instance mask features output by the instance branch as contextual information and superimposes them onto the semantic features, as shown in the following equation:

[0074]

[0075] Among them, F e For the enhanced semantic features, M T Let Q' be the predicted mask probability after transposition, and Q′ be the query feature after interaction.

[0076] Preferably, combined with Figure 2 Let's take a look. Figure 2 This is a pseudocode image of the panoramic fusion module provided by the present invention. The panoramic fusion module is a post-processing module, comprising:

[0077] 1) Integrate the instance branch prediction results and semantic branch prediction results for each point. First, traverse the K instance proposals. In this embodiment, K is 1000.

[0078] 2) Check the confidence score of the current instance proposal. inst Does [k] exceed the threshold τ? conf In this embodiment, τ conf Take 0.5;

[0079] 3) Binarize the instance mask to generate a binary mask. k Binary mask threshold τ mask In this embodiment, it is set to 0.5;

[0080] 4) Calculate the total number N of masks in the masked region. total The number N predicted to be the "thing" category thing And the proportion ρ predicted to be the "thing" category;

[0081] 5) Determine whether the ratio ρ is greater than the threshold τ. ratio In this embodiment, τ ratio Take 0.5;

[0082] 6) Assign panoramic segmentation labels and instance IDs to instances that are determined to be true.

[0083] 7) Obtain the fused output, including panoramic segmentation labels and instance ID mapping.

[0084] 8) The core part is a conflict resolution logic. For an instance proposal, if enough points in the points contained in the proposal mask are predicted as the thing category in the semantic segmentation branch, then the instance proposal is considered valid.

[0085] Preferably, combined with Figure 3 Let's take a look. Figure 3The above is a framework diagram of the improved PTv3 model for point cloud panoramic segmentation provided by the present invention. The improved PTv3 model for point cloud panoramic segmentation includes:

[0086] Topology layer: By adding linear and circular features, the feature extraction capability for point clouds of key power equipment such as insulators in power distribution line point cloud scenarios is enhanced. Among them, the linear feature F Si ′ and circular feature F Ci The calculation method for ′ is as follows:

[0087]

[0088]

[0089] Where x, y, z represent the three-dimensional coordinates of the point cloud.

[0090] Wherein: F sxi ′、F syi ′、F szi '' represents the x, y, and z values ​​of the linear feature corresponding to the i-th point, respectively, and the subscript -1 indicates the last point; F xyzi ′、F xyi ′、F yzi ′、F xzi ′ represent the spherical feature, the circular feature projected onto the xy plane, the circular feature projected onto the yz plane, and the circular feature projected onto the xz plane, respectively.

[0091] Serialization layer: Neighborhood search of 3D point clouds is achieved using Z-space filling curves and Hilbert space filling curves;

[0092] Encoder layer: contains several encoders, each encoder consists of a pooling layer and several feature extraction blocks. Each feature extraction block contains conditional position encoding, self-attention mechanism and multilayer perceptron. In this embodiment, there are 4 encoders and 3 feature extraction blocks in each encoder layer.

[0093] Decoder layer: Contains several decoders, each of which consists of depooling and several feature extraction blocks. It is also the main component of the semantic segmentation branch. In this embodiment, there are also 4 decoders, and each decoder layer has 3 feature extraction blocks.

[0094] Instance segmentation branch: consists of a learnable query vector generation module, a category mapping module, and a mask generation module;

[0095] Branch interaction module: uses a cross-attention mechanism;

[0096] Panoramic fusion module: This is a post-processing module;

[0097] Loss function: Includes instance segmentation loss and semantic segmentation loss, and combines the two types of losses during training.

[0098] Example 2: This embodiment of the invention provides a system, or computing device, for panoramic segmentation of distribution line point clouds based on an improved PTv3 model. It is characterized by including a memory, a processing unit, and a display. The memory stores a computer program for storing and reading point cloud data from the model. The processing unit is used for training and testing the panoramic segmentation model, and the display outputs the panoramic semantic segmentation results. When the computer program is executed by the processor, it implements the aforementioned panoramic segmentation method for distribution line point clouds based on the improved PTv3 model.

[0099] This invention discloses a method and system for panoramic segmentation of distribution network point clouds based on an improved PTv3 model. The method includes: collecting and acquiring distribution network point cloud data, dividing it into training, validation, and test sets; semantically labeling the segmentation categories of the distribution network overhead line point cloud dataset; labeling instances belonging to the same semantic category in the distribution network overhead line point cloud dataset; improving the original decoder of the Point Transformer V3 model by introducing a dual-branch decoder containing semantic and instance branches; designing a branch interaction module to realize feature interaction between the semantic and instance branches; designing a panoramic fusion module to fuse the semantic results output by the semantic branch and the instance masks output by the instance branch; comprehensively constructing an improved PTv3 model suitable for panoramic point cloud segmentation; importing the labeled training and validation set data into the improved PTv3 model for training; and finally inputting the distribution network overhead line point cloud test set data into the trained model to achieve panoramic segmentation of the distribution network overhead line point cloud.

[0100] The technical solution of this invention creatively combines the topology of key power elements in the overhead power line corridor. Based on the PTv3 base model, a topology layer design is added, and the PTv3 model is improved to realize the automated panoramic segmentation function, which meets the application requirements of intelligent large-scale tree obstacle hazard identification and large-scale digital ledger management in the process of digital transformation of distribution network.

[0101] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A panoramic segmentation method for distribution line point clouds based on an improved PTv3 model, characterized in that, include: Point cloud data of distribution network lines is collected and divided into training set, validation set and test set; Semantic labeling of segmentation categories is performed on the point cloud dataset of overhead power distribution lines; Instance annotation is performed on instances belonging to the same semantic category in the point cloud dataset of overhead distribution lines; Improvements were made to the original decoder of the Point Transformer V3 model by introducing a dual-branch decoder that includes semantic branches and instance branches; Design a branch interaction module to enable feature interaction between semantic branches and instance branches; Design a panoramic fusion module to fuse the semantic results output by the semantic branch and the instance mask output by the instance branch; An improved PTv3 model suitable for point cloud panoramic segmentation is constructed. Import the labeled training and validation sets into the improved PTv3 model for training. Finally, the test set data of the distribution network overhead line point cloud is input into the trained model to achieve panoramic segmentation of the distribution network overhead line point cloud.

2. The method for panoramic segmentation of distribution line point clouds based on the improved PTv3 model according to claim 1, characterized in that, The ratio of the training set, validation set, and test set is 8:1:

1.

3. The method for panoramic segmentation of distribution line point clouds based on the improved PTv3 model according to claim 1, characterized in that, The semantic annotation method for segmenting the point cloud dataset of overhead distribution lines is as follows: using point cloud editing software with annotation function, each point in the point cloud dataset is assigned semantic category information. The categories include ten categories: overhead lines, ground wires, insulators, towers, vegetation, shrubs, houses, vehicles, obstacles, and ground.

4. The method for panoramic segmentation of distribution line point clouds based on the improved PTv3 model according to claim 3, characterized in that, Instance annotation methods include: First, the ten semantic categories are divided into the stuff category and the thing category, with the ground category being the stuff category and the other categories being the thing category. Secondly, for different instances of the same semantics in the thing class, instance marking is performed; for single-span three-phase overhead power distribution conductors, they are marked as 'conductor-1', 'conductor-2', and 'conductor-3' respectively.

5. The method for panoramic segmentation of distribution line point clouds based on the improved PTv3 model according to claim 1, characterized in that, The dual-branch decoder includes: Semantic segmentation branch: Starting from the low-level features output by the backbone network encoder, the feature aggregation is performed using the layer-by-layer decoder module in PTv3, and the semantic category probability vector of each point is output. Instance segmentation branch: Instance queries are generated using the instance query generation method. The queries are connected to the multi-scale features in the backbone network encoder to output instance features. The instance features are mapped to instance categories. Dot product attention is then used to generate K instance masks, where K is the maximum number of things instances defined in the initialization.

6. The method for panoramic segmentation of distribution line point clouds based on the improved PTv3 model according to claim 1, characterized in that, The branch interaction module enables the instance queries generated by the instance segmentation branch to be connected with the semantic features in the semantic segmentation branch through cross-attention in each layer of the decoder module, thereby generating instance features and semantic features respectively.

7. The method for panoramic segmentation of distribution line point clouds based on the improved PTv3 model according to claim 1, characterized in that, The panoramic fusion module is a post-processing module that fuses the instance branch prediction results and semantic branch prediction results of each point to obtain the fused output of that point.

8. The method for panoramic segmentation of distribution line point clouds based on the improved PTv3 model according to claim 1, characterized in that, Improved PTv3 models for point cloud panorama segmentation include: Topology layer: By adding linear and circular features, it enhances the feature extraction capability for point clouds and point clouds of key power equipment such as insulators in power distribution line point cloud scenarios; Serialization layer: Uses space-filling curves to achieve neighborhood search of 3D spatial point clouds; Encoder layer: Contains several encoders, each encoder consists of a pooling layer and several feature extraction blocks, each feature extraction block contains conditional position encoding, self-attention mechanism and multilayer perception mechanism; Decoder layer: Contains several decoders, each consisting of depooling and several feature extraction blocks, and is also the main component of the semantic segmentation branch; Instance segmentation branch: consists of a learnable query vector generation module, a category mapping module, and a mask generation module; Branch interaction module: uses a cross-attention mechanism; Panoramic fusion module: This is a post-processing module; Loss function: includes instance segmentation loss and semantic segmentation loss.

9. A system for panoramic segmentation of distribution line point clouds based on an improved PTv3 model, characterized in that, The device includes a memory, a processing unit, and a display. The memory stores a computer program for storing and reading point cloud data for the model. The processing unit is used for training and testing the panoramic segmentation model. The display is used to output and show the panoramic semantic segmentation results. When the computer program is executed by the processor, it implements the panoramic segmentation method for power distribution line point clouds based on the improved PTv3 model according to any one of claims 1-8.

Images