An elongated member point cloud fine segmentation method based on superpoint enhanced transformer

Abstract

The application discloses an elongated member point cloud fine segmentation method based on a super point enhanced Transformer, comprising the following steps: preprocessing the point cloud; feature coding is performed on the preprocessed point cloud to extract multi-scale geometric features; the point cloud is divided into multiple super point regions, and feature aggregation is performed on the super points; the super points and their aggregated features are input into a Transformer module containing a self-attention mechanism to output global context features; the local point-level features of the super points and their global context features are dynamically weighted and fused according to complexity to output fusion structure features; semantic segmentation convolution kernels and instance segmentation convolution kernels are generated to output point-level semantic labels and candidate instance masks; the candidate instance masks are trained and reasoned to output segmentation results. The application realizes high-precision semantic, instance and panoramic segmentation of spatially dense and severely mutually interfering elongated members, and improves the structure consistency and engineering applicability of the segmentation results.

Description

TECHNICAL FIELD

[0001] The present application relates to computer vision technology, in particular to an elongated member point cloud fine segmentation method based on super point enhanced Transformer. BACKGROUND

[0002] With the continuous improvement of the intelligentization and digitization level of the construction industry, three-dimensional point cloud data has been widely used in construction site modeling, structure detection and safety compliance analysis, etc. due to its ability to efficiently record the site space structure and provide rich geometric and semantic information. Scaffolding, reinforcement, section steel and other elongated components are numerous in construction engineering and are key to structure and function. However, due to their elongated shape, dense arrangement and complex spatial interference, traditional manual inspection methods have low efficiency, strong subjectivity and high omission rate, and cannot meet the high-precision requirements in large-scale and complex scenarios. Therefore, it is urgent to rely on automated and intelligent three-dimensional point cloud segmentation and recognition methods to achieve accurate detection, instance-level segmentation and positioning of elongated components, and to provide basic data support for subsequent structure health monitoring and safety evaluation.

[0003] Currently, the main technical routes for target component segmentation and recognition in three-dimensional point cloud include rule-based and traditional image processing segmentation methods, projection and voxel-based geometric processing methods, and deep learning-based point cloud segmentation networks. Traditional methods rely on point cloud projection, voxelization, boundary extraction and manual threshold setting, and have limited ability to express target geometric features, especially difficult to distinguish spatially adjacent and morphologically similar elongated rods, leading to instance confusion or under-segmentation. In recent years, point cloud deep learning models such as PointNet have improved the ability of point cloud semantic segmentation through end-to-end training, but when dealing with complex structures and similar instances of elongated targets, they still face problems such as insufficient global semantic modeling, local feature loss, and limited instance separation resolution. Furthermore, although Transformer-based networks enhance the global dependency modeling of point clouds, they lack structural constraints on the main axis direction of elongated structures and the adjacency relationship between instances, which can easily cause mask adhesion and instance misjudgment. Therefore, existing technologies cannot achieve high-precision segmentation of scaffolding and other elongated components in high-density and complex background construction scenes, and an innovative segmentation method that integrates super point expression, global modeling and structural constraints is needed to overcome the above shortcomings. SUMMARY

[0004] In order to overcome the problems of structure adhesion, instance confusion and insufficient principal axis direction constraint existing in the prior art when the point cloud of the slender member such as the scaffold is segmented in the construction scene, the purpose of the present application is to provide a slender member point cloud fine segmentation method based on super point enhanced Transformer, which realizes high-precision semantic, instance and panoramic segmentation of the spatially dense and severely interfered slender member, and improves the structure consistency and engineering applicability of the segmentation result.

[0005] The purpose of the present application is realized by the following technical solutions:

[0006] The present application provides a slender member point cloud fine segmentation method based on super point enhanced Transformer, comprising the following steps:

[0007] S1 obtains the original three-dimensional point cloud data of the construction site of the slender member, and pre-processes the point cloud;

[0008] S2 uses a sparse three-dimensional convolutional network to encode the features of the pre-processed point cloud, and extracts multi-scale geometric features; the multi-scale geometric features include point-level geometric shape, normal vector direction and spatial scale;

[0009] S3 based on the point-level features and the spatial adjacency relationship, constructs a point-to-point graph structure, and performs region clustering according to the geometric consistency principle, divides the point cloud into multiple super point regions, and aggregates the features of each super point to form a super point-level structure expression;

[0010] S4 inputs the super point and its aggregated features into a Transformer module containing a self-attention mechanism, captures the global dependency relationship between the super points, and outputs the global context features;

[0011] S5 adopts an adaptive weighted fusion mechanism based on the geometric complexity of the super point, dynamically weights and fuses the local point-level features of each super point and its global context features according to the complexity, and outputs the fusion structure features of each super point;

[0012] S6 inputs the fusion structure features into a query decoder, generates semantic segmentation convolution kernels and instance segmentation convolution kernels using a multi-layer Transformer structure, and outputs point-level semantic labels and multiple candidate instance masks respectively, completing preliminary mask prediction;

[0013] S7 trains and reasons the candidate instance masks obtained in step S6, and outputs the final segmentation result;

[0014] In the training stage, the principal axis direction and the center point are fitted for each candidate instance mask, based on the fitted principal axis, the minimum distance of all points in the mask to the principal axis and the nearest distance of the principal axes between instances are calculated respectively, and a shape constraint loss combining the two is introduced, and the instance segmentation result is jointly optimized in terms of physical rationality and spatial separation.

[0015] In the inference stage, the candidate instance mask and its principal axis parameters obtained by the preliminary prediction are utilized to make collinearity judgment and spatial relationship analysis on the mask principal axis, and the mask refinement optimization and boundary repair are performed on the adhered or broken instances, so as to output the final segmentation result with both structural continuity and instance boundary integrity.

[0016] Preferably, in step S5, an adaptive weighted fusion mechanism based on the geometric complexity of the super point is adopted to dynamically weight and fuse the local point-level feature of each super point and its global context feature according to the complexity, specifically as follows:

[0017] For a super point s i , the point-level features of all points inside the super point are aggregated to obtain the local point-level feature F L (s i ):

[0018]

[0019] wherein F UNet (p j ) is the geometric feature of the jth point p i inside the super point s j , and |s i | is the number of points included in the super point s i .

[0020] The geometric complexity p i of the super point s i is calculated as follows:

[0021]

[0022] wherein c is the centroid of the super point s i , and p j is a point inside the super point;

[0023] According to the complexity p i , the fusion weight is adaptively calculated as follows:

[0024] β i =σ(γ·ρ i )

[0025] wherein s is the Sigmoid function, and g is an adjustable hyperparameter;

[0026] The local point-level feature F L (s i ) and the global context feature F ST (s i ) are fused to obtain the final super point fusion structure feature F LS (s i):

[0027] F LS (s i )=β i ·F L (s i )+(1-β i )F ST (s i )。

[0028] Preferably, the Transformer module of the self-attention mechanism in step S4 realizes global aggregation of hyperpoint features and modeling of long-distance dependency relationships through multi-layer multi-head self-attention units, and the outputs of each layer are fused across layers through residual connection to realize efficient flow and deep abstraction of information.

[0029] Preferably, in the training phase, the shape constraint loss is calculated as follows:

[0030] For each instance mask, the principal axis direction and the center c q After fitting, the minimum distance from each point in the mask to the principal axis is calculated:

[0031]

[0032] where p i is the point coordinate, c q is the principal axis center, is the principal axis direction;

[0033] The cylindrical shape constraint loss is defined as:

[0034]

[0035] where r is the theoretical radius of the rod, and n is the number of points in the mask.

[0036] The minimum distance constraint loss between instances is defined as:

[0037]

[0038] where D min is the minimum allowed distance between instances, is an indicator function that takes a value of 1 when the condition in the parentheses is true, and 0 otherwise.

[0039] The total shape constraint loss is

[0040]

[0041] where IoU is the overlap weight of the segmentation mask and the real mask.

[0042] Preferably, step S7 uses the candidate instance mask and its principal axis parameters obtained by preliminary prediction to perform collinearity judgment and spatial relationship analysis on the mask principal axis, and performs mask refinement optimization and boundary repair on the adhesion or broken instance, specifically including:

[0043] S71 Initial mask output and score screening, output the candidate instance mask, and select the preliminary segmentation result according to the score threshold and spatial distribution;

[0044] S72 Principal axis fitting and spatial structure analysis:

[0045] Principal axis fitting: for each candidate instance mask, collect the spatial coordinates of all points, fit the principal axis direction and center point, and calculate the principal axis length, endpoint coordinates and other attributes;

[0046] Spatial structure feature extraction: based on the principal axis parameters, judge the spatial proximity and arrangement features between instances;

[0047] S73 Splitting and merging strategy for adhesion or broken instance:

[0048] For adhesion instances, if the included angle between the two mask principal axes is less than a set threshold and the distance between the end points is less than a distance threshold, the mask is fused into the same instance;

[0049] For broken instances, if the included angle between the principal axes is greater than a threshold or the distance between the principal axis end points exceeds a distance threshold, the mask is split into multiple sub-instances;

[0050] S74 Mask boundary fine-tuning and label correction;

[0051] S75 Panoptic label mapping and final output of semantic, instance and panoptic segmentation: integrate instance mask and semantic segmentation output, assign a panoptic label to each point, and output structured segmentation result data.

[0052] Preferably, the mask boundary fine-tuning of step S74 is specifically: for all instance boundaries, based on point-level mask score and spatial distribution, filter out low-confidence isolated points, and expand or shrink the instance boundary to fit the real rod structure.

[0053] Preferably, the label correction of step S74 is specifically: check the consistency of instance assignment and semantic label, and correct the instance ID confusion or semantic label anomaly caused by adhesion / breakage.

[0054] The application also provides a computer program product, comprising a computer program which, when executed by a processor, implements the steps of the method for fine segmentation of point clouds of elongated members based on superpoint enhanced Transformer.

[0055] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method for fine segmentation of an elongated member point cloud based on a superpoint enhanced Transformer.

[0056] The application further provides a computer device, which comprises a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the steps of the method for fine segmentation of an elongated member point cloud based on a superpoint enhanced Transformer.

[0057] Compared with the prior art, the application has the following advantages and beneficial effects:

[0058] (1) The method for fine segmentation of an elongated member point cloud based on a superpoint enhanced Transformer can solve the problem of how to adaptively fuse global structure perception and local geometric details in three-dimensional point cloud segmentation, and a dynamic feature weighting fusion method based on superpoint density and geometric complexity is proposed. Each superpoint can adaptively perceive whether it needs more global information or more local details, so that the segmentation model can realize robust and high-precision segmentation under high-difficulty and complex structures.

[0059] (2) The method for fine segmentation of an elongated member point cloud based on a superpoint enhanced Transformer can solve the problem of the spatial structure characteristics of scaffolds and other elongated rods, and a geometric constraint loss based on the principal axis direction and radius constraint is proposed. The model is forced to divide each instance in space to be closer to the ideal physical shape (cylindrical rod), and the geometric and physical knowledge is directly introduced into the deep segmentation, which effectively enhances the engineering value and geometric precision of the segmentation.

[0060] (3) The method for fine segmentation of an elongated member point cloud based on a superpoint enhanced Transformer proposes an instance mask post-processing strategy based on principal axis direction analysis and spatial relationship, which can geometrically optimize and finely correct the common instance adhesion and fracture problems in the inference stage, significantly improve the instance boundary continuity and integrity of the final segmentation result, eliminate common adhesion / fracture errors, enable the model to also “intelligently correct errors” in the inference stage, ensure that the segmentation output result is highly consistent with the actual physical structure, and meet the fine needs of structure recognition and compliance detection. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 The flowchart of the method for fine segmentation of an elongated member point cloud based on a superpoint enhanced Transformer of the embodiment of the application is shown.

[0062] Figure 2 The specific flowchart of step S5 in the embodiment of the application is shown.

[0063] Figure 3 A flowchart of the calculation process of the shape constraint loss in the embodiment of the application.

[0064] Figure 4 A flowchart of the inference stage in the embodiment of the application. DETAILED DESCRIPTION

[0065] The application will be further described in conjunction with the embodiments. However, the embodiments of the application are not limited thereto.

[0066] As shown in the figure, the slender member point cloud fine segmentation method based on the superpoint enhanced Transformer in the embodiment of the application comprises the following steps: Figure 1

[0067] S1: Obtain the original three-dimensional point cloud data of the construction site of the slender member, and pre-process the point cloud;

[0068] S2: Use a sparse three-dimensional convolutional network to encode the pre-processed point cloud and extract multi-scale geometric features; the multi-scale geometric features include point-level geometric shapes, normal vector directions and spatial scales;

[0069] S3: Based on the point-level features and the spatial adjacency relationship, a point graph structure is constructed, and region clustering is performed according to the geometric consistency principle, so as to divide the point cloud into a plurality of superpoint regions, aggregate the features of each superpoint, and form a superpoint-level structure expression;

[0070] S4: Input the superpoint and its aggregated features into a Transformer module containing a self-attention mechanism to capture the global dependency relationship between the superpoints, and output the global context features;

[0071] S5: An adaptive weighted fusion mechanism based on the geometric complexity of the superpoint is adopted to dynamically weight and fuse the local point-level features of each superpoint and its global context features according to the complexity, and output the fusion structure features of each superpoint;

[0072] S6: Input the fusion structure features into a query decoder to generate semantic segmentation convolution kernels and instance segmentation convolution kernels using a multi-layer Transformer structure, and output point-level semantic labels and a plurality of candidate instance masks respectively to complete preliminary mask prediction;

[0073] S7: Train and infer the candidate instance masks obtained in step S6 to obtain the final segmentation result.

[0074] ​In the training stage, first, the main axis direction and the center point of each candidate instance mask are fitted, based on the fitted main axis, the minimum distance of all points in the mask to the main axis and the nearest distance of the main axes between instances are calculated respectively, a shape constraint loss combining the two is introduced, and the instance segmentation result is jointly optimized in terms of physical rationality and spatial separation;

[0075] In the inference stage, the candidate instance mask and the main axis parameters obtained by preliminary prediction are combined with the adjacency relationship between super points, the collinearity of the mask main axis is judged and the spatial relationship is analyzed, the mask refinement optimization and boundary repair are performed on the adhered or broken instances, and the final segmentation result with both structural continuity and instance boundary integrity is output.

[0076] In some embodiments of the present application, the preprocessing of step S1 includes irrelevant scene rejection and downsampling operation; wherein the irrelevant scene rejection operation uses the CloudCompare tool to open the original point cloud data, selects the scaffold structure area, and rejects all irrelevant points such as buildings, ground, workers and instruments outside the scene, and only retains the required scaffold target area. The downsampling operation adopts a random downsampling function to uniformly reduce the point cloud density of each scene to 30,000-400,000 points, which facilitates subsequent network input and improves training efficiency.

[0077] In some embodiments of the present application, the sparse three-dimensional convolutional network of step S2 uses a sparse 3DU-Net as an encoder-decoder backbone, and the structure includes a plurality of down-sampling and up-sampling layers. In the down-sampling stage, the resolution is reduced and high-level features are extracted through sparse voxel convolution and pooling operation; in the up-sampling stage, the resolution is restored and multi-scale information is fused.

[0078] In some embodiments of the present application, in step S3, neighborhood construction and feature similarity analysis (such as k-NN, threshold, etc.) are performed, and a region segmentation algorithm based on feature consistency is used to divide points with high similarity and spatial proximity into the same "super point". The super point aggregation feature not only encodes the local geometry and structure information of all points inside the super point, but also provides efficient and dimension-reduced representation for subsequent global modeling, greatly accelerating subsequent network calculation and reducing storage pressure.

[0079] In some embodiments of the present application, the super point and its aggregation feature are input into a Transformer module containing a self-attention mechanism to capture the global dependency relationship between super points, and output global context features, which specifically include the following steps:

[0080] The super point feature obtained in step S3 is organized into a super point feature matrix according to the super point order;

[0081] The super point adjacency relationship is encoded into an adjacency matrix or an adjacency edge set using the super point graph structure obtained in step S3, which provides topological prior for subsequent structure modeling.

[0082] The superpoint feature matrix H is input into the Transformer module to perform multi-layer self-attention (Self-Attention) and feedforward neural network layer stacking;

[0083] For each layer, the self-attention between all superpoints is calculated:

[0084]

[0085] where Q = HW Q , K = HW K , V = HW V are the query, key and value features of the superpoints, respectively, W Q , W K , W V are the linear transformation weight parameters of the query, key and value, respectively, H is the superpoint feature matrix, and d k is the dimension of the query / key vector.

[0086] In some embodiments of the present application, the Transformer module of the self-attention mechanism in step S4 realizes global aggregation of superpoint features and long-distance dependency modeling through multi-layer multi-head self-attention units, the outputs of each layer are fused across layers through residual connection to realize efficient flow and deep abstraction of information.

[0087] In the above multi-head self-attention mechanism, each superpoint can not only "focus" on adjacent superpoints, but also "focus" on structurally distant but semantically related superpoints in the scene to realize global dependency modeling; multi-head parallel implementation, each head can learn different structural relationships and spatial dependencies. The output of each layer is the superpoint feature after global enhancement, and all layers can be stacked or only the output of the last layer is used; the output feature vector of each superpoint has encoded the global semantic relationship and spatial structure feature of the entire scene, providing stronger discriminative ability for subsequent dynamic feature fusion and segmentation prediction.

[0088] The Transformer module of the self-attention mechanism in the embodiment is a special module for modeling the superpoint global structure relationship, overcomes the limitations brought by pure KNN or local clustering, and enables each superpoint to fuse the context structure information from the global scene. The global information interaction between superpoints plays a key role in instance segmentation of complex structures such as scaffolds and differentiation of spatially adherent areas. Through the global modeling based on the Transformer, the feature of each superpoint has both local information and fused global scene context. The biggest difference between it and the ordinary Transformer is that the self-attention of it is completely for the structure interaction of “superpoint-superpoint”, not point-level features, nor subsequent “mask query” or “decoding” stages. The design intention is to enable superpoints that are structurally adjacent or spatially related to directly transmit semantics through Attention (for example, two superpoints that are far away but belong to the same rod can be connected through Attention, helping to distinguish instance adhesion and maintain structure), which is particularly important for slender (emphasis on long) rods, because if only the method of identifying the center point of the target object is used, the point cloud information of the points far away from the center point at both ends of the long rod will inevitably be ignored.

[0089] In some embodiments of the application, in step S5, an adaptive weighted fusion mechanism based on superpoint geometric complexity is proposed to solve the problem of how to adaptively fuse global structure perception and local geometric details in three-dimensional point cloud segmentation. The adaptive weighted fusion mechanism based on superpoint geometric complexity dynamically weights and fuses the local point-level features of each superpoint and its global context features according to the complexity, as shown in Figure 2 The specific steps are as follows:

[0090] For a superpoint s i , aggregate the point-level features of all points inside it to calculate the local point-level feature F L (s i ):

[0091]

[0092] Where F UNet (p j ) is the geometric feature of the jth point p i inside the superpoint s j , and |s i | is the number of points included in the superpoint s i .

[0093] Calculate the geometric complexity p i of the superpoint s i :

[0094]

[0095] wherein, is the centroid of the super point s i , p j is a point inside the super point;

[0096] According to the complexity p i , the fusion weight b i is adaptively calculated:

[0097] b i = s (g * p i )

[0098] wherein s is a Sigmoid function, and g is an adjustable hyperparameter;

[0099] The local point-level feature F L (s i ) and the global context feature F ST (s i ) are fused to obtain the final super point fusion structure feature F LS (s i ):

[0100] F LS (s i ) = b i * F L (s i ) + (1-b i ) F ST (s i ).

[0101] The adaptive weighting fusion mechanism based on the super point geometric complexity of the embodiment can dynamically calculate the global and local feature fusion weights according to the density and geometric complexity of the internal points of each super point, so as to avoid fixed feature weighting proportion. The global feature is automatically strengthened in the slender and simple structure region (the structure consistency is improved); the local feature proportion is automatically improved in the complex, dense or adhesion region (the boundary and details are enhanced), so that the segmentation accuracy and structure continuity of the slender rod in the spatial complex scene are significantly improved; the generalization and adaptation ability of the model to different types and different difficulty scenes is improved, and the model is significantly better than the traditional scheme using only global or local features.

[0102] In some embodiments of the application, in step S7, the loss function of the training stage adopts a multi-task joint optimization strategy, including: instance classification loss, semantic segmentation loss, super point mask matching loss, shape constraint loss, each loss is weighted and summed according to the set weight coefficient to form the overall training objective function. Specifically, the instance classification loss adopts cross-entropy loss, the semantic segmentation loss adopts binary cross-entropy loss, the super point mask matching loss is the sum of binary cross-entropy and Dice loss, and the shape constraint loss is based on the consistency of the principal axis fitting and the distance from the point to the principal axis.

[0103] Instance classification loss: for each candidate instance, the cross-entropy loss is used to supervise the classification probability, so as to optimize the accuracy of instance category prediction.

[0104] Semantic segmentation loss: for point-level semantic labels, the binary cross-entropy loss is used for supervision to optimize the semantic segmentation accuracy.

[0105] Superpoint mask matching loss: for superpoint mask prediction, the binary cross-entropy loss is used and Dice loss are added to balance the overall coverage and boundary accuracy of the mask.

[0106] Shape constraint loss: for each candidate instance mask, the main axis direction and center point are fitted, based on the fitted main axis, the minimum distance of all points in the mask to the main axis and the nearest distance of the main axes between instances are calculated, and a shape constraint loss combining the two is introduced to jointly optimize the physical rationality and spatial separation of the instance segmentation result; as shown in Figure 3 the specific calculation process of the shape constraint loss is as follows:

[0107] For each instance mask, the main axis direction and the center c q After fitting, the minimum distance of each point in the mask to the main axis is:

[0108]

[0109] where p i is the point coordinate, c q is the center of the main axis, is the direction of the main axis;

[0110] The cylindrical shape constraint loss is defined as:

[0111]

[0112] where r is the theoretical radius of the rod, and n is the number of points in the mask.

[0113] The minimum distance constraint loss between instances is defined as:

[0114]

[0115] where D min is the minimum allowed distance between instances, is an indicator function that takes the value 1 when the condition in the parentheses is true, and 0 otherwise;

[0116] The total shape constraint loss is

[0117]

[0118] where IoU is the overlap weight of the segmentation mask and the real mask.

[0119] The shape constraint loss mechanism of the embodiment fits a principal axis (such as SVD fitting) for each segmentation instance mask, minimizes the deviation of the distance from the principal axis to the theoretical radius, and realizes the convergence of the spatial structure of the mask to the ideal rod shape; effectively constrains the segmentation result to prevent non-physical adhesion, skew or breakage, and improves the geometric rationality and engineering interpretability of instance segmentation; significantly improves the consistency of the segmentation shape of the slender target, reduces the misjudgment and post-processing burden caused by irregular segmentation; supports the active learning of the model on the physical structure "shape prior", so that the segmentation result is more in line with the actual engineering application requirements.

[0120] Loss weighting and multi-task combination: the final total loss is the weighted sum of the above items:

[0121]

[0122] where, is the instance classification loss, is the semantic segmentation loss, and are the binary cross-entropy loss and the Dice loss, respectively, is the shape constraint loss, and λ is the instance classification loss weighting coefficient. The weights of each loss term can be set or adaptively adjusted according to experimental experience or cross-validation to balance the training effect of different segmentation tasks.

[0123] In some embodiments of the application, in step S7, the mask matching mechanism, the training process, the hyperparameter setting and the convergence criterion are included.

[0124] The mask matching mechanism is: the Hungarian algorithm is used to establish a one-to-one correspondence between the predicted mask and the real mask, minimize the assignment cost (such as binary mask cross-entropy), and ensure the accurate correspondence of the label and the instance prediction; the specific process is:

[0125] Calculate the loss cost matrix of all predicted masks and all real masks;

[0126] Find a one-to-one assignment scheme with the minimum total cost by the Hungarian matching algorithm;

[0127] The loss function only optimizes the successfully assigned mask pair, and the unmatched part can be counted as false positive / false negative processing.

[0128] The training process includes:

[0129] Read the point cloud data that has been processed and labeled by batch;

[0130] Randomly sample a training batch and input it into the sparse U-Net backbone, super point module, Transformer, dynamic pooling, and Query Decoder model structures;

[0131] Forward propagation to obtain all outputs (semantics, instance masks, etc.);

[0132] Calculate the total loss according to the multi-task loss function and matching mechanism, and back-propagate the optimization parameters.

[0133] Common hyperparameter settings for the training process:

[0134] Batch size: 2-8 (depending on the memory and scene size)

[0135] Learning rate: 1e-3 (with Cosine Annealing or Step Decay scheduling)

[0136] Optimizer: AdamW

[0137] Dropout / normalization parameters depend on the main network settings.

[0138] The convergence criterion is:

[0139] The validation set mIoU, AP, PQ, and other indicators are steadily improved, and the loss curve tends to be smooth and convergent.

[0140] In some embodiments of the present application, in step S7, during the inference stage, the candidate instance mask and its principal axis parameters obtained through preliminary prediction are used to perform collinearity judgment and spatial relationship analysis on the mask principal axis, and mask refinement optimization and boundary repair are performed on the adhesion or broken instances, and the final segmentation result with continuous structure and complete instance boundary integrity is output, as shown in Figure 4 The specific implementation steps are as follows:

[0141] S71 Initial mask output and score screening: output the candidate instance mask, select the preliminary segmentation result according to the score threshold and spatial distribution;

[0142] S72 Principal axis fitting and spatial structure analysis:

[0143] Principal axis fitting: collect the spatial coordinates of all points in each candidate instance mask, fit the principal axis direction and center point, and calculate the principal axis length, endpoint coordinates, and other attributes;

[0144] Spatial structure feature extraction: based on the principal axis parameters, judge the spatial proximity and arrangement features between instances;

[0145] S73 Splitting and merging strategy for sticking or broken instances:

[0146] For sticking instances, if the angle between the two mask principal axes is less than a set threshold and the distance between the end points is less than a distance threshold, the masks are merged into the same instance.

[0147] For broken instances, if the angle between the principal axes is greater than a threshold or the distance between the principal axis end points exceeds a distance threshold, the masks are split into multiple sub-instances.

[0148] S74 Fine-tuning of mask boundaries and label correction; the fine-tuning of the mask boundary is specifically: based on the point-level mask score and spatial distribution, filtering out low-confidence isolated points, expanding or shrinking the instance boundary to closely fit the real bar structure; the label correction is specifically: checking the consistency of instance assignment and semantic label, correcting the instance ID confusion or semantic label abnormality caused by breaking / sticking;

[0149] S75 Panoptic label mapping and final output of semantic, instance and panoptic segmentation: integrating instance mask and semantic segmentation output, assigning a panoptic label to each point, and outputting structured segmentation result data.

[0150] The inference stage fine segmentation of the embodiment proposes an instance mask post-processing strategy based on principal axis direction analysis and spatial relationship, which geometrically optimizes and finely corrects the common instance sticking and breaking problems in the inference stage; the direction and spatial position of the fitted principal axis are used to automatically determine the merging (colinear and close distance) or splitting (large angle, broken center of gravity) of adjacent masks, accurately repairing the instance boundary and spatial structure; combined with the superpoint adjacency relationship, the sticking, crossing and disconnected bars are adaptively optimized, completely solving the problem of incoherent segmentation in high-density and complex scenes; significantly improving the instance boundary continuity and integrity of the final segmentation result, eliminating common sticking / broken errors; enabling the model to "intelligently correct" in the inference stage, ensuring that the segmentation output is highly consistent with the actual physical structure, and meeting the fine needs of structure recognition and compliance detection.

[0151] Some embodiments of the present application also provide a computer program product comprising a computer program which, when executed by a processor, implements the steps of the superpoint-enhanced Transformer-based fine segmentation method for point clouds of elongated members.

[0152] Some embodiments of the present application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the superpoint-enhanced Transformer-based fine segmentation method for point clouds of elongated members.

[0153] Some embodiments of the present application also provide a computer device comprising a memory, a processor and a computer program stored on the memory, wherein the processor executes the computer program to implement the steps of the method for fine segmentation of an elongated member point cloud based on a hyperpoint enhanced Transformer.

[0154] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement modes and shall be included in the protection scope of the present application.

Claims

1. An ultra-point-enhanced Transformer-based fine segmentation method for an elongated member point cloud, characterized in that, The method comprises the following steps: S1: obtaining original three-dimensional point cloud data of a construction site of an elongated member, and preprocessing the point cloud; S2: using a sparse three-dimensional convolution network to encode features of the preprocessed point cloud, and extracting multi-scale geometric features; the multi-scale geometric features include point-level geometric shapes, normal vector directions, and spatial scales; S3: constructing a point-to-point graph structure based on point-level features and spatial adjacency relationships, performing regional clustering according to a geometric consistency principle, dividing the point cloud into a plurality of superpoint regions, and performing feature aggregation on each superpoint to form a superpoint-level structure expression; S4: inputting the superpoint and the aggregated features into a Transformer module containing a self-attention mechanism to capture global dependency relationships between superpoints, and outputting global context features; S5: using an adaptive weighted fusion mechanism based on superpoint geometric complexity to dynamically weight and fuse local point-level features of each superpoint and global context features thereof according to complexity, and outputting fusion structure features of each superpoint; S6: inputting the fusion structure features into a query decoder, using a multi-layer Transformer structure to generate semantic segmentation convolution kernels and instance segmentation convolution kernels, and outputting point-level semantic labels and a plurality of candidate instance masks respectively to complete preliminary mask prediction; S7: training and reasoning the candidate instance masks obtained in step S6 to output a final segmentation result; In the training stage, a main axis direction and a center point are fitted for each candidate instance mask, the minimum distance from all points in the mask to the main axis and the nearest distance between the main axes of the instances are calculated based on the fitted main axis, a shape constraint loss combining the two is introduced, and the instance segmentation result is jointly optimized in terms of physical reasonableness and spatial separation; In the reasoning stage, the main axis of the mask is subjected to collinearity judgment and spatial relationship analysis based on the main axis parameters of the candidate instance masks obtained through preliminary prediction and the adjacency relationships between the superpoints, mask refinement optimization and boundary repair are performed on the instances that are adhered or broken, and a final segmentation result with both structure continuity and instance boundary integrity is output.

2. The superpoint-enhanced Transformer-based elongated member point cloud fine segmentation method according to claim 1, wherein, In step S5, an adaptive weighted fusion mechanism based on superpoint geometric complexity is used to dynamically weight and fuse local point-level features of each superpoint and global context features thereof according to complexity, specifically as follows: For superpoint s i , aggregate the point-level features of all points inside it, and calculate the local point-level feature F L (s i ): Among them, F UNet (p j ) is super point s i The j-th point p inside j Geometric features, |s i |For super points s i The number of points contained therein; Computing the geometry complexity p of superpoint s i i :​ in, For the super point s i The center of mass, p j It is a point inside the superpoint; According to the complexity p i , the adaptive calculation fusion weight: β i = σ(γ·ρ i ) wherein σ is a Sigmoid function and γ is an adjustable hyperparameter; Fusion local point-level feature F L (s i ) and global context feature F ST (s i ) to obtain the final superpoint fusion structure feature F LS (s i ): F LS (s i )=β i ·F L (s i )+(1-β i )F ST (s i )。 3. The superpoint-enhanced Transformer-based slender member point cloud fine segmentation method according to claim 1, wherein, In step S4, the Transformer module with the self-attention mechanism models the global aggregation of superpoint features and long-distance dependency relationships through a plurality of multi-head self-attention units, the outputs of the layers are fused across the layers through residual connection to realize efficient flow and deep abstraction of information.

4. The superpoint-enhanced Transformer-based elongated member point cloud fine segmentation method according to claim 1, wherein, In the training stage, the shape constraint loss is calculated as follows: For each instance mask, the principal axis direction and center c q After the fitting is complete, the minimum distance of each point within the mask to the principal axis is calculated: where p i is a point coordinate, c q is the center of the principal axis, is the principal axis direction; The cylindrical shape constraint loss is defined as: wherein r is the theoretical radius of the rod and n is the number of points in the mask; The minimum distance constraint loss between instances is defined as: where D min is the minimum allowed distance between instances, is an indicator function that takes the value 1 when the condition in the parentheses is true and 0 otherwise. The total shape constraint loss is wherein IoU is the overlap weight of the segmentation mask and the real mask.

5. The superpoint-enhanced Transformer-based elongated member point cloud fine segmentation method according to claim 1, wherein, In step S7, the main axis of the mask is subjected to collinearity judgment and spatial relationship analysis based on the main axis parameters of the candidate instance masks obtained through preliminary prediction and the adjacency relationships between the superpoints, mask refinement optimization and boundary repair are performed on the instances that are adhered or broken, specifically as follows: S71 Initial mask output and score screening, output candidate instance masks, select preliminary segmentation results according to score threshold and spatial distribution; S72 Principal axis fitting and spatial structure analysis: Principal axis fitting: for each candidate instance mask, collect the spatial coordinates of all points, fit the principal axis direction and center point, and calculate the principal axis length, endpoint coordinates and other attributes; Spatial structure feature extraction: based on the principal axis parameters, judge the spatial proximity and arrangement characteristics between instances; S73 Splitting and merging strategy for adherent or broken instances: For adherent instances, if the angle between the two mask principal axes is less than the set threshold and the distance between the endpoints is less than the distance threshold, the masks are merged into the same instance; For broken instances, if the angle between the principal axes is greater than the threshold or the distance between the principal axis endpoints exceeds the distance threshold, the mask is split into multiple sub-instances; S74 Mask boundary fine-tuning and label correction; S75 Panoramic label mapping and final output of semantic, instance and panoramic segmentation: integrate instance masks and semantic segmentation output, assign panoramic labels to each point, and output structured segmentation result data.

6. The superpoint-enhanced Transformer-based elongated member point cloud fine segmentation method according to claim 5, characterized in that, The mask boundary fine-tuning of step S74 is specifically: for all instance boundaries, based on point-level mask score and spatial distribution, filter out low-confidence isolated points, and expand or shrink the instance boundary to fit the real bar structure.

7. The superpoint-enhanced Transformer-based elongated member point cloud fine segmentation method according to claim 6, characterized in that, The label correction of step S74 is specifically: check the consistency of instance assignment and semantic label, and correct the instance ID confusion or semantic label anomaly caused by fragmentation / adhesion.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the superpoint enhanced Transformer-based fine segmentation method of the elongated member point cloud according to any one of claims 1-7.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the superpoint enhanced Transformer-based fine segmentation method of the elongated member point cloud according to any one of claims 1-7.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program comprises instructions that, when executed by the processor, cause the processor to perform the method of any one of claims 1-9. The processor executes the computer program to realize the steps of the superpoint enhanced Transformer-based fine segmentation method of the elongated member point cloud according to any one of claims 1-7.

Images