Self-training semantic segmentation method and device for small sample bridge point cloud, medium and equipment

Abstract

This invention relates to the field of bridge digital operation and maintenance technology, and particularly to a self-training semantic segmentation method, apparatus, medium, and equipment for small-sample bridge point clouds. The method includes a pre-training stage and a self-training stage. In the pre-training stage, bridge point cloud data from a support set and a query set are input into a point cloud semantic segmentation network for pre-training. In the self-training stage, the query set bridge point cloud data is simultaneously divided into two branches for processing: one branch inputs the pre-trained point cloud semantic segmentation network and outputs initial semantic prediction results and semantic feature representations; the other branch performs supervoxel segmentation to obtain supervoxels. Based on the output results of the two branches, semantic segmentation of the query set bridge point cloud data is achieved through the self-training process. This solution provides an efficient and feasible technical path for small-sample bridge point cloud semantic segmentation, strongly supporting the intelligent advancement of the Scan2BIM process.

Description

Technical Field

[0001] This invention relates to the field of digital bridge operation and maintenance technology, and in particular to a self-training semantic segmentation method, apparatus, medium and equipment for small sample bridge point clouds. Background Technology

[0002] Currently, bridges are prone to aging and damage during long-term service. Traditional manual inspections suffer from data fragmentation and integration difficulties. Building Information Modeling (BIM) provides technical support for the digital operation and maintenance of bridges. Among these, the Scan2BIM workflow, which generates a current-state BIM of bridges based on point cloud data (i.e., capturing point cloud data from the real-world scene and creating a 3D digital model with semantic information), is one of the core paths to achieving digital bridge operation and maintenance. Point cloud semantic segmentation, as the first step in the Scan2BIM workflow, directly determines the efficiency and accuracy of generating the current-state BIM of the bridge.

[0003] In existing technologies, point cloud-based deep learning models have become the mainstream technique for point cloud semantic segmentation due to their accurate extraction capabilities of original point cloud features. These models can achieve efficient feature extraction and semantic classification of point clouds through operations such as hierarchical local feature aggregation and random sampling. However, this type of method has significant limitations in bridge scenarios: on the one hand, it heavily relies on large-scale labeled training samples, while point cloud labeling requires manual labeling of semantic categories region by region, which is not only time-consuming and labor-intensive but also prone to labeling errors due to human mistakes, resulting in extremely high labeling costs; on the other hand, real bridge point cloud data that can be directly used for model training is extremely scarce, further exacerbating the data dilemma for model training and making it difficult to adapt to the actual application needs of bridge point cloud semantic segmentation.

[0004] To alleviate the reliance on large-scale real-world labeled samples, transfer learning has been attempted for few-sample semantic segmentation tasks. This involves training a model using synthetic bridge point cloud data as the source domain dataset, and then transferring the learned features to the target domain of measured bridge point clouds, thereby reducing the need for labeling real samples in the target domain. However, synthetic point cloud data is generated under idealized scenarios and differs significantly from measured bridge point clouds in real engineering environments (e.g., real-world point clouds commonly suffer from data gaps, target occlusion, and noise interference), making it difficult for the features learned from the source domain to effectively adapt to the target domain. Even when other types of bridge point cloud data from real-world scenarios are used as the source domain dataset, large-scale sample acquisition still faces high costs in data collection and labeling, severely limiting the generalization ability of transfer learning in few-sample semantic segmentation of bridges.

[0005] In summary, existing technologies suffer from two shortcomings in the scenario of small-sample semantic segmentation of bridges: On the one hand, traditional deep learning methods rely on large-scale labeled samples, which cannot adapt to the high cost of labeling bridge point clouds, nor can they cope with the scarcity of real bridge training data; on the other hand, transfer learning is either limited by the domain difference between synthetic bridge point cloud data (source domain) and measured bridge point cloud data (target domain), or by the cost of collecting and labeling real source domain data, making it difficult to meet the generalization ability of actual engineering needs. Therefore, there is an urgent need for a technical solution that can achieve accurate semantic segmentation of unlabeled target domain bridge point cloud data (query set) with only a small amount of labeled source domain bridge point cloud data (support set), in order to overcome the small-sample bottleneck of deep learning point cloud semantic segmentation in bridge digital operation and maintenance. Summary of the Invention

[0006] The main objective of this invention is to provide a self-trained semantic segmentation method, apparatus, medium, and device for small-sample bridge point clouds, aiming to solve the aforementioned problems.

[0007] To achieve the above objectives, the first aspect of this invention proposes a self-training semantic segmentation method for small sample bridge point clouds, which includes a pre-training stage and a self-training stage.

[0008] The pre-training phase includes the following steps:

[0009] S1: Obtain bridge point cloud data from the support set and the query set; simultaneously input the bridge point cloud data from the support set and the query set into the point cloud semantic segmentation network for pre-training, and output the semantic prediction results of the bridge point cloud from the support set and the bridge point cloud from the query set; wherein, the bridge point cloud data from the support set is small sample point cloud data with semantic labels, and the bridge point cloud data from the query set is point cloud data without semantic labels;

[0010] S2: Calculate the first semantic segmentation loss between the semantic prediction result of the bridge point cloud in the support set and the real label, and calculate the mutual information maximization loss between the semantic prediction result of the bridge point cloud in the support set and the semantic prediction result of the bridge point cloud in the query set.

[0011] S3: The first semantic segmentation loss and the mutual information maximization loss are weighted and summed to obtain the first total loss. The parameters of the point cloud semantic segmentation network are updated based on the first total loss. If the update requirement does not meet the preset update requirement, the process returns to step S1. If the update requirement meets the preset update requirement, the updated point cloud semantic segmentation network is obtained. At the same time, the query set obtains the initial semantic prediction result and semantic feature representation. The initial network parameters of the updated point cloud semantic segmentation network are adapted to the bridge point cloud data in the query set.

[0012] The self-training phase includes the following steps:

[0013] S4: The bridge point cloud data in the query set is synchronously processed by branches. The first branch inputs the updated point cloud semantic segmentation network and outputs the semantic prediction result and the semantic feature representation. The first output is the initial semantic prediction result and the semantic feature representation. The second branch performs supervoxel segmentation and outputs multiple supervoxels.

[0014] S5: Perform a supervoxel pooling operation based on the initial semantic prediction results and initial semantic feature representations of all points within the same supervoxel from multiple supervoxels to obtain the pooled semantic prediction results and semantic feature representations.

[0015] S6: Calculate the confidence of each supervoxel based on the semantic prediction results after pooling; and select the semantic prediction results after pooling with high confidence as pseudo-labels; at the same time, adopt the momentum update strategy for the semantic feature representations corresponding to the pseudo-labels to generate and store semantic features and store them in the semantic feature memory bank to form the current semantic feature memory bank.

[0016] S7: Calculate the second semantic segmentation loss between the pseudo-label and the initial semantic prediction result, and the contrast loss between the pooled semantic feature representation and the stored semantic features; sum the second semantic segmentation loss and the contrast loss by weight to obtain the second total loss;

[0017] S8: Update the point cloud semantic segmentation network parameters for the current iteration based on the second total loss; if the iteration requirement does not meet the preset iteration requirement, return to step S4 to continue iterating; when the iteration requirement meets the preset iteration requirement, stop the iteration and output the semantic prediction of the final query set bridge point cloud data.

[0018] Optionally, the second branch performs supervoxel segmentation, outputting multiple supervoxels, specifically including the following steps:

[0019] S401: Calculate the local neighborhood shape geometric features of each point in the bridge point cloud data of the query set; the geometric features include linearity, flatness, scattering, verticality, and elevation;

[0020] S402: For each point in the bridge point cloud data of the query set, find the preset n nearest neighbor points and construct an n-nearest neighbor graph; where n is a natural number greater than 3;

[0021] S403: Construct a global energy function based on the geometric features and the n-nearest neighbor graph;

[0022] S404: The L0 norm segmentation and tracing algorithm is used to quickly solve the approximate solution of the global energy function, and a geometrically uniform supervoxel is obtained.

[0023] Optionally, step S5 specifically includes the following steps:

[0024] S501: Calculate the mean of the initial semantic prediction results and semantic feature representations for all points within the same supervoxel, and obtain the mean of the semantic prediction results and the mean of the semantic feature representations respectively;

[0025] S502: Assign the mean of the semantic prediction result and the mean of the semantic feature representation to all points within the supervoxel to obtain the pooled semantic prediction result and semantic feature representation; so that all points within the same supervoxel have consistent semantic prediction results and semantic feature representations.

[0026] Optionally, the pooled semantic prediction results with high confidence are selected as pseudo-labels. Specifically, the soft labels of the pooled semantic prediction results with high confidence are converted into hard labels as pseudo-labels.

[0027] Optionally, a momentum update strategy is adopted for the semantic feature representation corresponding to the pseudo-label to generate and store semantic features and store them in the semantic feature memory to form the current semantic feature memory. This specifically includes the following steps:

[0028] S601: Construct the initial semantic feature memory;

[0029] S602: The current semantic feature memory is obtained by weighted summation of the semantic feature representation corresponding to the pseudo-label and the previous semantic feature memory; wherein, in the first calculation of the current semantic feature memory, the previous semantic feature memory is the initial semantic feature memory; the sum of the weights of the semantic feature representation corresponding to the pseudo-label and the previous semantic feature memory is 1.

[0030] Optionally, constructing the initial semantic feature memory specifically includes:

[0031] S6011: Extract the initial semantic feature representation of the query set in step S3;

[0032] S6012: After the initial semantic feature representation of the query set is processed by supervoxel pooling, the average value of the semantic feature representations of the same category is calculated.

[0033] S6013: Use the mean as the semantic feature representation in the initial semantic feature memory.

[0034] Optionally, in step S3, the preset update requirements include a preset average intersection-union ratio (IU) requirement for the support set;

[0035] When the average intersection-union ratio (OCR) of the support set reaches the preset average OCR requirement, the update will stop.

[0036] The preset iteration requirements in step S8 include the number of iterations required.

[0037] The iteration stops when the preset number of iterations is reached.

[0038] The second aspect of this invention proposes a self-trained semantic segmentation device for small sample bridge point clouds, including a pre-training module and a self-training module.

[0039] The pre-training module includes:

[0040] A pre-training unit is used to acquire support set bridge point cloud data and query set bridge point cloud data; the support set bridge point cloud data and query set bridge point cloud data are simultaneously input into the point cloud semantic segmentation network for pre-training, and the support set bridge point cloud semantic prediction results and query set bridge point cloud semantic prediction results are output; wherein, the support set bridge point cloud data is small sample point cloud data with semantic labels, and the query set bridge point cloud data is point cloud data without semantic labels.

[0041] The first computing unit is used to calculate the first semantic segmentation loss between the semantic prediction result of the support set bridge point cloud and the real label, and to calculate the mutual information maximization loss between the semantic prediction result of the support set bridge point cloud and the semantic prediction result of the query set bridge point cloud.

[0042] The update unit is used to obtain a first total loss by weighted summation of the first semantic segmentation loss and the mutual information maximization loss, and to update the parameters of the point cloud semantic segmentation network based on the first total loss; when the update requirement does not meet the preset update requirement, the process returns to the pre-training unit; when the update requirement meets the preset update requirement, the updated point cloud semantic segmentation network is obtained; at the same time, the query set obtains the initial semantic prediction result and semantic feature representation; the initial network parameters of the updated point cloud semantic segmentation network are adapted to the bridge point cloud data of the query set;

[0043] The self-training module includes:

[0044] The branch processing module is used to synchronously perform branch processing on the bridge point cloud data of the query set. The first branch inputs the updated point cloud semantic segmentation network and outputs the semantic prediction result and the semantic feature representation. The first output is the initial semantic prediction result and semantic feature representation. The second branch performs supervoxel segmentation and outputs multiple supervoxels.

[0045] The supervoxel pooling module is used to perform supervoxel pooling operation on the initial semantic prediction results and initial semantic feature representations of all points within the same supervoxel based on multiple supervoxels, to obtain the pooled semantic prediction results and semantic feature representations.

[0046] The pseudo-label filtering module is used to calculate the confidence level of each hypervoxel based on the pooled semantic prediction results; and to filter out the pooled semantic prediction results with high confidence as pseudo-labels.

[0047] The semantic feature memory module is used to generate and store semantic features by adopting a momentum update strategy on the semantic feature representation corresponding to the pseudo-label and storing them in the semantic feature memory to form the current semantic feature memory.

[0048] The second calculation module is used to calculate the second semantic segmentation loss between the pseudo-label and the initial semantic prediction result, and the contrast loss between the pooled semantic feature representation and the stored semantic features; and to obtain the second total loss by weighted summation of the second semantic segmentation loss and the contrast loss.

[0049] The iterative output module is used to update the point cloud semantic segmentation network parameters of the current iteration based on the second total loss; when the iteration requirement does not meet the preset iteration requirement, it returns to the branch processing module to continue iterating; when the iteration requirement meets the preset iteration requirement, it stops iterating and outputs the semantic prediction of the final query set bridge point cloud data.

[0050] A third aspect of the present invention proposes a readable storage medium storing computer program instructions, which, when executed by a processor, implement the self-training semantic segmentation method for small sample bridge point clouds.

[0051] A fourth aspect of the present invention provides an electronic device comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, wherein the computer program instructions are executed by the processor to provide a self-training semantic segmentation method for small sample bridge point clouds.

[0052] In this invention, a two-stage framework of "pre-training-self-training" is constructed: In the pre-training stage, a bridge point cloud consisting of a support set with semantic labels (small sample source domain) and a query set without labels (target domain) is input into the point cloud semantic segmentation network. After network processing and output of semantic prediction results, the network parameters are updated by calculating the semantic segmentation loss between the support set predictions and the true labels, the mutual information maximization loss between the support set and query set prediction results, and weighted summation. This yields the initial network parameters adapted to the query set, as well as the initial semantic prediction results and semantic feature representations. In the self-training stage, the query set point cloud is synchronously divided into two branches for parallel processing. One branch is input into the pre-training network. The training network outputs semantic prediction results and semantic feature representations. Another branch performs supervoxel segmentation to obtain geometrically uniform supervoxels. Based on the results of the two branches, supervoxel pooling, high-confidence pseudo-label screening, momentum update of semantic feature memory, calculation of semantic segmentation loss and contrast loss and weighted optimization of network parameters are completed in sequence. After multiple rounds of iterative training, only a small amount of source domain annotation is needed to break through the dependence of traditional methods on large-scale labeled data. It adapts to the complex geometric characteristics of bridge point clouds and ensures the accuracy of semantic segmentation. It provides an efficient and feasible technical path for semantic segmentation of small sample bridge point clouds and strongly supports the intelligent advancement of the Scan2BIM process. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating a self-trained semantic segmentation method for small-sample bridge point clouds in an embodiment of the present invention.

[0055] Figure 2 This is a visualization of the distribution of bridge point cloud data supporting the set and query set in an embodiment of the present invention;

[0056] Figure 3 This is a visualization of the semantic segmentation effect of the method of the present invention on bridge point cloud data of the support set and query set;

[0057] Figure 4 A visualization showing the semantic segmentation performance of the RandLA-Net point cloud semantic segmentation network on bridge point cloud data in both the support set and query set;

[0058] Figure 5 This is a visualization of the semantic segmentation performance of the PointNet point cloud semantic segmentation network on bridge point cloud data in both the support set and query set.

[0059] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0061] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0062] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0063] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0064] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0065] Example 1:

[0066] This invention proposes a self-training semantic segmentation method for small sample bridge point clouds, which includes a pre-training stage and a self-training stage.

[0067] The pre-training phase includes the following steps:

[0068] S1: Obtain bridge point cloud data from the support set and the query set; simultaneously input the bridge point cloud data from the support set and the query set into the point cloud semantic segmentation network for pre-training, and output the semantic prediction results of the bridge point cloud from the support set and the bridge point cloud from the query set; wherein, the bridge point cloud data from the support set is small sample point cloud data with semantic labels, and the bridge point cloud data from the query set is point cloud data without semantic labels;

[0069] S2: Calculate the first semantic segmentation loss between the semantic prediction result of the bridge point cloud in the support set and the real label, and calculate the mutual information maximization loss between the semantic prediction result of the bridge point cloud in the support set and the semantic prediction result of the bridge point cloud in the query set.

[0070] The first semantic segmentation loss is calculated using the cross-entropy loss function, and its calculation formula (1) is shown below:

[0071] (1);

[0072] In the formula: The loss is denoted by N; N is the number of point clouds; C is the number of categories. Let be the true label distribution of the i-th sample in the k-th class; Predict the probability of the i-th sample in the k-th class using the model;

[0073] The first semantic segmentation loss can constrain the consistency between the semantic prediction results and the labels of the support set bridge point cloud data, ensuring that the pre-trained network has basic semantic segmentation capabilities.

[0074] The formula (2) for calculating the mutual information maximization loss is shown below:

[0075] (2);

[0076] In the formula: To maximize the loss of mutual information; The mutual information between the input point cloud data x and the network semantic prediction result y; The overall entropy of the prediction results for all samples; The average entropy of the predicted probability distribution for a single sample; Given the semantic prediction probability distribution corresponding to the input point cloud x;

[0077] The mutual information maximization loss modulates the overall entropy and single-sample average entropy of the prediction results of the query set and support set, so that the prediction results of the query set not only cover all target categories, but also have high confidence per sample, providing the initial semantic prediction results of the query set with "full category coverage and high confidence per label" for screening pseudo-labels in the self-training stage;

[0078] S3: The first semantic segmentation loss and the mutual information maximization loss are weighted and summed to obtain the first total loss. The parameters of the point cloud semantic segmentation network are updated based on the first total loss. If the update requirement does not meet the preset update requirement, the process returns to step S1. If the update requirement meets the preset update requirement, the updated point cloud semantic segmentation network is obtained. At the same time, the query set obtains the initial semantic prediction result and semantic feature representation. The initial network parameters of the updated point cloud semantic segmentation network are adapted to the bridge point cloud data in the query set.

[0079] The self-training phase includes the following steps:

[0080] S4: The bridge point cloud data in the query set is synchronously processed by branches. The first branch inputs the updated point cloud semantic segmentation network and outputs the semantic prediction result and the semantic feature representation. The first output is the initial semantic prediction result and the semantic feature representation. The second branch performs supervoxel segmentation and outputs multiple supervoxels.

[0081] S5: Perform a supervoxel pooling operation based on the initial semantic prediction results and the initial semantic feature representations of all points within the same supervoxel from multiple supervoxels to obtain the pooled semantic prediction results and semantic feature representations.

[0082] S6: Calculate the confidence of each supervoxel based on the semantic prediction results after pooling; and select the semantic prediction results after pooling with high confidence as pseudo-labels; at the same time, adopt the momentum update strategy for the semantic feature representations corresponding to the pseudo-labels to generate and store semantic features and store them in the semantic feature memory bank to form the current semantic feature memory bank.

[0083] The confidence level is calculated using the following formula (3):

[0084] (3);

[0085] In the formula: For supravoxels Confidence level; For supravoxels Semantic prediction probability after pooling;

[0086] The confidence calculation formula takes the negative logarithm of the predicted probability after pooling all points within the supervoxel, mapping low probability (high uncertainty) to high value and high probability (low uncertainty) to low value. Subsequently, by setting a confidence threshold, high-confidence supervoxels can be quickly screened out, providing a quantitative basis for the reliability of pseudo-labels.

[0087] S7: Calculate the second semantic segmentation loss between the pseudo-label and the initial semantic prediction result, and the contrast loss between the pooled semantic feature representation and the stored semantic features; sum the second semantic segmentation loss and the contrast loss by weight to obtain the second total loss; the calculation formula of the second semantic segmentation loss is the same as that of the first semantic segmentation loss, only the parameters are different, which will not be described in detail here;

[0088] The formula for calculating the contrast loss (4) is shown below:

[0089] (4);

[0090] in, For comparison of loss; M is the number of hypervoxels involved in the calculation in the current iteration; This represents the semantic features of the pseudo-labels after hypervoxel pooling. For the stored semantic features in the memory that are consistent with the pseudo-label category (i.e., the first... (Prototype characteristics of a class) The stored semantic features are for all categories (C categories in total) in the memory bank; This is the temperature hyperparameter (in this embodiment, it is set to 0.07, used to control the concentration of the similarity distribution). This represents the vector dot product operation, used to calculate the semantic similarity between features.

[0091] The contrastive loss formula quantifies feature similarity matching through "Softmax cross-entropy": the numerator focuses on the similarity between the pooled features and similar stored semantic features, guiding similar features to cluster in the embedding space; the denominator covers the similarity of stored semantic features across all categories, constraining the pooled features to maintain distance from dissimilar stored semantic features. This constraint strengthens the correlation between the pooled features and historically stable features in the memory, while also improving the class discriminative power of semantic features. This provides feature-level supervision signals for network parameter updates during the self-training phase, further ensuring the accuracy and robustness of the query set bridge point cloud semantic segmentation.

[0092] S8: Update the point cloud semantic segmentation network parameters for the current iteration based on the second total loss; if the iteration requirement is not met, return to step S4 to continue iterating; if the iteration requirement is met, stop iterating and output the semantic prediction of the final query set bridge point cloud data.

[0093] The self-trained semantic segmentation method of this invention has the following advantages:

[0094] By constructing a two-stage framework of "pre-training-self-training": In the pre-training stage, the bridge point cloud between the support set with semantic labels and the query set without labels is input into the point cloud semantic segmentation network. After the network processes the data and outputs semantic prediction results, the network parameters are updated by calculating the semantic segmentation loss between the support set predictions and the ground truth labels, the mutual information maximization loss between the support set and query set prediction results, and weighted summation. This yields the initial network parameters adapted to the query set, as well as the initial semantic prediction results and semantic feature representations. In the self-training stage, the query set point cloud is synchronously divided into two branches for parallel processing. One branch is input into the pre-training network and outputs semantic predictions. The results and semantic feature representations show that another branch performs supervoxel segmentation to obtain geometrically uniform supervoxels. Based on the results of the two branches, supervoxel pooling, high-confidence pseudo-label screening, momentum update of semantic feature memory, calculation of semantic segmentation loss and contrast loss and weighted optimization of network parameters are completed in sequence. After multiple rounds of iterative training, only a small amount of source domain annotation is needed to break through the dependence of traditional methods on large-scale labeled data. It adapts to the complex geometric characteristics of bridge point clouds and ensures the accuracy of semantic segmentation. It provides an efficient and feasible technical path for semantic segmentation of small sample bridge point clouds and strongly supports the intelligent advancement of the Scan2BIM process.

[0095] Optionally, the second branch performs supervoxel segmentation, outputting multiple supervoxels, specifically including the following steps:

[0096] S401: Calculate the local neighborhood shape geometric features of each point in the bridge point cloud data of the query set; the geometric features include linearity, flatness, scattering, verticality, and elevation;

[0097] S402: For each point in the bridge point cloud data of the query set, find the preset n nearest neighbor points and construct an n-nearest neighbor graph; where n is a natural number greater than 3;

[0098] S403: Construct a global energy function based on the geometric features and the n-nearest neighbor graph; in this embodiment, n is 10;

[0099] The formula (5) for calculating the global energy function is shown below:

[0100] (5);

[0101] Where: P is the set of points in the input query set of bridge point cloud data; and The partition assignment vectors for points i and j are respectively. Points with the same value will be grouped into the same superbody; Let i be the geometric characteristic vector (including linearity, flatness, scattering, perpendicularity, and elevation). Geometric feature dimension; This is the regularization strength parameter (used to control the thickness of the partitions); Let be the set of edges of the n-nearest neighbor graph of the point cloud; The weight of edge (i,j) decreases linearly with the edge length. Iverson brackets (when) The value is 1 if the condition is met, and 0 otherwise.

[0102] The global energy function balances the partitioning effect through two constraints: the first minimizes the difference between the partition assignment and geometric features of the points, ensuring that the partitions fit the local geometry of the point cloud; the second penalizes different partition assignments of adjacent points, ensuring the spatial smoothness and geometric uniformity of the partitions. Combined with the L0 norm segmentation and tracking algorithm for fast solution, it can efficiently obtain supervoxels with strong geometric consistency, providing a reliable geometric partitioning basis for subsequent supervoxel pooling and pseudo-label selection, and adapting to the structural characteristics of complex components in bridge point clouds;

[0103] S404: The L0 norm segmentation and tracing algorithm is used to quickly solve the approximate solution of the global energy function, and a geometrically uniform supervoxel is obtained.

[0104] Optionally, step S5 specifically includes the following steps:

[0105] S501: Calculate the mean of the initial semantic prediction results and semantic feature representations for all points within the same supervoxel, respectively, to obtain the mean of the semantic prediction results and the mean of the semantic feature representations; the formula for calculating the mean (6) is as follows:

[0106] (6);

[0107] In the formula: For supravoxels Semantic prediction probability after pooling; For supravoxels Semantic feature representation after pooling; For a single supervoxel; For supravoxels The number of point clouds contained within; For supravoxels The original semantic prediction probability of the i-th point within the range; For supravoxels The original semantic feature representation of the i-th point;

[0108] Hypervoxel pooling aggregates the features and prediction results of points within the same geometric structure, enabling all points within the hypervoxel to have consistent semantic prediction results and semantic feature representations, thereby enhancing prediction stability and reducing point cloud noise interference, and providing a reliable foundation for subsequent high-confidence pseudo-label screening.

[0109] S502: Assign the mean of the semantic prediction result and the mean of the semantic feature representation to all points within the supervoxel to obtain the pooled semantic prediction result and semantic feature representation; so that all points within the same supervoxel have consistent semantic prediction results and semantic feature representations.

[0110] Optionally, the step of selecting the pooled semantic prediction results from the high-confidence supervoxes as pseudo-labels specifically involves converting the soft labels of the pooled semantic prediction results that have been selected as high-confidence into hard labels as pseudo-labels.

[0111] Optionally, the semantic feature representation corresponding to the pseudo-label is updated using a momentum update strategy to generate and store semantic features in a semantic feature memory, forming the current semantic feature memory. This specifically includes the following steps:

[0112] S601: Construct the initial semantic feature memory;

[0113] S602: The current semantic feature memory is obtained by weighted summation of the semantic feature representation corresponding to the pseudo-label and the previous semantic feature memory; wherein, in the first calculation of the current semantic feature memory, the previous semantic feature memory is the initial semantic feature memory; the sum of the weights of the semantic feature representation corresponding to the pseudo-label and the previous semantic feature memory is 1.

[0114] The momentum update strategy achieves stable updates of the memory bank by fusing historical features with current features. The calculation formula (7) is shown below:

[0115] (7);

[0116] in, For the semantic feature memory of the first The class stores semantic features (i.e., category prototype features); m is the momentum coefficient (the value range is usually 0.8~0.95, and in this embodiment it is 0.9); In the current iteration, the pseudo-label corresponding to the supervoxel pooling is the first... Class semantic feature representation; "" indicates an assignment update operation, which means that the new stored semantic features are generated through this formula and overwrite the original memory with the first value. Characteristics of a class;

[0117] The core function of this momentum update formula lies in balancing the "stability" and "updateability" of features: on the one hand, by using a larger weight (m), it retains the core information of historically stored semantic features, avoiding distortion of the memory features due to feature fluctuations in a single iteration; on the other hand, by using a smaller weight (1-m), it incorporates new features from the current iteration, enabling the stored semantic features to be gradually optimized as the network trains. Ultimately, the stably updated stored semantic features in the memory can provide reliable "benchmark features" for calculating the contrastive loss in S7, ensuring the consistency of semantic feature categories and further improving the accuracy of network parameter updates during the self-training phase.

[0118] Optionally, constructing the initial semantic feature memory specifically includes:

[0119] S6011: Extract the initial semantic feature representation of the query set in step S3;

[0120] S6012: After the initial semantic feature representation of the query set is processed by supervoxel pooling, the average value of the semantic feature representations of the same category is calculated.

[0121] S6013: Use the mean as the semantic feature representation in the initial semantic feature memory.

[0122] Optionally, in step S3, the preset update requirements include a preset average intersection-union ratio (IU) requirement for the support set;

[0123] When the average intersection-union ratio (OCR) of the support set reaches the preset average OCR requirement, the update will stop.

[0124] The preset iteration requirements in step S8 include the number of iterations required.

[0125] The iteration stops when the preset number of iterations is reached.

[0126] Experimental Data: Based on open-source point cloud data of 10 concrete bridges, a small-sample bridge semantic segmentation experiment was conducted using RandLA-Net as the point cloud semantic segmentation network. To comprehensively evaluate the semantic segmentation performance of this method, two mainstream point cloud semantic segmentation networks, RandLA-Net and PointNet, were selected as comparison methods.

[0127] Supports the collection and querying of bridge point cloud data, such as Figure 2 As shown, the point cloud data of 3 bridges is the support set, and the point cloud data of 7 bridges is the query set. To enhance the diversity of the support set data, the point cloud data of each bridge in the support set is divided into blocks for enhancement to obtain its enhanced point cloud data (each bridge is divided into two parts, such as bridge 1-1 and bridge 1-2).

[0128] The semantic segmentation results of the method of this invention, RandLA-Net, and PointNet on bridge point cloud data in the support set and query set are shown in Table 1 and Table 2, respectively. The visualization effect of the semantic segmentation results of bridge point clouds is shown in Table 2. Figures 3 to 5 As shown.

[0129] Table 1 Comparison of average intersection-union ratio (mIoU / %) ​​for semantic segmentation of bridge point clouds supported by the dataset

[0130]

[0131] Table 2 Comparison of average intersection-union ratio (mIoU / %) ​​for semantic segmentation of bridge point clouds in query sets

[0132]

[0133] Table 1 presents the semantic segmentation results (mIoU as the evaluation metric) of the bridge point cloud data on the support set after training on the three methods (where the method of this invention is pre-trained in conjunction with the query set). It can be seen that the semantic segmentation performance of the three methods on the support set is quite similar. The method of this invention achieves a total mIoU of 91.76%, RandLA-Net achieves 88.98%, and PointNet achieves 84.97%.

[0134] Table 2 presents the semantic segmentation results of the three methods on the query set bridge point cloud data. The method of this invention achieves a total mIoU of 84.78%, which is significantly better than RandLA-Net (47.43%) and PointNet (53.57%), demonstrating superior semantic segmentation performance for the query set.

[0135] As can be seen from Tables 1 and 2, under the premise of similar semantic segmentation performance for support sets, the method of the present invention significantly improves the semantic segmentation performance of bridge point cloud data in query sets, which fully verifies the effectiveness of the self-training paradigm in the semantic segmentation scenario of small sample bridge point clouds.

[0136] Example 2:

[0137] A self-trained semantic segmentation device for small sample bridge point clouds includes a pre-training module and a self-training module.

[0138] The pre-training module includes:

[0139] A pre-training unit is used to acquire support set bridge point cloud data and query set bridge point cloud data; the support set bridge point cloud data and query set bridge point cloud data are simultaneously input into the point cloud semantic segmentation network for pre-training, and the support set bridge point cloud semantic prediction results and query set bridge point cloud semantic prediction results are output; wherein, the support set bridge point cloud data is small sample point cloud data with semantic labels, and the query set bridge point cloud data is point cloud data without semantic labels.

[0140] The first computing unit is used to calculate the first semantic segmentation loss between the semantic prediction result of the support set bridge point cloud and the real label, and to calculate the mutual information maximization loss between the semantic prediction result of the support set bridge point cloud and the semantic prediction result of the query set bridge point cloud.

[0141] The update unit is used to obtain a first total loss by weighted summation of the first semantic segmentation loss and the mutual information maximization loss, and to update the parameters of the point cloud semantic segmentation network based on the first total loss; when the update requirement does not meet the preset update requirement, the process returns to the pre-training unit; when the update requirement meets the preset update requirement, the updated point cloud semantic segmentation network is obtained; at the same time, the query set obtains the initial semantic prediction result and semantic feature representation; the initial network parameters of the updated point cloud semantic segmentation network are adapted to the bridge point cloud data of the query set;

[0142] The self-training module includes:

[0143] The branch processing module is used to synchronously perform branch processing on the bridge point cloud data of the query set. The first branch inputs the updated point cloud semantic segmentation network and outputs the semantic prediction result and the semantic feature representation. The first output is the initial semantic prediction result and semantic feature representation. The second branch performs supervoxel segmentation and outputs multiple supervoxels.

[0144] The supervoxel pooling module is used to perform supervoxel pooling operation on the initial semantic prediction results and the initial semantic feature representation of all points within the same supervoxel based on multiple supervoxels, to obtain the pooled semantic prediction results and semantic feature representation.

[0145] The pseudo-label filtering module is used to calculate the confidence level of each hypervoxel based on the pooled semantic prediction results; and to filter out the pooled semantic prediction results with high confidence as pseudo-labels.

[0146] The semantic feature memory module is used to generate and store semantic features by adopting a momentum update strategy on the semantic feature representation corresponding to the pseudo-label and storing them in the semantic feature memory to form the current semantic feature memory.

[0147] The second calculation module is used to calculate the second semantic segmentation loss between the pseudo-label and the initial semantic prediction result, and the contrast loss between the pooled semantic feature representation and the stored semantic features; and to obtain the second total loss by weighted summation of the second semantic segmentation loss and the contrast loss.

[0148] The iterative output module is used to update the point cloud semantic segmentation network parameters of the current iteration based on the second total loss; when the iteration requirement is not met, it returns to the branch processing module to continue iterating; when the iteration requirement is met, it stops iterating and outputs the semantic prediction of the final query set bridge point cloud data.

[0149] Example 3:

[0150] A readable storage medium storing computer program instructions that, when executed by a processor, implement a self-training semantic segmentation method for small-sample bridge point clouds as described in Embodiment 1.

[0151] The computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0152] Example 4:

[0153] An electronic device includes: at least one processor, at least one memory, and computer program instructions stored in the memory, wherein the computer program instructions are executed by the processor as described in Embodiment 1, a self-trained semantic segmentation method for small sample bridge point clouds.

[0154] The electronic device can be a mobile phone, desktop computer, laptop, handheld computer, cloud server, or other computing device. The electronic device may include, but is not limited to, processors and memory. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0155] The above description is only a preferred embodiment of the present invention and does not limit the scope of the present invention. All equivalent structural transformations made under the inventive concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the protection scope of the present invention.

Claims

1. A self-trained semantic segmentation method for small-sample bridge point clouds, characterized in that, Includes a pre-training phase and a self-training phase; The pre-training phase includes the following steps: S1: Obtain bridge point cloud data from the support set and the query set; simultaneously input the bridge point cloud data from the support set and the query set into the point cloud semantic segmentation network for pre-training, and output the semantic prediction results of the bridge point cloud from the support set and the bridge point cloud from the query set; wherein, the bridge point cloud data from the support set is small sample point cloud data with semantic labels, and the bridge point cloud data from the query set is point cloud data without semantic labels; S2: Calculate the first semantic segmentation loss between the semantic prediction result of the bridge point cloud in the support set and the real label, and calculate the mutual information maximization loss between the semantic prediction result of the bridge point cloud in the support set and the semantic prediction result of the bridge point cloud in the query set. S3: The first semantic segmentation loss and the mutual information maximization loss are weighted and summed to obtain the first total loss. The parameters of the point cloud semantic segmentation network are updated based on the first total loss. If the update requirement does not meet the preset update requirement, the process returns to step S1. If the update requirement meets the preset update requirement, the updated point cloud semantic segmentation network is obtained. At the same time, the query set obtains the initial semantic prediction result and semantic feature representation. The initial network parameters of the updated point cloud semantic segmentation network are adapted to the bridge point cloud data in the query set. The self-training phase includes the following steps: S4: The bridge point cloud data in the query set is synchronously processed by branches. The first branch inputs the updated point cloud semantic segmentation network and outputs the semantic prediction result and the semantic feature representation. The first output is the initial semantic prediction result and the semantic feature representation. The second branch performs supervoxel segmentation and outputs multiple supervoxels. S5: Perform a supervoxel pooling operation based on the initial semantic prediction results and initial semantic feature representations of all points within the same supervoxel from multiple supervoxels to obtain the pooled semantic prediction results and semantic feature representations. S6: Calculate the confidence of each supervoxel based on the semantic prediction results after pooling; and select the semantic prediction results after pooling with high confidence as pseudo-labels; at the same time, adopt the momentum update strategy for the semantic feature representations corresponding to the pseudo-labels to generate and store semantic features and store them in the semantic feature memory bank to form the current semantic feature memory bank. S7: Calculate the second semantic segmentation loss between the pseudo-label and the initial semantic prediction result, and the contrast loss between the pooled semantic feature representation and the stored semantic features; sum the second semantic segmentation loss and the contrast loss by weight to obtain the second total loss; S8: Update the point cloud semantic segmentation network parameters for the current iteration based on the second total loss; if the iteration requirement does not meet the preset iteration requirement, return to step S4 to continue iterating; when the iteration requirement meets the preset iteration requirement, stop the iteration and output the semantic prediction of the final query set bridge point cloud data.

2. The self-trained semantic segmentation method for small-sample bridge point clouds as described in claim 1, characterized in that, The second branch performs supervoxel segmentation and outputs multiple supervoxels, specifically including the following steps: S401: Calculate the local neighborhood shape geometric features of each point in the bridge point cloud data of the query set; the geometric features include linearity, flatness, scattering, verticality, and elevation; S402: For each point in the bridge point cloud data of the query set, find the preset n nearest neighbor points and construct an n-nearest neighbor graph; where n is a natural number greater than 3; S403: Construct a global energy function based on the geometric features and the n-nearest neighbor graph; S404: The L0 norm segmentation and tracing algorithm is used to quickly solve the approximate solution of the global energy function, and a geometrically uniform supervoxel is obtained.

3. The self-trained semantic segmentation method for small-sample bridge point clouds as described in claim 1, characterized in that, Step S5 specifically includes the following steps: S501: Calculate the mean of the initial semantic prediction results and semantic feature representations for all points within the same supervoxel, and obtain the mean of the semantic prediction results and the mean of the semantic feature representations respectively; S502: Assign the mean of the semantic prediction result and the mean of the semantic feature representation to all points within the supervoxel to obtain the pooled semantic prediction result and semantic feature representation; so that all points within the same supervoxel have consistent semantic prediction results and semantic feature representations.

4. The self-trained semantic segmentation method for small-sample bridge point clouds as described in claim 1, characterized in that, The high-confidence pooled semantic prediction results are selected as pseudo-labels. Specifically, the soft labels of the selected high-confidence pooled semantic prediction results are converted into hard labels as pseudo-labels.

5. The self-trained semantic segmentation method for small-sample bridge point clouds as described in claim 1, characterized in that, The semantic feature representation corresponding to the pseudo-label is applied using a momentum update strategy to generate and store semantic features in a semantic feature memory, forming the current semantic feature memory. This process includes the following steps: S601: Construct the initial semantic feature memory; S602: The current semantic feature memory is obtained by weighted summation of the semantic feature representation corresponding to the pseudo-label and the previous semantic feature memory; wherein, in the first calculation of the current semantic feature memory, the previous semantic feature memory is the initial semantic feature memory; the sum of the weights of the semantic feature representation corresponding to the pseudo-label and the previous semantic feature memory is 1.

6. The self-trained semantic segmentation method for small-sample bridge point clouds as described in claim 5, characterized in that, The construction of the initial semantic feature memory specifically includes: S6011: Extract the initial semantic feature representation of the query set in step S3; S6012: After the initial semantic feature representation of the query set is processed by supervoxel pooling, the average value of the semantic feature representations of the same category is calculated. S6013: Use the mean as the semantic feature representation in the initial semantic feature memory.

7. The self-trained semantic segmentation method for small-sample bridge point clouds as described in claim 1, characterized in that, In step S3, the preset update requirements include the preset average intersection-union ratio (IURR) index requirement for the support set; When the average intersection-union ratio (OCR) of the support set reaches the preset average OCR requirement, the update will stop. The preset iteration requirements in step S8 include the number of iterations required. The iteration stops when the preset number of iterations is reached.

8. A self-trained semantic segmentation device for small-sample bridge point clouds, characterized in that, Includes pre-trained modules and self-trained modules; The pre-training module includes: A pre-training unit is used to acquire support set bridge point cloud data and query set bridge point cloud data; the support set bridge point cloud data and query set bridge point cloud data are simultaneously input into the point cloud semantic segmentation network for pre-training, and the support set bridge point cloud semantic prediction results and query set bridge point cloud semantic prediction results are output; wherein, the support set bridge point cloud data is small sample point cloud data with semantic labels, and the query set bridge point cloud data is point cloud data without semantic labels. The first computing unit is used to calculate the first semantic segmentation loss between the semantic prediction result of the support set bridge point cloud and the real label, and to calculate the mutual information maximization loss between the semantic prediction result of the support set bridge point cloud and the semantic prediction result of the query set bridge point cloud. The update unit is used to obtain a first total loss by weighted summation of the first semantic segmentation loss and the mutual information maximization loss, and to update the parameters of the point cloud semantic segmentation network based on the first total loss; when the update requirement does not meet the preset update requirement, it returns to the pre-training unit; when the update requirement meets the preset update requirement, the updated point cloud semantic segmentation network is obtained; at the same time, the query set obtains the initial semantic prediction result and semantic feature representation; the initial network parameters of the updated point cloud semantic segmentation network are adapted to the bridge point cloud data of the query set; The self-training module includes: The branch processing module is used to synchronously perform branch processing on the bridge point cloud data of the query set. The first branch inputs the updated point cloud semantic segmentation network and outputs the semantic prediction result and the semantic feature representation. The first output is the initial semantic prediction result and semantic feature representation. The second branch performs supervoxel segmentation and outputs multiple supervoxels. The supervoxel pooling module is used to perform supervoxel pooling operation on the initial semantic prediction results and initial semantic feature representations of all points within the same supervoxel based on multiple supervoxels, to obtain the pooled semantic prediction results and semantic feature representations. The pseudo-label filtering module is used to calculate the confidence level of each hypervoxel based on the pooled semantic prediction results; and to filter out the pooled semantic prediction results with high confidence as pseudo-labels. The semantic feature memory module is used to generate and store semantic features by adopting a momentum update strategy on the semantic feature representation corresponding to the pseudo-label and storing them in the semantic feature memory to form the current semantic feature memory. The second calculation module is used to calculate the second semantic segmentation loss between the pseudo-label and the initial semantic prediction result, and the contrast loss between the pooled semantic feature representation and the stored semantic features; and to obtain the second total loss by weighted summation of the second semantic segmentation loss and the contrast loss. The iterative output module is used to update the point cloud semantic segmentation network parameters of the current iteration based on the second total loss; when the iteration requirement does not meet the preset iteration requirement, it returns to the branch processing module to continue iterating; when the iteration requirement meets the preset iteration requirement, it stops iterating and outputs the semantic prediction of the final query set bridge point cloud data.

9. A readable storage medium, characterized in that, It stores computer program instructions, which, when executed by a processor, implement a self-training semantic segmentation method for small sample bridge point clouds as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: At least one processor, at least one memory, and computer program instructions stored in the memory, wherein the computer program instructions are executed by the processor, as described in any one of claims 1 to 7, a self-trained semantic segmentation method for small sample bridge point clouds.

Images