A track structure point cloud weak supervision segmentation method and system based on SAM and PointNet++
By using a multi-stage point cloud processing method based on SAM and PointNet++, pseudo-labels and confidence weights are generated, solving the problem of high-precision segmentation of railway track structure point clouds under weak supervision, and realizing efficient labeling and accurate segmentation of track structure point cloud data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NAT ENG LAB FOR HIGH SPEED RAILWAY CONSTR
- Filing Date
- 2026-05-12
- Publication Date
- 2026-07-21
Smart Images
Figure CN122176319B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of point cloud processing technology, and in particular to a weakly supervised segmentation method and system for orbital structure point clouds based on SAM and PointNet++. Background Technology
[0002] Point cloud data of railway track structures contains rich spatial location information and geometric features, and is less affected by external environmental factors such as lighting and climate, enabling high-precision 3D reconstruction and analysis of track structures in complex scenarios. Fine segmentation of the track structure point cloud is not only a key step in achieving safe inspection, digital modeling, and visual management of railway track structures, but also an important foundation for accurate maintenance and condition assessment of track equipment.
[0003] Traditional point cloud segmentation methods mainly rely on geometric feature extraction and clustering algorithms. These methods can achieve certain results in scenarios with obvious track structure features and less noise. However, due to their limited ability to express complex morphological and semantic information, they are often difficult to adapt to real railway scenarios with occlusion, degradation, or complex morphology.
[0004] With the rapid development of computer vision and deep learning technologies, point cloud segmentation methods based on deep learning have become a research hotspot. Among them, network structures represented by PointNet and its series of models can directly perform end-to-end learning on the original point cloud, thereby achieving significant improvements in feature representation and semantic recognition. However, the performance of such methods largely depends on a large amount of high-quality, accurately labeled training data. However, due to the complex acquisition environment and high cost of manual annotation, it is difficult to obtain sufficient labeled samples for orbital structure point cloud data.
[0005] Compared to point cloud segmentation, the field of 2D image segmentation has achieved a landmark breakthrough. Meta AI's "Segment Anything Model" (SAM), as a general-purpose image segmentation foundation model, demonstrates powerful zero-shot generalization and interactive segmentation capabilities. SAM can accurately segment any target in images that have not been trained for a specific task, based on sparse cues such as points and bounding boxes. Its unprecedented versatility is attributed to a dataset built from over a billion masks. However, SAM is essentially a model for 2D images and cannot directly understand and process the spatial geometric relationships of 3D point clouds, which limits its direct application in orbital 3D perception tasks.
[0006] Existing technologies still have significant limitations in the accurate segmentation of railway track structure point clouds. A systematic solution for achieving high-precision segmentation and recognition of railway track structure point clouds under weakly supervised conditions remains lacking. Therefore, it is necessary to provide a weakly supervised segmentation method for railway track structure point clouds based on SAM and PointNet++ to address these technical problems. Summary of the Invention
[0007] The purpose of this invention is to provide a weakly supervised segmentation method and system for track structure point clouds based on SAM and PointNet++. It aims to fully utilize advanced image segmentation models and combine them with minimal human prompting to achieve weakly supervised segmentation of large amounts of unlabeled track structure point cloud data. Under weakly supervised conditions, it achieves training results close to those of fully supervised point cloud data, reducing the dependence of track structure point cloud segmentation on high-precision labeled point cloud data. The specific technical solution is as follows: A weakly supervised segmentation method for orbital structure point clouds based on SAM and PointNet++ includes the following steps: Step S1: Collect point cloud data of the track structure to obtain an unlabeled point cloud dataset with RGB colors; Step S2: Construct a point cloud processing module based on SAM and a weakly supervised point cloud segmentation module based on PointNet++; wherein: The SAM-based point cloud processing module comprises four parts: Stage0, Stage1, Stage2, and Stage3, which are connected in sequence. The SAM-based point cloud processing module is used to process the input point cloud data and generate point cloud training data with pseudo-labels and confidence weights. The PointNet++-based weakly supervised point cloud segmentation module includes a data loading module and a weakly supervised training module; the PointNet++-based weakly supervised point cloud segmentation module is used to perform weakly supervised training using the point cloud training data with pseudo-labels and confidence weights. Step S3: Preprocess the unlabeled point cloud dataset obtained in step S1, and combine it with the SAM-based point cloud processing module to obtain point cloud training data with pseudo-labels and confidence weights. Step S4: Input the point cloud training data obtained in Step S3 into the weakly supervised point cloud segmentation module based on PointNet++ for training to obtain the trained point cloud semantic segmentation model. Use the trained point cloud semantic segmentation model to perform semantic segmentation on the unlabeled point cloud obtained in Step S1 to obtain semantic segmentation point cloud components.
[0008] Furthermore, Stage0 includes a preprocessing module and a rendering module connected in series. The preprocessing module is used to standardize the coordinates of the input point cloud data. The rendering module is used to generate multi-view rendering maps, depth maps, index maps, and index lists based on the standardized coordinate information, color information, and view parameters. Stage 1 includes a first-view SAM segmentation module, a unified component construction module, and an auxiliary-view SAM automatic segmentation module connected in series. The first-view SAM segmentation module is used to perform SAM segmentation on the first-view information generated in Stage 0 based on human interaction prompts, and output the first-view segmentation result. The unified component construction module is used to construct a standardized component point set across perspectives based on the first-view segmentation result. The auxiliary-view SAM automatic segmentation module is used to perform SAM segmentation on the auxiliary-view rendering data and the standardized component point set output by the unified component construction module, and output the auxiliary-view segmentation result. Stage 2 includes a cross-view component pairing module and a statistical scoring fusion module. The cross-view component pairing module is used to establish corresponding component groups based on the main view segmentation results and the original point cloud 3D coordinate output obtained from Stage 1, so as to realize the association of the same physical component under different views. The statistical scoring fusion module is used to generate preliminary segmentation labels and confidence weights of the point cloud through statistical scoring based on the main view segmentation results, auxiliary view segmentation results and component groups output by the cross-view component pairing module. Stage 3 includes a label optimization module, a semantic mapping module, and a data export module connected in sequence. The label optimization module is used to optimize the initial segmentation labels of the point cloud and output the final point cloud segmentation labels. The semantic mapping module is used to map the final point cloud segmentation labels to semantic categories and generate a mapping file. The data export module is used to export a point cloud training dataset that meets the requirements.
[0009] Furthermore, in Stage 0: The preprocessing module performs the following geometric transformations on the input point cloud data in sequence: PCA principal component analysis alignment, centering, rotation around the Y-axis, translation and scaling, and outputs standardized virtual mesh coordinates; The rendering module generates multi-view rendering maps, depth maps, index maps, and index lists by constructing a virtual camera, performing camera projection transformation, executing visibility filtering, and implementing Z-buffer rasterization, based on standardized virtual mesh coordinates, color information, and view parameters.
[0010] Furthermore, in Stage 1: The specific steps for outputting the main view segmentation results through the main view SAM segmentation module include: performing manual interactive annotation, drawing bounding boxes on the main view rendering map, generating prompt information for each track structure component, and outputting a series of bounding box coordinates; calling the SAM model to perform zero-shot segmentation to obtain the 2D segmentation mask corresponding to each bounding box; extracting visible and invisible points, and outputting the main view segmentation results containing bounding boxes, segmentation masks, a list of visible points, and a list of invisible points. The specific steps for outputting a standardized component point set through the unified component building module include: calculating the IoU overlap between the masks of each component in two main views and the distance between the component centers, determining and merging duplicate components based on a preset threshold, and outputting a list of standardized component point sets after eliminating viewpoint redundancy. The specific steps for outputting auxiliary view segmentation results through the auxiliary view SAM automatic segmentation module include: generating bounding box hints on the auxiliary view index map based on component point sets; batch calling SAM for automatic segmentation; extracting visibility information and aligning it across views, and outputting the segmentation mask and visibility information of the auxiliary view.
[0011] Furthermore, in Stage 2: The cross-view component pairing module achieves accurate association of the same physical component under different viewpoints. Specifically, it calculates the 3D center position of the visible points. Taking the visible point index and point cloud coordinates of each mask as input, it calculates the median position of the visible points of the mask in 3D space and outputs the 3D center coordinates of each mask. Next, it constructs a distance matrix and an IoU overlap matrix. Taking the center coordinates of all masks and the set of visible points of two main viewpoints as input, it calculates the Euclidean distance between the centers to obtain the distance matrix, and simultaneously calculates the intersection-union ratio of the set of visible points to obtain the IoU overlap matrix. Then, based on the threshold-based optimal matching, taking the distance matrix, the IoU overlap matrix, and preset distance and IoU thresholds as input, it searches for the best matching mask in viewpoint two that simultaneously satisfies the condition that the distance is less than the threshold and the IoU is greater than the threshold for each mask in viewpoint one. If no such mask exists, it selects the closest mask and outputs the component grouping that establishes the corresponding relationship, thus achieving accurate association of the same physical component under different viewpoints. The statistical scoring fusion module generates preliminary segmentation labels and confidence weights for the point cloud. Specifically, it first constructs a statistical scoring matrix, taking the total number of points in the point cloud, component grouping, and segmentation results from all views as input. By initializing a zero matrix of point count × component count, it accumulates high weights for visible points and low weights for invisible points of each component in each view, outputting the statistical scoring matrix. Next, it assigns labels based on the highest score principle, taking the statistical scoring matrix as input, and selecting the component with the highest score for each point as its label. If all component scores are zero, they are marked as background, outputting the preliminary segmentation labels for the point cloud. Simultaneously, it calculates confidence weights, taking the highest score for each point as input, and using the score as the confidence level of the point's label, outputting the point-level confidence weights.
[0012] Furthermore, the initial segmentation labels of the point cloud are optimized through a label optimization module. Specifically: First, iterative label propagation based on radius neighborhood is performed. Taking the point cloud coordinates and the original labels as input, a KD-Tree is constructed to query the neighbors of each background point within a fixed radius. If the proportion of non-background points in the neighbors exceeds a threshold and the number meets the requirements, the label of the background point is updated to the mode label among the neighbors. After multiple iterations until convergence, intermediate labels with enhanced geometric consistency are output. Next, DBSCAN clustering noise filtering is performed. Taking the above intermediate labels and point cloud coordinates as input, DBSCAN is used to perform spatial clustering on the points belonging to each non-background category. Clusters marked as noise or with too few points are identified and removed, and the labels of these points are reset to background. The labels after noise filtering are output. Finally, small cluster removal and optional reassignment are implemented. Taking the filtered labels and point cloud coordinates as input, a nearest neighbor search is performed by constructing a KD-Tree with the coordinates of the removed points and the remaining foreground points. The labels are reassigned according to the majority vote of the K nearest neighbors, and the optimized final point cloud segmentation labels are output. The semantic mapping module generates a mapping file by: First, performing component geometric feature statistical analysis, taking the point set coordinates of each instance component as input, calculating its 3D bounding box, bounding box size, number of points, PCA principal components, and median, and outputting the geometric feature description vector of each component; then, performing heuristic rule matching, taking the geometric feature description vector as input, matching through a preset rule base, outputting mapping suggestions and confidence scores for each instance component to a preset semantic category, and generating a mapping file. The data export module exports a point cloud training dataset that meets the requirements. Specifically, it applies semantic mapping and calculates point-level weights. Taking the original instance labels and mapping files as input, it converts instance IDs into semantic category IDs by querying the mapping files. At the same time, it uses a statistical scoring matrix to calculate the confidence score of each point's category as the point-level weight, and outputs a point cloud with semantic labels and weights. Next, it organizes a standardized dataset structure. Taking the processed point cloud, labels, and weights as input, it standardizes file naming, creates a standard directory, and generates metadata files, outputting a complete point cloud training dataset that meets the requirements.
[0013] Furthermore, the data loading module is used to load the point cloud training dataset and perform random sampling, data augmentation, and feature channel expansion to generate batch training samples.
[0014] Furthermore, the weakly supervised training module is used to train the PointNet++ model using the batch training samples and a weighted cross-entropy loss function that combines point-level confidence weights and class balance weights. The trained point cloud semantic segmentation model is obtained through dynamic learning rate decay and model validation.
[0015] Furthermore, the weakly supervised training module employs a weighted cross-entropy loss function. for: ; in: For point Confidence level, To balance the weights for each category, For PointNet++ model point The predicted probability distribution For point The true label category, Cross-entropy loss; ; in: For indicator functions, Point Belongs to the category of real labels The predicted probability, This indicates the total number of categories.
[0016] A weakly supervised segmentation system for orbital structure point clouds based on SAM and PointNet++ includes a data acquisition module, a SAM-based point cloud processing module, a PointNet++-based weakly supervised point cloud segmentation module, and a semantic segmentation execution module. The data acquisition module is used to acquire point cloud data of the track structure to obtain an unlabeled point cloud dataset with RGB colors. The SAM-based point cloud processing module is used to process the input point cloud data and generate point cloud training data with pseudo-labels and confidence weights. The weakly supervised point cloud segmentation module based on PointNet++ is used to perform weakly supervised training using the point cloud training data with pseudo-labels and confidence weights to obtain the trained point cloud semantic segmentation model. The semantic segmentation execution module is used to perform semantic segmentation on the input unlabeled track structure point cloud to be segmented using the trained point cloud semantic segmentation model.
[0017] The application of the technical solution of the present invention has at least the following beneficial effects: (1) This invention achieves automated generation of training data from unlabeled point clouds to point cloud data with pseudo-labels and confidence weights by constructing SAM-based point cloud processing modules (Stage 0 to Stage 3). This method utilizes multi-view rendering in Stage 0 and manual prompting in Stage 1 to guide SAM segmentation, combined with cross-view component pairing and statistical scoring fusion in Stage 2, and finally completes label optimization and semantic mapping in Stage 3. The entire process transforms the traditional high-density manual labor that requires point-by-point annotation into lightweight interactive operations based on components and viewpoints, greatly reducing the dependence on high-precision labeled data and significantly improving the annotation efficiency of track structure point cloud data.
[0018] (2) This invention introduces a weighted cross-entropy loss function with point-level confidence weights and class balance weights in the weakly supervised training module based on PointNet++. This mechanism reduces oscillations and overfitting during training by assigning greater weights to points with higher confidence in pseudo-labels, thereby suppressing the interference of low-confidence samples on gradient updates. At the same time, combined with the statistical scoring matrix generated by Stage2 and the label optimization strategies of Stage3 (such as radius neighborhood propagation and DBSCAN noise filtering), the quality and consistency of training data are effectively improved, enhancing the model's generalization ability and segmentation accuracy in complex orbital structure scenarios.
[0019] (3) This invention achieves accurate association and consistent fusion of the same physical component under multiple perspectives through the cross-view component pairing and statistical scoring fusion module in Stage 2; the label optimization module in Stage 3 further enhances the spatial continuity and geometric consistency of the segmentation results through neighborhood propagation and small cluster reassignment strategies. The semantic mapping module combines component geometric features (such as bounding box size, PCA principal components, etc.) and heuristic rules to automatically map instance labels to preset semantic categories, and supports manual verification and correction, ensuring the interpretability and engineering applicability of the final semantic segmentation results.
[0020] (4) By generating high-quality pseudo-labels and confidence weights, this invention enables the PointNet++ model to achieve near-fully supervised performance under weak supervision. At the same time, by standardizing data organization and sampling strategies, it reduces the data volume in a single scenario and the peak memory usage during training, reduces the dependence on high-performance hardware, and improves the portability and deployment efficiency of the PointNet++ model across different platforms.
[0021] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description
[0022] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart illustrating the weakly supervised segmentation method for orbital structure point clouds based on SAM and PointNet++ in this invention. Detailed Implementation
[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0024] Example: See Figure 1 This invention proposes a weakly supervised segmentation method for orbital structure point clouds based on SAM and PointNet++, comprising the following steps: Step S1: Collect track structure point cloud data to obtain an unlabeled point cloud dataset with RGB colors; In this embodiment, the track structure point cloud data is collected by an existing intelligent track inspection vehicle and includes the main track structure.
[0025] Step S2: Construct a point cloud processing module based on SAM and a weakly supervised point cloud segmentation module based on PointNet++; In this embodiment, the SAM consists of an image encoder, a cue encoder, and a lightweight mask interpreter. The image encoder employs a traditional visual transformer (ViT) architecture. The cue encoder encodes various forms of user cues (such as points, boxes, and masks) into sparse or dense feature embeddings, which are then combined with the image features extracted by the image encoder and fed into the mask decoder to achieve accurate segmentation of the target region. The mask decoder consists of two transformer layers, where learnable output labels interact with image and cue features to predict multiple candidate segmentation masks. An IoU prediction head is used to evaluate the quality of each mask, and the most accurate mask is selected as the final output.
[0026] The mask quality assessment formula is as follows: ; in: The mask quality score (IoU score) predicted by the SAM model. For IoU prediction, a multilayer perceptron; The sparse cue tokens (such as tokens corresponding to dot or box cues) output by the Transformer decoder are the input vectors for prediction quality.
[0027] In this embodiment, PointNet++ consists of a hierarchical sampling module, a local feature extraction module, and a point-level classification head. Its core idea is to introduce a hierarchical structure modeling mechanism for point cloud space based on the global feature learning framework of PointNet, thereby achieving effective processing of non-uniformly sampled point clouds. The hierarchical sampling module uses farthest point sampling (FPS) and spherical neighborhood query to group the original point cloud, enabling each layer to extract geometric features of local regions. The local feature extraction module uses an extended PointNet network to encode local structural information within each neighborhood and enhances the robustness of features through multi-scale grouping (MSG) or multi-resolution grouping (MRG).
[0028] Subsequently, the multi-level local features are propagated back to the high-resolution point set and fused with the low-level features, then input into the point-level classification head to predict the semantic category of each point. The point-level classification head is typically composed of a multilayer perceptron (MLP) and can generate multiple candidate semantic probability distributions. Combining the interpolation accuracy of feature propagation with local geometric consistency, the optimal probability response can be selected as the final point cloud semantic label. Through this bottom-up hierarchical geometric modeling approach, PointNet++ can maintain sensitivity to both local details and global structure, making it suitable for high-precision semantic segmentation of point cloud data in complex engineering scenarios.
[0029] The point-level classification formula is as follows: ; in: The semantic classification probability distribution for each point, This is the final feature vector of that point. , These are the parameters for the fully connected layer. Output the probability of each class.
[0030] In this embodiment, the SAM-based point cloud processing module includes four parts connected in sequence: Stage0, Stage1, Stage2, and Stage3. The SAM-based point cloud processing module is used to process the input point cloud data and generate point cloud training data with pseudo-labels and confidence weights. Stage0 includes a preprocessing module and a rendering module connected in series: The preprocessing module is used to standardize the coordinates of the input point cloud data. Specifically, the preprocessing module performs the following geometric transformations on the input point cloud data (3D coordinates) in sequence: First, it performs PCA principal component analysis alignment. The coordinates of the original point cloud are input, and the principal component directions are obtained through eigenvalue decomposition. A rotation matrix is constructed to align the first principal component to the Y-axis, the second principal component to the X-axis, and the third principal component to the Z-axis, outputting the intermediate coordinates of the principal axis alignment. Then, it performs centering, subtracting the geometric center from the aligned coordinates, outputting the coordinates with their own centroid as the origin. Next, it performs rotation around the Y-axis, using the centered coordinates and the user-provided yaw angle as input, constructing a rotation matrix around the Y-axis using yaw and applying it to the coordinates, outputting the coordinates after rotating horizontally by the specified angle. Finally, it performs translation and scaling, translating the rotated coordinates so that their minimum coordinate value is positive, and then normalizing them by dividing by a preset grid size parameter, outputting virtual grid coordinates located in the first quadrant and scaled.
[0031] The rendering module, based on standardized virtual mesh coordinates, color information, and viewpoint parameters (pitch is used to calculate camera height), generates multi-view rendering maps, depth maps, index maps, and index lists by constructing a virtual camera, performing camera projection transformation, executing visibility filtering, and implementing Z-buffer rasterization. Specifically: First, a virtual camera is constructed. Taking virtual mesh coordinates and pitch parameters as input, the camera's spatial position is determined by calculating the center of the point cloud bounding box, based on the diagonal length and pitch angle, and then the rotation and translation matrices are calculated based on the camera position, viewpoint, and upward direction, outputting the intrinsic and extrinsic parameter matrices of the virtual camera. Next, camera projection transformation is performed. Taking virtual mesh coordinates and camera parameters as input, 3D points are converted to the camera coordinate system and projected onto the 2D image plane using the intrinsic parameter matrix, outputting the pixel coordinates of each point and its depth value in the camera coordinate system. Finally, visibility filtering is performed. Using the projected pixel coordinates, depth values, and virtual mesh coordinates as input, and by setting the field of view (FOV) range and distance threshold, points located within the view frustum and at reasonable distances are selected, and the visible point mask and its corresponding original point cloud index are output. Finally, Z-buffer rasterization is performed, using the pixel coordinates, depth values, and point indices of the visible points as input. By retaining the point with the minimum depth for each pixel and coloring the corresponding pixel according to color information, the rendered image, the depth map recording the minimum depth, the index map recording the point index, and the index list of all visible points are output, thereby establishing the correspondence between pixels and the original point cloud index.
[0032] In this embodiment, the pixel coordinates of a three-dimensional spatial point projected onto the two-dimensional image plane in the camera coordinate system after rotation, translation, and intrinsic parameter matrix projection are: The camera projection transformation formula is as follows: ; in: These are the homogeneous coordinates of pixels on the image plane. To construct the intrinsic parameter matrix of the virtual camera, To construct the extrinsic parameter matrix of the virtual camera, For rotation matrix, It is a translation matrix. The coordinates of a point in three-dimensional space are homogeneous in the world coordinate system.
[0033] The visibility filtering formula is as follows: ; in: Indicates the first The depth value of a 3D point in the camera coordinate system after extrinsic parameter transformation (i.e., its depth value in the camera coordinate system). (Coordinates along the axis) and projected onto the pixel position. ; Indicates correspondence The minimum depth value at a pixel location is used to determine the corresponding 3D point as a visible point at that pixel, and a visible point index is constructed accordingly.
[0034] Stage 1 includes a main-view SAM segmentation module, a unified component construction module, and an auxiliary-view SAM automatic segmentation module connected in series: The main view SAM segmentation module is used to perform SAM segmentation on the main view information generated by Stage0 (such as the rendering images, index images, depth maps, and corresponding view angles of two main views, yaw=0°, pitch=0° and yaw=180°, pitch=0°) based on human interactive prompts, and output the main view segmentation results. Specifically, it includes: First, performing human interactive annotation. Taking the rendering images of the two main views as input, prompt information is generated for each track structure component by manually drawing bounding boxes on the image, and a series of bounding box coordinates are output. Next, SAM is called to perform zero-sample segmentation. Taking the rendering image and the manually annotated bounding boxes as input, the image encoder, prompt encoder, and mask decoder of the SAM model are used to output the accurate 2D segmentation mask corresponding to each bounding box. Finally, visible and invisible points are extracted. Taking the segmentation mask and the corresponding index image as input, the visible point index is obtained by mapping the mask region to the index image, and the difference between all visible points under this view and the visible points of the current mask is calculated as the invisible points. The main view segmentation results containing bounding boxes, segmentation masks, a list of visible points, and a list of invisible points are output. The unified component construction module is used to construct a standardized component point set across different viewpoints based on the main viewpoint segmentation results. The specific steps for outputting the standardized component point set through the unified component construction module include: First, calculating the cross-viewpoint mask overlap (IoU), taking the set of visible points of each mask in the two main viewpoints as input, and outputting the IoU overlap matrix by calculating the ratio of intersection to union; Next, evaluating the component center distance, taking the 3D coordinates of the visible points of each mask in the two main viewpoints as input, and outputting the distance matrix by calculating the median center of the point cloud and calculating the Euclidean distance (using Euclidean distance to measure the geometric proximity across viewpoints); Then, merging duplicate components, taking the IoU overlap matrix, the distance matrix, and a preset threshold as input, and by judging whether the IoU exceeds the threshold or the distance is less than the threshold, taking the union of the set of visible points of the mask that meets the conditions, and outputting a list of standardized component point sets to eliminate component redundancy caused by viewpoint differences; The formula for calculating the cross-view mask overlap (IoU) is as follows: ; in: , A set of point indices for candidates of the same component across different viewpoints; , Component center distance The calculation formula is as follows: ; in: , The set of point indices for the component to be matched. , They are respectively , The center point of the component; The component matching determination formula is as follows: ; in: This is an indicator function; it takes a value of 1 when the condition is true (indicating a match) and a value of 0 when the condition is false (indicating a mismatch); IoU represents the cross-view mask overlap. Distance between component centers Set a threshold for IoU matching. for Distance threshold.
[0035] The auxiliary view SAM automatic segmentation module is used to perform SAM segmentation based on the rendering data of the auxiliary view and the standardized component point set output by the unified component construction module, and output the auxiliary view segmentation result. The auxiliary view SAM automatic segmentation module takes the auxiliary view's rendering data (rendered image, index image, depth map) and the standardized component point set output by the unified component building module as input. After processing, it outputs the auxiliary view segmentation result. The specific steps include: First, generating tooltips based on the component point set reprojection. Taking the standardized component point set and the auxiliary view's index image as input, it finds the pixels belonging to each component point set and calculates their boundaries, outputting a series of bounding box tooltips. Next, it calls SAM in batches for automatic segmentation. Taking the auxiliary view's rendered image and the generated tooltips as input, it automatically segments using the SAM model and outputs the segmentation mask corresponding to each tooltip. Finally, it extracts visibility information and aligns across viewpoints. Taking the segmentation mask and index image as input, it extracts visible and invisible points using the same method, ensuring consistency with the unified component point set, and outputs the auxiliary view's segmentation mask and visibility information.
[0036] Stage 2 includes a cross-view component pairing module and a statistical scoring fusion module: The cross-view component pairing module is used to establish corresponding relationships between component groups based on the main view segmentation results obtained in Stage 1 and the original point cloud 3D coordinate output, so as to realize the association of the same physical component under different views. Using the segmentation results from the two main perspectives obtained in Stage 1 (including the visible and invisible point indices of each mask) and the original point cloud 3D coordinates as input, the cross-view component pairing module achieves accurate association of the same physical component under different perspectives. Specifically: 1) Calculate the 3D center position of the visible point. Using the visible point index and point cloud coordinates of each mask as input, the 3D center coordinates of each mask are output by calculating the median position of the visible point in 3D space. 2) Construct a distance matrix and an IoU overlap matrix. Using the center coordinates of all masks and the set of visible points in the two main perspectives as input, the distance matrix is obtained by calculating the Euclidean distance between the centers, and the IoU overlap matrix is obtained by calculating the intersection-union ratio of the visible point sets. 3) Optimal matching based on thresholds. Using the distance matrix, IoU matrix, and preset distance and IoU thresholds as input, the optimal matching mask in perspective 2 is found for each mask in perspective 1, which simultaneously satisfies the condition that the distance is less than the threshold and the IoU is greater than the threshold. If no such mask exists, the nearest mask is selected. The output is the component grouping that establishes the corresponding relationship, thus achieving accurate association of the same physical component under different perspectives. The accurate correlation process of the same physical component from different perspectives is represented by the following formula: ; in: For point right Support level, representing the number of points in multi-view rendering. How frequently is it judged to belong to a component? ; The index of the point; Assign component numbers; Numbering for the viewpoint; A collection of all perspectives; For point From the perspective The visible weights ; For point From the perspective Invisible weights ; and Representing components respectively From the perspective The set of visible points and the set of invisible points below.
[0037] The statistical scoring fusion module is used to generate preliminary segmentation labels and confidence weights for point clouds based on the main view segmentation results, auxiliary view segmentation results, and component grouping output by the cross-view component pairing module, through statistical scoring. Using the segmentation results from all perspectives (including primary and secondary perspectives) obtained in Stage 1 and the component grouping output by the cross-perspective component pairing module as input, the statistical scoring fusion module generates preliminary segmentation labels and confidence weights for the point cloud. Specifically: First, a statistical scoring matrix is constructed. Taking the total number of points in the point cloud, component grouping, and all perspective segmentation results as input, a zero matrix of point count × component count is initialized. For each component, visible points under each perspective are accumulated with high weights (e.g., 1.0), and invisible points are accumulated with low weights (e.g., 0.5), outputting the statistical scoring matrix. Next, labels are assigned based on the highest score principle. Taking the statistical scoring matrix as input, the component with the highest score for each point is selected as its label. If all component scores are zero, it is marked as background, outputting the preliminary segmentation labels for the point cloud. Simultaneously, confidence weights are calculated. Taking the highest score for each point as input, the score is used as the confidence level of the point's label, outputting the point-level confidence weights. This module ultimately outputs point cloud segmentation labels, a confidence weight matrix, and detailed statistical information.
[0038] The formula for assigning labels is shown below: ; in: For point The final tag value; For normalization , indicating a point Belongs to components The probability of.
[0039] The formula for the confidence level of the predicted point is as follows: ; in: For point The confidence level is used as the point weight during subsequent training.
[0040] Stage 3 includes a tag optimization module, a semantic mapping module, and a data export module connected in sequence: The label optimization module is used to optimize the initial segmentation labels of the point cloud and output the final point cloud segmentation labels. Taking the point cloud coordinates output from Stage 2 and the sparse initial segmentation labels (including background label 0) as input, the label optimization module optimizes the initial segmentation labels of the point cloud. Specifically: First, iterative label propagation based on radius neighborhood is performed. Taking the point cloud coordinates and the original labels as input, a KD-Tree is constructed to query the neighbors of each background point (label 0) within a fixed radius. If the proportion of non-background points in the neighbors exceeds a threshold and the number meets the requirements, the label of the background point is updated to the mode label among the neighbors. This process is repeated multiple times until convergence, outputting intermediate labels with enhanced geometric consistency. Next, DBSCAN is performed. Clustering noise filtering takes the aforementioned intermediate labels and point cloud coordinates as input. By traversing each non-background category, it performs spatial clustering using DBSCAN on its points, identifies and removes clusters labeled as noise (-1) or with too few points, resets the labels of these points to background, and outputs the noise-filtered labels. Finally, it performs small cluster removal and optional reassignment. Taking the filtered labels and point cloud coordinates as input, it performs nearest neighbor search by constructing a KD-Tree with the coordinates of the removed points and the remaining foreground points, and reassigns the labels based on the majority vote of the K nearest neighbors, outputting the optimized final point cloud segmentation labels. The iterative label propagation process based on radius neighborhood is implemented through the following formula: ; in: For point Tags; Candidate tags; Indicated by point Centered on The set of neighborhood points with radius; For neighborhood points The current tag; The threshold representing the percentage of neighbors required for label propagation; Set a lower limit on the number of neighbors to prevent excessive spread to isolated points.
[0041] To prevent misjudgment of isolated small clusters while maintaining spatial coherence with larger clusters, the small cluster distance is preserved as shown in the following formula: ; in: As the center of the small cluster, As the center of the large cluster, For the set of all large clusters, The distance threshold is set; when the distance between the center of a small cluster and any large cluster is less than the set distance threshold, the small cluster may be part of the large cluster and is retained without removal; otherwise, it is regarded as a noise point and can be removed.
[0042] The semantic mapping module is used to map the final point cloud segmentation labels to semantic categories and generate mapping files. Taking the point cloud coordinates output by the label optimization module and the instance segmentation label (component ID) as input, the semantic mapping module generates mapping files. Specifically: First, it performs component geometric feature statistical analysis. Taking the point set coordinates of each instance component as input, it calculates its 3D bounding box, bounding box size, number of points, PCA principal components (eigenvalues and principal axis directions), and median, outputting the geometric feature description vector for each component. Next, it performs heuristic rule matching. Taking the geometric feature description vector as input, it matches against a preset rule base (e.g., flat in the Z direction and large in the XY plane as "track slab," and slender in the horizontal direction as "rail"), outputting mapping suggestions and confidence scores for each instance component to preset semantic categories (such as beam, slab, pier, column). To ensure the reliability of the labels, the generated mapping suggestions do not directly generate mapping files; mapping files are only generated after confirmation. The final mapping file supports manual modification.
[0043] The data export module is used to export a point cloud training dataset that meets the requirements. Taking a semantic mapping file (a mapping table from instances to semantic categories), point cloud coordinates, original segmentation labels, and a scoring matrix generated by Stage 2 as input, the data export module exports a point cloud training dataset that meets the requirements. Specifically, it applies semantic mapping and calculates point-level weights. Taking the original instance labels and mapping file as input, it converts instance IDs to semantic category IDs by querying the mapping file. At the same time, it uses the statistical scoring matrix to calculate the confidence of each point's category as the point-level weight, and outputs a point cloud with semantic labels and weights. Then, it organizes a standardized dataset structure. Taking the processed point cloud, labels, and weights as input, it standardizes file naming, creates a standard directory, and generates metadata files, outputting a complete point cloud training dataset that meets the requirements of frameworks such as PointNet / PointNet2.
[0044] The PointNet++-based weakly supervised point cloud segmentation module includes a data loading module and a weakly supervised training module; the PointNet++-based weakly supervised point cloud segmentation module is used to perform weakly supervised training using the point cloud training data with pseudo-labels and confidence weights. The data loading module is used to load the point cloud training dataset and perform random sampling, data augmentation, and feature channel expansion to generate batch training samples. Specifically: First, scene data is loaded and sampled. Taking scene point cloud coordinates and labels as input, a fixed number of points (e.g., 4096) are randomly selected (if the number of points is insufficient, sampling is repeated), and a fixed number of point cloud blocks and corresponding labels are output. Next, data augmentation is performed. Taking the sampled point cloud blocks as input, the points are randomly rotated around the Z-axis and small-amplitude coordinate jitter is added to output augmented point cloud blocks. Then, feature channel expansion is performed. Taking the augmented point cloud blocks (containing only XYZ coordinates) as input, six zero-value channels are concatenated after the coordinates to output a nine-channel feature vector (shape [npoints, 9]). Finally, the module outputs batch training samples containing nine-channel feature vectors, semantic labels, and point-level weights.
[0045] The weakly supervised training module is used to train the PointNet++ model using the batch training samples and a weighted cross-entropy loss function that combines point-level confidence weights and class balance weights. The trained point cloud semantic segmentation model is obtained through dynamic learning rate decay and model validation. Specifically: First, point-level confidence weight loss is calculated. Taking the model's predicted probability distribution, true semantic labels, and point-level weights as input, the cross-entropy loss is calculated, multiplied by the point-level weights and class weights, and then weighted and normalized to output the weighted loss value for the current batch. Next, class balancing weights are adjusted. Taking the frequency of each semantic class in the training set as input, the loss weight for each class is calculated using inverse frequency smoothing (lower-frequency classes have higher weights), outputting the class weight vector used for loss calculation. Then, dynamic learning rate decay is performed. Taking the current training epoch and a preset decay step size as input, the optimizer's learning rate is adjusted using a step-down strategy, outputting the updated learning rate. Finally, model validation and saving are completed. Taking the validation set data as input, the model performance is evaluated by calculating the overall accuracy and the mean Intersection over Union (mIoU) for each class, and model parameters are saved when performance improves. This module ultimately outputs the trained point cloud semantic segmentation model and its performance evaluation report.
[0046] To improve the learning weights for points with high confidence while reducing the interference of noisy points or minority classes on the gradient, the weakly supervised training module employs a weighted cross-entropy loss function. for: ; in: For point Confidence level, To balance the weights for each category, For PointNet++ model point The predicted probability distribution For point The true label category, Cross-entropy loss; ; in: For indicator functions, when The time is 1; Point Belongs to the category of real labels The predicted probability, This indicates the total number of categories.
[0047] Step S3: Preprocess the unlabeled point cloud dataset obtained in step S1, and combine it with the SAM-based point cloud processing module to obtain point cloud training data with pseudo-labels and confidence weights. The point cloud data obtained in step S1 is imported into the SAM-based point cloud processing module constructed in step S2. The point cloud processing module maps the original point cloud into multi-view images based on the visualization relationship of the point cloud and establishes the correspondence between points and pixels. Combining the geometric features of the long strip structure of the track structure, the two main view directions of the point cloud are determined. With the prompts of the module, the component prompt boxes of the main view are drawn. The module will automatically call the SAM model to realize automatic mask generation and merge unified components. Based on the correspondence between the mask of the main view and the point cloud, the module will automatically generate prompt boxes and SAM object segmentation masks in the auxiliary view. Based on the multi-view mask fusion and component matching, the module will pass the mask segmentation results as pseudo-labels to the 3D point cloud and generate label mapping suggestions based on the geometric features of the point cloud with pseudo-labels. Combined with a small amount of manual judgment, the mapping relationship between component instance pseudo-labels and category pseudo-labels is established to obtain 3D point cloud with high confidence point-level pseudo-labels and point-by-point label confidence, which is imported into the point cloud semantic segmentation model as training data.
[0048] Step S4: Input the point cloud training data obtained in Step S3 into the weakly supervised point cloud segmentation module based on PointNet++ for training to obtain the trained point cloud semantic segmentation model. Use the trained point cloud semantic segmentation model to perform semantic segmentation on the unlabeled point cloud obtained in Step S1 to obtain semantic segmentation point cloud components for subsequent quantitative analysis of track structure damage.
[0049] To verify the effectiveness of the method of the present invention, the following experimental comparisons were conducted: A weakly supervised segmentation method for orbital structure point clouds based on SAM and PointNet++ was experimentally validated on the classic S3DIS dataset and compared with the performance of fully supervised PointNet and PointNet++ methods on the same dataset. In the experiments, the proposed method was trained and tested under two main perspectives and 16 auxiliary perspectives. Semantic segmentation evaluation metrics were used to assess the results, including overall accuracy (Overall Acc) and mean intersection over union (mIoU). A higher Overall Acc indicates more accurately identified and segmented points, while a higher mIoU indicates better results across all categories.
[0050] Table 1 Comparison of Experimental Results
[0051] Referring to Table 1, the experimental results show that by introducing SAM to generate pseudo-labels for weak supervision, this method achieves semantic segmentation accuracy close to that of the fully supervised PointNet++ model, significantly outperforming fully supervised PointNet, even with only a small number of labeled samples. Furthermore, the introduction of a pseudo-label confidence mechanism effectively improves the model's robustness to noisy labels, verifying the feasibility and effectiveness of the proposed weak supervision strategy in 3D point cloud segmentation tasks.
[0052] To verify the robustness of the proposed method in track structure scenarios, experiments were conducted on a self-built track structure point cloud dataset. To further demonstrate the engineering value of the proposed method, an additional evaluation metric, annotation time, was introduced. Shorter annotation time indicates a lower dependence of the proposed method on high-precision annotation.
[0053] Table 2 Segmentation Accuracy and Annotation Schedule
[0054] Referring to Table 2, the experimental results show that the method proposed in this invention achieves high accuracy in all classic categories of track structures, and also achieves an accuracy of nearly 70% in fasteners that are easily occluded. This accuracy is very close to and slightly higher than the segmentation accuracy of PointNet++ under fully supervised conditions. At the same time, the method of this invention requires a much shorter annotation time than that under fully supervised conditions. In practical engineering applications, this method can significantly reduce training costs and training time, and has high engineering application value.
[0055] This invention also provides a weakly supervised segmentation system for track structure point clouds based on SAM and PointNet++, including a data acquisition module, a SAM-based point cloud processing module, a PointNet++-based weakly supervised point cloud segmentation module, and a semantic segmentation execution module. The data acquisition module collects track structure point cloud data to obtain an unlabeled point cloud dataset with RGB color. The SAM-based point cloud processing module processes the input point cloud data to generate point cloud training data with pseudo-labels and confidence weights. The PointNet++-based weakly supervised point cloud segmentation module uses the point cloud training data with pseudo-labels and confidence weights for weakly supervised training to obtain a trained point cloud semantic segmentation model. The semantic segmentation execution module uses the trained point cloud semantic segmentation model to perform semantic segmentation on the input unlabeled track structure point cloud to be segmented.
[0056] The above description is only a preferred embodiment of the present invention and does not limit the scope of the present invention. All equivalent structural transformations made under the inventive concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the protection scope of the present invention.
Claims
1. A weakly supervised segmentation method for orbital structure point clouds based on SAM and PointNet++, characterized in that, Includes the following steps: Step S1: Collect point cloud data of the track structure to obtain an unlabeled point cloud dataset with RGB colors; Step S2: Construct a point cloud processing module based on SAM and a weakly supervised point cloud segmentation module based on PointNet++; wherein: The SAM-based point cloud processing module comprises four parts: Stage0, Stage1, Stage2, and Stage3, which are connected in sequence. The SAM-based point cloud processing module is used to process the input point cloud data and generate point cloud training data with pseudo-labels and confidence weights. Stage0 includes a preprocessing module and a rendering module connected in series. The preprocessing module is used to standardize the coordinates of the input point cloud data. The rendering module is used to generate multi-view rendering maps, depth maps, index maps, and index lists based on the standardized coordinate information, color information, and view parameters. Stage 1 includes a first-view SAM segmentation module, a unified component construction module, and an auxiliary-view SAM automatic segmentation module connected in series. The first-view SAM segmentation module is used to perform SAM segmentation on the first-view information generated in Stage 0 based on human interaction prompts, and output the first-view segmentation result. The unified component construction module is used to construct a standardized component point set across perspectives based on the first-view segmentation result. The auxiliary-view SAM automatic segmentation module is used to perform SAM segmentation on the auxiliary-view rendering data and the standardized component point set output by the unified component construction module, and output the auxiliary-view segmentation result. The specific steps for outputting a standardized component point set through the unified component building module include: First, calculating the cross-view mask overlap (IoU). Taking the set of visible points of each mask in the two main views as input, the IoU overlap matrix is output by calculating the ratio of the intersection to the union. Next, evaluating the component center distance. Taking the 3D coordinates of the visible points of each mask in the two main views as input, the distance matrix is output by calculating the median center of the point cloud and the Euclidean distance. Then, merging duplicate components. Taking the IoU overlap matrix, the distance matrix, and a preset threshold as input, the set of visible points of the masks that meet the conditions is merged by determining whether the IoU exceeds the threshold or whether the distance is less than the threshold, and outputting the standardized component point set. Stage 2 includes a cross-view component pairing module and a statistical scoring fusion module. The cross-view component pairing module is used to establish corresponding component groups based on the main view segmentation results and the original point cloud 3D coordinate output obtained from Stage 1, so as to realize the association of the same physical component under different views. The statistical scoring fusion module is used to generate preliminary segmentation labels and confidence weights of the point cloud through statistical scoring based on the main view segmentation results, auxiliary view segmentation results and component groups output by the cross-view component pairing module. Stage 3 includes a label optimization module, a semantic mapping module, and a data export module connected in sequence. The label optimization module is used to optimize the initial segmentation labels of the point cloud and output the final point cloud segmentation labels. The semantic mapping module is used to map the final point cloud segmentation labels to semantic categories and generate a mapping file. The data export module is used to export a point cloud training dataset that meets the requirements. The PointNet++-based weakly supervised point cloud segmentation module includes a data loading module and a weakly supervised training module; the PointNet++-based weakly supervised point cloud segmentation module is used to perform weakly supervised training using the point cloud training data with pseudo-labels and confidence weights. Step S3: Preprocess the unlabeled point cloud dataset obtained in step S1, and combine it with the SAM-based point cloud processing module to obtain point cloud training data with pseudo-labels and confidence weights. Step S4: Input the point cloud training data obtained in Step S3 into the weakly supervised point cloud segmentation module based on PointNet++ for training to obtain the trained point cloud semantic segmentation model. Use the trained point cloud semantic segmentation model to perform semantic segmentation on the unlabeled point cloud obtained in Step S1 to obtain semantic segmentation point cloud components.
2. The weakly supervised segmentation method for orbital structure point clouds based on SAM and PointNet++ as described in claim 1, characterized in that, In Stage 0: The preprocessing module performs the following geometric transformations on the input point cloud data in sequence: PCA principal component analysis alignment, centering, rotation around the Y-axis, translation and scaling, and outputs standardized virtual mesh coordinates; The rendering module generates multi-view rendering maps, depth maps, index maps, and index lists by constructing a virtual camera, performing camera projection transformation, executing visibility filtering, and implementing Z-buffer rasterization, based on standardized virtual mesh coordinates, color information, and view parameters.
3. The weakly supervised segmentation method for orbital structure point clouds based on SAM and PointNet++ as described in claim 2, characterized in that, In Stage 1: The specific steps for outputting the main view segmentation results through the main view SAM segmentation module include: performing manual interactive annotation, drawing bounding boxes on the main view rendering map, generating prompt information for each track structure component, and outputting a series of bounding box coordinates; calling the SAM model to perform zero-shot segmentation to obtain the 2D segmentation mask corresponding to each bounding box; extracting visible and invisible points, and outputting the main view segmentation results containing bounding boxes, segmentation masks, a list of visible points, and a list of invisible points. The specific steps for outputting auxiliary view segmentation results through the auxiliary view SAM automatic segmentation module include: generating bounding box hints on the auxiliary view index map based on component point sets; batch calling SAM for automatic segmentation; extracting visibility information and aligning it across views, and outputting the segmentation mask and visibility information of the auxiliary view.
4. The weakly supervised segmentation method for orbital structure point clouds based on SAM and PointNet++ as described in claim 3, characterized in that, In Stage 2: The cross-view component pairing module achieves accurate association of the same physical component under different viewpoints. Specifically, it calculates the 3D center position of the visible points. Taking the visible point index and point cloud coordinates of each mask as input, it calculates the median position of the visible points of the mask in 3D space and outputs the 3D center coordinates of each mask. Next, it constructs a distance matrix and an IoU overlap matrix. Taking the center coordinates of all masks and the set of visible points of two main viewpoints as input, it calculates the Euclidean distance between the centers to obtain the distance matrix, and simultaneously calculates the intersection-union ratio of the set of visible points to obtain the IoU overlap matrix. Then, based on the threshold-based optimal matching, taking the distance matrix, the IoU overlap matrix, and preset distance and IoU thresholds as input, it searches for the best matching mask in viewpoint two that simultaneously satisfies the condition that the distance is less than the threshold and the IoU is greater than the threshold for each mask in viewpoint one. If no such mask exists, it selects the closest mask and outputs the component grouping that establishes the corresponding relationship, thus achieving accurate association of the same physical component under different viewpoints. The statistical scoring fusion module generates preliminary segmentation labels and confidence weights for the point cloud. Specifically, it first constructs a statistical scoring matrix, taking the total number of points in the point cloud, component grouping, and segmentation results from all views as input. By initializing a zero matrix of point count × component count, it accumulates high weights for visible points and low weights for invisible points of each component in each view, outputting the statistical scoring matrix. Next, it assigns labels based on the highest score principle, taking the statistical scoring matrix as input, and selecting the component with the highest score for each point as its label. If all component scores are zero, they are marked as background, outputting the preliminary segmentation labels for the point cloud. Simultaneously, it calculates confidence weights, taking the highest score for each point as input, and using the score as the confidence level of the point's label, outputting the point-level confidence weights.
5. The weakly supervised segmentation method for orbital structure point clouds based on SAM and PointNet++ as described in claim 4, characterized in that, The initial segmentation labels of the point cloud are optimized through a label optimization module. Specifically, firstly, iterative label propagation based on radius neighborhood is performed. Taking the point cloud coordinates and the original labels as input, a KD-Tree is constructed to query the neighbors of each background point within a fixed radius. If the proportion of non-background points in the neighbors exceeds a threshold and the number meets the requirements, the label of the background point is updated to the mode label among the neighbors. After multiple iterations until convergence, intermediate labels with enhanced geometric consistency are output. Next, DBSCAN clustering noise filtering is performed. Taking the above intermediate labels and point cloud coordinates as input, DBSCAN is used to perform spatial clustering on the points belonging to each non-background category. Clusters marked as noise or with too few points are identified and removed, and the labels of these points are reset to background. The labels after noise filtering are output. Finally, small cluster removal and optional reassignment are implemented. Taking the filtered labels and point cloud coordinates as input, a nearest neighbor search is performed by constructing a KD-Tree with the coordinates of the removed points and the remaining foreground points. The labels are reassigned based on the majority vote of the K nearest neighbors, and the optimized final point cloud segmentation labels are output. The semantic mapping module generates a mapping file by: First, performing component geometric feature statistical analysis, taking the point set coordinates of each instance component as input, calculating its 3D bounding box, bounding box size, number of points, PCA principal components, and median, and outputting the geometric feature description vector of each component; then, performing heuristic rule matching, taking the geometric feature description vector as input, matching through a preset rule base, outputting mapping suggestions and confidence scores for each instance component to a preset semantic category, and generating a mapping file. The data export module exports a point cloud training dataset that meets the requirements. Specifically, it applies semantic mapping and calculates point-level weights. Taking the original instance labels and mapping files as input, it converts instance IDs into semantic category IDs by querying the mapping files. At the same time, it uses a statistical scoring matrix to calculate the confidence score of each point's category as the point-level weight, and outputs a point cloud with semantic labels and weights. Next, it organizes a standardized dataset structure. Taking the processed point cloud, labels, and weights as input, it standardizes file naming, creates a standard directory, and generates metadata files, outputting a complete point cloud training dataset that meets the requirements.
6. The weakly supervised segmentation method for orbital structure point clouds based on SAM and PointNet++ as described in claim 5, characterized in that, The data loading module is used to load the point cloud training dataset and perform random sampling, data augmentation, and feature channel expansion to generate batch training samples.
7. The weakly supervised segmentation method for orbital structure point clouds based on SAM and PointNet++ as described in claim 6, characterized in that, The weakly supervised training module is used to train the PointNet++ model using the batch training samples and a weighted cross-entropy loss function that combines point-level confidence weights and class balance weights. The trained point cloud semantic segmentation model is obtained through dynamic learning rate decay and model validation.
8. The weakly supervised segmentation method for orbital structure point clouds based on SAM and PointNet++ as described in claim 7, characterized in that, The weakly supervised training module uses the weighted cross-entropy loss function. for: ； in: For point Confidence level, To balance the weights for each category, For PointNet++ model point The predicted probability distribution For point The true label category, Cross-entropy loss; ； in: For indicator functions, Point Belongs to the category of real labels The predicted probability, This indicates the total number of categories.
9. A weakly supervised segmentation system for orbital structure point clouds based on SAM and PointNet++, used to implement the method as described in any one of claims 1-8, characterized in that, It includes a data acquisition module, a SAM-based point cloud processing module, a PointNet++-based weakly supervised point cloud segmentation module, and a semantic segmentation execution module; The data acquisition module is used to acquire point cloud data of the track structure to obtain an unlabeled point cloud dataset with RGB colors. The SAM-based point cloud processing module is used to process the input point cloud data and generate point cloud training data with pseudo-labels and confidence weights. The weakly supervised point cloud segmentation module based on PointNet++ is used to perform weakly supervised training using the point cloud training data with pseudo-labels and confidence weights to obtain the trained point cloud semantic segmentation model. The semantic segmentation execution module is used to perform semantic segmentation on the input unlabeled track structure point cloud to be segmented using the trained point cloud semantic segmentation model.