A bridge point cloud zero-sample instance segmentation method fusing geometric prior and semantic guidance
By integrating geometric priors and semantic guidance, and using SAM and SegGPT models to select unobstructed viewpoints and generate high-precision two-dimensional segmentation masks, the problem of insufficient cross-scene adaptability and robustness in bridge point cloud segmentation is solved, and efficient zero-sample instance segmentation of bridge point clouds is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA RAILWAY BRIDGE BUREAU GRP EIGHTH ENG CO LTD
- Filing Date
- 2026-04-10
- Publication Date
- 2026-07-31
AI Technical Summary
Existing bridge point cloud instance segmentation methods lack cross-scene adaptability and robustness. In particular, the scarcity of labeled bridge scene data leads to high training costs and poor generalization. Furthermore, existing 2D vision models lack effective viewpoint quality assessment and instance consistency processing in 3D point cloud segmentation, resulting in insufficient segmentation accuracy and geometric consistency.
By integrating geometric priors and semantic guidance, and utilizing the SAM and SegGPT models, combined with visual prompting and hierarchical mask fusion strategies, unobstructed viewpoints are selected to generate high-precision two-dimensional segmentation masks, which are then inversely mapped to three-dimensional space. Finally, zero-shot instance segmentation of bridge point clouds is achieved by combining graph connectivity analysis.
Zero-sample instance segmentation of bridge point clouds was achieved, reducing engineering deployment costs, improving generalization ability across bridge scenarios, ensuring the accuracy and geometric consistency of segmentation results, and solving the data bottleneck and segmentation inconsistency problems of existing methods.
Smart Images
Figure CN122493039A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of information technology and bridge safety management, and in particular to a zero-sample instance segmentation method for bridge point clouds that integrates geometric priors and semantic guidance. Background Technology
[0002] With the rapid development of 3D laser scanning and photogrammetry technologies, point cloud data has become an important data source for bridge reverse modeling, digital operation and maintenance, and structural health monitoring. Semantic segmentation and instance segmentation of bridge point clouds, as core tasks of 3D scene understanding, aim to automatically classify the massive amounts of collected discrete points into semantically meaningful component categories and further distinguish different individual instances of the same type of component. This is a key technological support for realizing bridge digital twins and intelligent operation and maintenance.
[0003] In the existing technical approach, bridge point cloud instance segmentation methods mainly fall into two categories. The first is heuristic methods, which typically segment components based on the geometric features of the point cloud combined with manually designed rules. These methods offer advantages such as high computational efficiency and no need for training data, making them suitable for scenarios with simple structures and good point cloud quality. However, heuristic methods heavily rely on prior domain knowledge; rule design and parameter adjustment must be tailored to specific bridge structures, making it difficult to adapt to structural differences between different bridge types. They also have limited generalization ability and are sensitive to point cloud noise and occlusion, resulting in insufficient robustness. The second category is deep neural network methods based on supervised learning. These methods train a point cloud processing network on a large-scale labeled dataset, using an end-to-end approach to directly predict semantic labels and instance attribution from the point cloud, achieving good performance on general 3D scene segmentation benchmarks. However, supervised learning-based methods require a large number of manually labeled 3D point cloud samples for training. Due to the complexity of bridge structures, high data acquisition costs, and strong confidentiality requirements, publicly available bridge point cloud labeled datasets are extremely scarce, leading to high training costs and poor cross-scene generalization for these methods, resulting in a serious data bottleneck in practical engineering applications.
[0004] In recent years, with the rise of basic visual models, 2D visual models such as SAM, SegGPT, and SEEM, which possess powerful zero-shot and few-shot segmentation capabilities, have received widespread attention and are gradually being introduced into 3D point cloud segmentation tasks. The core idea of these methods is to render point clouds as multi-view images, leverage the strong generalization ability of 2D basic models to obtain semantic segmentation masks, and then inversely map the mask results back to 3D space, thereby bypassing the dependence on large-scale 3D labeled data and providing a new technical direction for point cloud segmentation in open scenes. However, existing technical solutions still have the following significant shortcomings and defects:
[0005] A. Existing methods for transferring 2D vision basic models to 3D point cloud segmentation lack an effective viewpoint quality assessment mechanism in the multi-view image rendering stage, failing to exclude viewpoints with low information content due to occlusion, which affects the quality and coverage integrity of subsequent segmentation masks.
[0006] B. Existing transfer methods neglect instance consistency between views during cross-view reasoning. The segmentation results of the same component from different views lack effective correlation and fusion, which can easily lead to instance loss or inconsistent cross-view masks, reducing the accuracy of the final 3D segmentation.
[0007] C. Existing methods do not fully utilize the geometric characteristics of the point cloud itself when inversely mapping a two-dimensional mask to three-dimensional space. They ignore the spatial distribution and normal vectors of the point cloud, resulting in missegmentation at the component boundaries and poor geometric consistency of the segmentation results.
[0008] D. Existing methods lack an effective conflict resolution mechanism in the semantic label allocation stage, and their handling of superpoints with multiple semantic labels and unlabeled superpoints is simple and crude, resulting in insufficient accuracy of superpoint semantic normalization, which in turn affects the accuracy of subsequent instance segmentation.
[0009] Therefore, developing a zero-sample instance segmentation method for bridge point clouds that integrates geometric priors and semantic guidance is of great significance. Summary of the Invention
[0010] The purpose of this invention is to provide a zero-sample instance segmentation method for bridge point clouds that integrates geometric priors and semantic guidance, so as to solve the problems existing in the prior art.
[0011] The technical solution adopted to achieve the purpose of this invention is as follows: a zero-shot instance segmentation method for bridge point clouds that integrates geometric priors and semantic guidance, comprising the following steps:
[0012] S1) Preprocess the input bridge point cloud data to generate a superpoint set and filter out unobstructed rendering viewpoints.
[0013] S2) Based on the unobstructed viewpoints selected in step S1), generate multi-view RGB images, depth images, and the mapping relationship between point clouds and pixels.
[0014] S3) The SAM and SegGPT models are concatenated, and a visual self-cue and hierarchical mask fusion strategy is used to obtain a high-precision two-dimensional segmentation mask for each viewpoint.
[0015] S4) Inverse map the high-precision two-dimensional segmentation mask from each perspective to the three-dimensional superpoint space and extract the confidence semantic superpoints.
[0016] S5) Based on semantic vectors, semantic optimization is performed on the superpoints, and a unique semantic label is assigned to each superpoint.
[0017] S6) Construct a semantically identical superpoint graph for superpoints belonging to the same semantics based on spatial relationships, and complete the three-dimensional instance segmentation of bridge components based on graph connectivity analysis.
[0018] S7) For superpoints that still have no instance labels, calculate the frequency of instance labels of superpoints in their neighborhood, and take the label with the highest frequency as the instance label of the superpoint to obtain the final bridge point cloud semantics and instance segmentation results.
[0019] Furthermore, step S1) specifically includes the following sub-steps:
[0020] S1.1) By combining spatial vectors, normal vectors, and RGB information, and using a combination of VCCS and region growing, the point cloud is divided into superpoints with similar geometric features, generating a set of superpoints.
[0021] S1.2) Generate a set of candidate viewpoints around the center of the point cloud at fixed angular intervals. Use an orthogonal camera to render each candidate viewpoint to obtain the corresponding image. Calculate the image entropy of each candidate viewpoint based on the grayscale distribution. Filter and remove low-entropy occluded viewpoints whose image entropy is lower than the threshold.
[0022] S1.3) Calculate the disparity of each candidate viewpoint. If the disparity distribution is large, it is determined that there is an occluded viewpoint. Viewpoints with disparities greater than the mean disparity are marked as occluded viewpoints and removed. K viewpoints are uniformly selected from the remaining unoccluded viewpoints as the final rendering viewpoints. If the disparity distribution is uniform, it is determined that there are no occluded viewpoints. K viewpoints are uniformly selected directly from the candidate viewpoint set as the final rendering viewpoints.
[0023] Furthermore, step S3) specifically includes the following sub-steps:
[0024] S3.1) Manually select a small number of semantic points and background points as cues, and input them into the SAM model along with the first image to generate an initial mask for the target component. Select the mask with the highest SAM prediction confidence as the initial visual cue.
[0025] S3.2) Input the initial set of visual cues and the previous view image into SegGPT, perform visual self-cue reasoning, and output the original reasoning mask.
[0026] S3.3) Utilizing the camera's extra-camera depth information, the fused mask from the previous viewpoint is projected onto the current image to obtain a cross-viewpoint mask. Using the cross-viewpoint mask as a reliable anchor point, K-nearest neighbor pixels are selected in the original SegGPT inference mask for region expansion to correct local missegmentation and obtain a single-viewpoint optimized mask.
[0027] S3.4) Perform the inference and optimization processes of S3.2) and S3.3) on the RGB image and depth image respectively. Using the depth image mask as a basis, fuse the semantic details of the RGB image mask to obtain a single-view high-precision two-dimensional segmentation mask, represented as:
[0028]
[0029] in, and They represent the first The final fusion mask obtained from the RGB image and depth image from each viewpoint.
[0030] Furthermore, step S4) specifically includes the following sub-steps:
[0031] S4.1) Based on the mapping relationship between points and superpoints and the rendering relationship of each viewpoint, the two-dimensional semantic mask of each viewpoint is inversely mapped to the three-dimensional superpoint space to obtain the superpoint mask corresponding to each viewpoint. The set of superpoint masks of all viewpoints is summarized as the superpoint of interest.
[0032] S4.2) For each superpoint in the superpoints of interest, if more than half of the points fall within the corresponding semantic mask, then the superpoint is determined to be a confidence semantic superpoint.
[0033] Furthermore, step S5) specifically includes the following sub-steps:
[0034] S5.1) Assign a binary semantic vector to each superpoint and determine its initial semantic label.
[0035] S5.2) For a superpoint with multiple semantic labels, calculate the score of each semantic component by combining the semantic vector of the neighboring superpoints. The weights are set to 0.8 and 0.2 respectively, and the semantic component with the highest score is used as the unique semantic label.
[0036] S5.3) For unlabeled superpoints, assign the semantic label that appears most frequently in the neighborhood to complete the semantic normalization of all superpoints.
[0037] Furthermore, step S6) specifically includes the following sub-steps:
[0038] S6.1) Construct a superpoint graph for superpoints with the same semantic meaning, and establish connecting edges between superpoints that are neighbors of each other.
[0039] S6.2) Based on graph connectivity, the superpoint graph is divided into several connected components. Connected components with a number of superpoints reaching a preset threshold are determined as independent component instances, thus completing the instance segmentation of semantic components.
[0040] The technical effects of this invention are beyond doubt:
[0041] A. By merging the SAM and SegGPT 2D visual models and introducing a visual self-hinting mechanism, zero-shot instance segmentation of bridge point clouds was achieved. This effectively overcomes the data bottleneck caused by the extreme scarcity of labeled bridge scene data in supervised learning methods, significantly reduces the deployment cost in actual engineering projects, and also has the ability to generalize across bridge scenes.
[0042] B. By calculating the image entropy of the rendered image corresponding to the candidate viewpoint, viewpoints with low information content due to occlusion are automatically identified and eliminated. The unoccluded area is further determined by combining the disparity of the opposite side, thereby ensuring that the final selected rendering viewpoint has high information richness and spatial coverage uniformity, laying the foundation for obtaining high-quality segmentation masks in the future.
[0043] C. Project the fusion mask of the previous view onto the current image as a reliable anchor point, combine it with the K-nearest neighbor pixel region expansion to correct the original SegGPT inference mask, and fuse the mask results of the RGB image and the depth image. This effectively ensures the consistency of the segmentation results of the same component instance across multiple views, avoids the problem of instance loss and cross-view mask inconsistency, and improves the accuracy of the two-dimensional segmentation mask.
[0044] D. In the point cloud preprocessing stage, a super-pointing method based on VCCS and region growing is introduced, which makes full use of the prior geometric structure of the point cloud, effectively avoids the boundary missegmentation problem caused by direct mask mapping at the point level, and ensures the geometric consistency of the 3D segmentation results. Attached Figure Description
[0045] Figure 1 A flowchart illustrating an embodiment of the present invention.
[0046] Figure 2 Example diagram for viewing point selection using entropy-based methods
[0047] Figure 3 Schematic diagram of the mask optimization process
[0048] Figure 4 This is a diagram showing the results of partial semantic segmentation and instance segmentation. Detailed Implementation
[0049] The present invention will be further described below with reference to embodiments, but it should not be construed that the scope of the present invention is limited to the following embodiments. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the above-described technical concept of the present invention should be included within the scope of protection of the present invention.
[0050] Example 1:
[0051] This embodiment provides a zero-shot instance segmentation method for bridge point clouds that integrates geometric priors and semantic guidance, including the following steps:
[0052] S101. Preprocess the bridge point cloud data, generate a set of superpoints based on geometric features, and filter viewpoints using an image entropy-based method to complete the rendering of multi-view RGB images and depth images.
[0053] S102, the SAM and SegGPT models are cascaded together. A strategy of visual self-prompting and hierarchical mask fusion is adopted. Based on semantic prompting, the domain transfer of the two-dimensional model is realized to obtain a high-precision segmentation mask.
[0054] S103. Inversely map the two-dimensional segmentation mask to three-dimensional space, complete the matching of superpoints and the mask, and extract the confidence semantic superpoints;
[0055] S104. Perform semantic optimization on the superpoints based on semantic vectors and assign a unique semantic label to each superpoint;
[0056] S105. Based on spatial relationships, construct a semantic superpoint graph for superpoints belonging to the same semantic type, and complete the three-dimensional instance segmentation of bridge components based on graph connectivity analysis.
[0057] S106. For superpoints without instance labels, use the frequency of instance labels of superpoints in the neighborhood of the superpoint as the instance label to obtain the final point cloud instance segmentation result.
[0058] Preferably, step S101 includes:
[0059] First, including but not limited to using a combination of VCCS and region growing, the point cloud is divided into superpoints with similar geometric features by integrating spatial vectors, normal vectors and RGB information.
[0060] Secondly, for the input bridge point cloud, generate around the center of the point cloud. A set of candidate viewpoints is used. Images are rendered using an orthogonal camera. Image entropy is calculated for each candidate viewpoint. Image entropy indicates the information richness of the image. Occluded viewpoints have low image entropy due to concentrated grayscale distribution, and vice versa. Low-entropy occluded viewpoints are thus selected. The set of low-entropy viewpoints is as follows: From the perspective The corresponding image entropy;
[0061]
[0062] Finally, the unobstructed area is determined based on heterolateral disparity. The heterolateral disparity calculation formula is as follows: From the perspective The set of viewpoints, The number of viewpoints is determined by the disparity distribution on the opposite side. If the disparity distribution is large, it is determined that there are obstructed viewpoints. Viewpoints with disparity on the opposite side greater than the mean disparity are marked as obstructed viewpoints and removed. The remaining unobstructed areas are then evenly selected. If one viewpoint is selected as the final rendering viewpoint, and conversely, if the disparity distribution on the opposite side is uniform, then an unobstructed viewpoint is determined, and a viewpoint is directly and uniformly selected from the candidate viewpoint set. Each viewpoint is used as the final rendering viewpoint, thereby rendering an RGB image, a depth image, and a mapping relationship between point clouds and pixels from the selected viewpoint.
[0063]
[0064] Preferably, step S102 includes:
[0065] First, a small number of semantic and background points are manually selected and input into the SAM model along with the first image to generate the initial mask for the target components in the image. The mask with the highest SAM prediction confidence is selected as the basis for subsequent hints;
[0066] Secondly, the initial visual cues Inputting the mask result from the previous image into SegGPT yields the original inference mask. The corresponding formula is as follows;
[0067]
[0068] Then, the prediction results are optimized through a layered mask fusion strategy, which combines the fusion mask from the previous viewpoint. Project onto the current image This yields an accurate but sparse cross-view mask. .by For reliable anchor points, in the original SegGPT mask The K nearest neighbor pixels are selected for region expansion to correct local missegmentation and obtain a single-view optimized mask. ;
[0069] Finally, in the result fusion stage, the fused depth image mask is used as the initial mask to optimize the RGB image and the depth image, resulting in a high-precision two-dimensional segmentation mask from a single viewpoint. The corresponding optimization formula is as follows. and They represent the first The final fusion mask obtained from the RGB image and depth image from each viewpoint.
[0070]
[0071] Preferably, step S103 includes:
[0072] First, given components A set of 2D prediction masks The mapping relationship between the base point and the superpoint and the rendering relationship of each viewpoint The k-th semantic mask Inverse mapping to the corresponding superpoint yields the superpoint mask. ,in It is a mapping operator, and the set of superpoint masks of all viewpoints is used as a component. More than 100 points of interest;
[0073]
[0074]
[0075] Secondly, for the superpoints in the superpoints of interest If more than half of the points fall within the semantic mask In this case, the superpoint is considered a confidence semantic superpoint. For the semantic mask that falls on The number of points in the middle, for The number of points contained.
[0076]
[0077]
[0078] Preferably, step S104 includes:
[0079] First, assign a binary semantic vector to each superpoint. ,in The number of components determines the semantics of the superpoint and its corresponding semantic tag;
[0080] Secondly, for multi-semantic tag superpoints One of the multi-semantic superpoints Its semantic vector is Combined with its One neighbor exceeded the limit. The semantic score is calculated using the following formula, where, It belongs to the neighbor's excess points The semantic vector of the superpoint. It belongs to the neighbor's excess points The semantic vector of the superpoint, in the experiment, weight , The values were set to 0.8 and 0.2 respectively, and the semantic component with the highest score was used as the sole semantic label to reassign the label. ;
[0081]
[0082] Finally, for unlabeled superpoints, assign the semantic label that appears most frequently in the neighborhood to complete the semantic normalization of all superpoints. Classified according to semantic tags , .
[0083] Preferably, step S105 includes:
[0084] First, construct a superpoint graph for superpoints with the same semantic meaning, and establish connecting edges between superpoints that are neighbors of each other; second, divide the superpoint graph into several connected components based on connectivity; finally, determine the connected components with a sufficient number of superpoints as independent instances, thus completing the instance segmentation of components with the same semantic meaning.
[0085] Preferably, step S106 includes:
[0086] For a superpoint without an instance label, calculate the frequency of instance labels of superpoints in its neighborhood, and use the label with the highest frequency as the instance label of that superpoint.
[0087] Example 2:
[0088] This embodiment provides a zero-shot instance segmentation method for bridge point clouds that integrates geometric priors and semantic guidance for image segmentation, including the following steps:
[0089] S101. Divide the point cloud into superpoints based on geometric features, and filter unobstructed viewpoints based on image entropy, then render RGB and depth images.
[0090] In specific implementation, firstly, including but not limited to using a combination of VCCS and region growing, the point cloud is divided into superpoints with similar geometric features by integrating spatial vectors, normal vectors and RGB information;
[0091] Secondly, for the input bridge point cloud, generate around the center of the point cloud. A set of candidate viewpoints is used. Images are rendered using an orthogonal camera. Image entropy is calculated for each candidate viewpoint. Image entropy indicates the information richness of the image. Occluded viewpoints have low image entropy due to concentrated grayscale distribution, and vice versa. Low-entropy occluded viewpoints are thus selected. The set of low-entropy viewpoints is as follows: From the perspective The corresponding image entropy;
[0092]
[0093] Finally, the unobstructed area is determined based on heterolateral disparity. The heterolateral disparity calculation formula is as follows: From the perspective The set of viewpoints, The number of viewpoints is determined by the disparity distribution on the opposite side. If the disparity distribution is large, it is determined that there are obstructed viewpoints. Viewpoints with disparity on the opposite side greater than the mean disparity are marked as obstructed viewpoints and removed. The remaining unobstructed areas are then evenly selected. If one viewpoint is selected as the final rendering viewpoint, and conversely, if the disparity distribution on the opposite side is uniform, then an unobstructed viewpoint is determined, and a viewpoint is directly and uniformly selected from the candidate viewpoint set. Each viewpoint is used as the final rendering viewpoint, thereby rendering an RGB image, a depth image, and a mapping relationship between point clouds and pixels from the selected viewpoint.
[0094]
[0095] In an example, the viewpoint selection process based on entropy in this invention is as follows: Figure 2 As shown.
[0096] S102 and SAM generate initial hints, SegGPT self-prompting inference, and layered mask fusion optimizes the mask result;
[0097] In practice, firstly, a small number of semantic and background points are manually selected and input into the SAM model along with the first image to generate the initial mask for the target component in the image. The mask with the highest SAM prediction confidence is selected as the basis for subsequent hints;
[0098] Secondly, the initial visual cues Inputting the mask result from the previous image into SegGPT yields the original inference mask. The corresponding formula is as follows;
[0099]
[0100] Then, the prediction results are optimized through a layered mask fusion strategy, which combines the fusion mask from the previous viewpoint. Project onto the current image This yields an accurate but sparse cross-view mask. .by For reliable anchor points, in the original SegGPT mask The K nearest neighbor pixels are selected for region expansion to correct local missegmentation and obtain a single-view optimized mask. ;
[0101] Finally, in the result fusion stage, the fused depth image mask is used as the initial mask to optimize the RGB image and the depth image, resulting in a high-precision two-dimensional segmentation mask from a single viewpoint. The corresponding optimization formula is as follows. and They represent the first The final fusion mask obtained from the RGB image and depth image from each viewpoint.
[0102]
[0103] In this invention, the optimized masking process is integrated into the example as follows: Figure 3 As shown.
[0104] S103. Inverse map the two-dimensional mask to three-dimensional superpoints and filter the confidence semantic superpoints.
[0105] In practice, firstly, given the components A set of 2D prediction masks The mapping relationship between the base point and the superpoint and the rendering relationship of each viewpoint The k-th semantic mask Inverse mapping to the corresponding superpoint yields the superpoint mask. ,in It is a mapping operator, and the set of superpoint masks of all viewpoints is used as a component. More than 100 points of interest;
[0106]
[0107]
[0108] Secondly, for the superpoints in the superpoints of interest If more than half of the points fall within the semantic mask In this case, the superpoint is considered a confidence semantic superpoint. For the semantic mask that falls on The number of points in the middle, for The number of points contained.
[0109]
[0110]
[0111] S104. Construct semantic vectors and iteratively optimize multi-label and unlabeled superpoints to ensure that each superpoint has only a unique semantic label.
[0112] In practice, firstly, a binary semantic vector is assigned to each superpoint. ,in The number of components determines the semantics of the superpoint and its corresponding semantic tag;
[0113] Secondly, for multi-semantic tag superpoints One of the multi-semantic superpoints Its semantic vector is Combined with its One neighbor exceeded the limit. The semantic score is calculated using the following formula, where, It belongs to the neighbor's excess points The semantic vector of the superpoint. It belongs to the neighbor's excess points The semantic vector of the superpoint, in the experiment, weight , The values were set to 0.8 and 0.2 respectively, and the semantic component with the highest score was used as the sole semantic label to reassign the label. ;
[0114]
[0115] Finally, for unlabeled superpoints, assign the semantic label that appears most frequently in the neighborhood to complete the semantic normalization of all superpoints. Classified according to semantic tags , .
[0116] S105. Construct a superpoint graph for superpoints with the same semantic meaning, and generate independent component instances using connectivity analysis;
[0117] In practice, firstly, a superpoint graph is constructed for superpoints with the same semantic meaning, and connecting edges are established between superpoints that are neighbors of each other; secondly, the superpoint graph is divided into several connected components based on connectivity; finally, connected components with a sufficient number of superpoints are determined as independent instances, thus completing the instance segmentation of components with the same semantic meaning.
[0118] S106. Assign high-frequency instance labels to the neighborhood of superpoints without instance labels to obtain the final segmentation result.
[0119] In practice, for superpoints without instance labels, the frequency of instance labels of superpoints in their neighborhood is calculated, and the label with the highest frequency is used as the instance label of that superpoint.
[0120] The specific semantic segmentation and instance segmentation results of this method are shown in [link to relevant documentation]. Figure 4 The above experimental results demonstrate that the zero-shot instance segmentation method for bridge point clouds that integrates geometric priors and semantic guidance disclosed in this invention can complete zero-shot instance segmentation of bridge point cloud data in an unsupervised manner, and the method described in this invention is indeed effective.
Claims
1. A zero-sample instance segmentation method for bridge point clouds that integrates geometric priors and semantic guidance, characterized in that, Includes the following steps: S1) Preprocess the input bridge point cloud data to generate a superpoint set and filter out unobstructed rendering viewpoints; S2) Based on the unobstructed viewpoints selected in step S1), generate multi-view RGB images, depth images, and the mapping relationship between point clouds and pixels. S3) Connect the SAM and SegGPT models and use a visual self-prompting and hierarchical mask fusion strategy to obtain a high-precision two-dimensional segmentation mask for each viewpoint; S4) Inverse map the high-precision two-dimensional segmentation mask of each viewpoint to the three-dimensional superpoint space and extract the confidence semantic superpoints; S5) Perform semantic optimization on the superpoints based on semantic vectors and assign a unique semantic label to each superpoint; S6) Construct a semantically identical superpoint graph for superpoints belonging to the same semantics based on spatial relationships, and complete the three-dimensional instance segmentation of bridge components based on graph connectivity analysis; S7) For superpoints that still have no instance labels, calculate the frequency of instance labels of superpoints in their neighborhood, and take the label with the highest frequency as the instance label of the superpoint to obtain the final bridge point cloud semantics and instance segmentation results.
2. The method for zero-sample instance segmentation of bridge point clouds by fusing geometric priors and semantic guidance as described in claim 1, characterized in that, Step S1) specifically includes the following sub-steps: S1.1) By combining spatial vectors, normal vectors, and RGB information, and using a combination of VCCS and region growing, the point cloud is divided into superpoints with similar geometric features, generating a set of superpoints; S1.2) Generate a set of candidate viewpoints around the center of the point cloud at fixed angle intervals. Use an orthogonal camera to render each candidate viewpoint to obtain the corresponding image. Calculate the image entropy of each candidate viewpoint based on the grayscale distribution. Filter and remove low-entropy occluded viewpoints whose image entropy is lower than the threshold. S1.3) Calculate the disparity of each candidate viewpoint. If the disparity distribution is large, it is determined that there is an occluded viewpoint. Viewpoints with disparity greater than the mean disparity are marked as occluded viewpoints and removed. K viewpoints are uniformly selected from the remaining unoccluded viewpoints as the final rendering viewpoints. If the disparity distribution is uniform, it is determined that there is no occluded viewpoint. K viewpoints are uniformly selected directly from the candidate viewpoint set as the final rendering viewpoints.
3. The method for zero-sample instance segmentation of bridge point clouds integrating geometric priors and semantic guidance according to claim 1, characterized in that, Step S3) specifically includes the following sub-steps: S3.1) Manually select a small number of semantic points and background points as cues, and input them into the SAM model along with the first image to generate an initial mask for the target component. Select the mask with the highest SAM prediction confidence as the initial visual cue. S3.2) Input the initial set of visual cues and the previous view image into SegGPT, perform visual self-cue reasoning, and output the original reasoning mask; S3.3) Using the camera's extra-camera depth information, the fusion mask of the previous view is projected onto the current image to obtain a cross-view mask; using the cross-view mask as a reliable anchor point, K nearest neighbor pixels are selected in the original SegGPT inference mask for region expansion, local missegmentation is corrected, and a single-view optimized mask is obtained. S3.4) Perform the inference and optimization processes of S3.2) and S3.3) on the RGB image and depth image respectively. Using the depth image mask as a basis, fuse the semantic details of the RGB image mask to obtain a single-view high-precision two-dimensional segmentation mask, represented as: in, and They represent the first The final fusion mask obtained from the RGB image and depth image from each viewpoint.
4. The zero-sample instance segmentation method for bridge point clouds integrating geometric priors and semantic guidance according to claim 1, characterized in that, Step S4) specifically includes the following sub-steps: S4.1) Based on the mapping relationship between points and superpoints and the rendering relationship of each viewpoint, the two-dimensional semantic mask of each viewpoint is inversely mapped to the three-dimensional superpoint space to obtain the superpoint mask corresponding to each viewpoint. The set of superpoint masks of all viewpoints is summarized as the superpoint of interest. S4.2) For each superpoint in the superpoints of interest, if more than half of the points fall within the corresponding semantic mask, then the superpoint is determined to be a confidence semantic superpoint.
5. The method for zero-sample instance segmentation of bridge point clouds by fusing geometric priors and semantic guidance according to claim 1, characterized in that, Step S5) specifically includes the following sub-steps: S5.1) Assign a binary semantic vector to each superpoint and determine its initial semantic label; S5.2) For a superpoint with multiple semantic labels, calculate the score of each semantic component by combining the semantic vector of the neighboring superpoints, with weights set to 0.8 and 0.2 respectively, and use the semantic component with the highest score as the unique semantic label. S5.3) For unlabeled superpoints, assign the semantic label that appears most frequently in the neighborhood to complete the semantic normalization of all superpoints.
6. The method for zero-sample instance segmentation of bridge point clouds integrating geometric priors and semantic guidance according to claim 1, characterized in that, Step S6) specifically includes the following sub-steps: S6.1) Construct a superpoint graph for superpoints with the same semantic meaning, and establish connecting edges between superpoints that are neighbors of each other; S6.2) Based on graph connectivity, the superpoint graph is divided into several connected components. Connected components with a number of superpoints reaching a preset threshold are determined as independent component instances, thus completing the instance segmentation of semantic components.