Label-free three-dimensional point cloud segmentation method based on visual large model

Abstract

The invention relates to the field of three-dimensional vision, in particular to a label-free three-dimensional point cloud segmentation method based on a visual large model, which comprises the following steps: acquiring three-dimensional point cloud data, and extracting a boundary point set according to the three-dimensional point cloud data in combination with curvature and normal vector to construct a target function; solving the objective function by using an optimization algorithm, selecting an optimal projection visual angle combination, and extracting two-dimensional features of all two-dimensional projection images and three-dimensional features of three-dimensional point cloud data by using a visual large model; performing consistency matching on the two-dimensional features and the three-dimensional features through a matching algorithm to obtain a plurality of matching results; acquiring a definition weight factor and a coverage rate weight factor to calculate an importance weight; fusing the two-dimensional features according to the weight factors to obtain overall fused two-dimensional features; and inversely mapping the integrally fused two-dimensional features to a three-dimensional space to realize three-dimensional point cloud unmarked segmentation. The three-dimensional point cloud segmentation method has the effect of realizing three-dimensional point cloud automatic segmentation without labels.

Description

Technical Field

[0001] This application relates to the field of 3D vision, and in particular to a method for unlabeled 3D point cloud segmentation based on large visual models. Background Technology

[0002] A 3D point cloud is a data set composed of a large number of discrete points in three-dimensional space. Each point contains its coordinate information in 3D space. In addition, points can have additional attributes, such as color and classification labels. Currently, 3D point clouds, as a powerful way to represent 3D spatial information, serve as a bridge connecting the physical and digital worlds. With the continuous maturation of sensor technology and the rapid development of artificial intelligence (especially deep learning) algorithms, the processing power, accuracy, and application scope of point cloud technology are rapidly expanding, making it undoubtedly one of the core technologies of the future intelligent sensing and digital age.

[0003] For segmentation tasks based on 3D point clouds, the mainstream paradigm is still to train the network with labeled supervised data. Large visual segmentation models already have powerful semantic segmentation capabilities in the 2D image domain, and can accurately output pixel-level semantic masks and high-dimensional semantic features of the target. 3D point clouds provide spatial geometric information. The feature-level alignment of the two is essentially a cross-modal fusion of "semantic features" and "geometric features". The accuracy of feature alignment is the core prerequisite for ensuring the reliability of cross-modal fusion.

[0004] When aligning large visual segmentation models with 3D scenes at the feature level, large visual segmentation models are often trained on 2D image datasets, which are not robust enough to boundary segmentation of perspective distortion and occluded scenes, resulting in boundary blurring issues. Summary of the Invention

[0005] To improve boundary clarity, this application provides a label-free 3D point cloud segmentation method based on a large visual model:

[0006] Acquire 3D point cloud data and calculate the saliency of boundary features of each point in the point cloud data; adaptively adjust the neighborhood of each point based on the saliency of boundary features, and extract the set of boundary points by combining curvature and normal vector mutation features.

[0007] The boundary point set is clustered based on its spatial location and normal vector direction. A candidate viewpoint location set is obtained based on the normal vector direction of the clustering results. All combinations of candidate projection viewpoint sets are traversed to construct an objective function with the overall point cloud coverage and the projection sharpness of the boundary points as optimization objectives. The optimal projection viewpoint combination is selected by solving the objective function using an optimization algorithm.

[0008] The two-dimensional features of all two-dimensional projection images and the three-dimensional features of three-dimensional point cloud data are extracted by using a visual large model; consistency matching is performed on the two-dimensional features and the three-dimensional features by using a matching algorithm, and a plurality of matching results are obtained; all local fused two-dimensional features are obtained based on the matching results;

[0009] The projection clarity of the boundary points in the specified view angle is calculated as a clarity weight factor; the coverage rate of the specified view angle relative to the overall point cloud quantity is calculated as a coverage rate weight factor, and the importance weight of the specified view angle is calculated according to the clarity weight factor and the coverage rate weight factor; the importance weights of the specified view angles are obtained in the same way; the local fused two-dimensional features are obtained by weighting the corresponding two-dimensional features according to the importance weights based on the matching results.

[0010] The local fused two-dimensional features are used to be inversely mapped to the three-dimensional space, and a three-dimensional point cloud segmentation result is output.

[0011] Optionally, the boundary feature salience calculation method is as follows:

[0012] The overall density of the three-dimensional point cloud data is calculated according to the three-dimensional point cloud data; a target point is selected, the local density of the target point under different neighborhood sizes is calculated, and the neighborhood size closest to the overall density is selected as the initial neighborhood size of the point; the curvature and the normal vector of each point in the point cloud data are calculated.

[0013] The ratio of the curvature of the target point to the average curvature of all points in the initial neighborhood size is taken as a first ratio, and the exponential function value of the first ratio is calculated; the cosine similarity between the target point and all points in the initial neighborhood size is negatively correlated and normalized, and the average value is taken; the product of the exponential function value and the average value is taken as the boundary feature salience of the target point in the initial neighborhood size; the boundary feature saliences of the points are obtained in the same way.

[0014] Optionally, the boundary point set calculation method is as follows: the product of the boundary feature salience of the target point and the initial neighborhood size is taken as the self-adaptive adjustment neighborhood of the target point; the self-adaptive adjustment neighborhoods of the points are obtained in the same way; the adjusted boundary feature salience of each point is calculated according to the self-adaptive adjustment neighborhood; the points with the boundary feature salience greater than a preset threshold are selected as the marker points to obtain the boundary point set.

[0015] Optionally, the candidate view angle position set calculation method is as follows: the product of the normalized Euclidean distance between any two points and the cosine similarity between the normalized normal vector directions is taken as the clustering distance between any two points, a plurality of clustering clusters and the clustering centers of the clustering clusters are obtained, and the normal vector direction of the clustering center is taken as the candidate view angle; the three-dimensional point cloud centroid is calculated, and the candidate view angle position set is obtained according to the distances between the candidate view angle, the clustering center and the three-dimensional point cloud centroid.

[0016] Optionally, the calculation method of the target function is: taking any candidate view position as a target projection view position; calculating a cosine similarity between a projection direction in the target projection view position and a normal vector of any boundary point as a first cosine similarity; taking a difference between 1 and an absolute value of the first cosine similarity as a first difference value, calculating a first normalized Euclidean distance between the target projection view position and any boundary point, calculating a ratio of a product of the first difference value and the first normalized Euclidean distance to a number of boundary points in the target projection view position, traversing the ratio of each boundary point in all target projection view positions to obtain a first definition of the target projection view position, traversing the first definition of all projection view combinations to obtain a first sum value, and calculating a product of the first sum value and the first difference value to obtain the target function.

[0017] Optionally, the calculation method of the definition weight factor is: calculating a second cosine similarity between a projection direction normal vector in a specified view and a normal vector of any boundary point, taking a difference between 1 and an absolute value of the second cosine similarity as a second difference value, calculating a second normalized Euclidean distance between the specified view and any boundary point in the specified view, calculating a ratio of a product of the second difference value and the second normalized Euclidean distance to a number of boundary points in the specified view, traversing the ratio of each boundary point in the specified view to obtain a second definition of the specified view, and obtaining the definition weight factor after normalization.

[0018] Optionally, the calculation method of the importance weight is: taking a ratio of a number of point clouds in a specified projection view position to a total number of point clouds as a coverage weight factor; and calculating a product of the definition weight factor and the coverage weight factor to obtain the importance weight.

[0019] Optionally, the calculation method of the local fused two-dimensional feature is: calculating a weighted average of two-dimensional features of a local object according to the importance weight to obtain the local fused two-dimensional feature.

[0020] Optionally, the method of screening points with a boundary feature saliency greater than a threshold value as marker points is: calculating a threshold value by using an OTSU algorithm, and screening out points with a boundary feature saliency greater than the threshold value.

[0021] The present application has the following technical effects:

[0022] 1. By designing the dual weights of projection definition and view coverage, the contribution value of each projection view is quantitatively calculated. Among them, the definition weight factor considers the view direction consistency and observation distance, effectively highlighting the view information with clear boundaries and rich details; and the coverage weight factor ensures that the view covering a wider range of point clouds occupies a higher proportion in the fusion process, and optimizes the utilization efficiency of view resources as a whole.

[0023] 2. The consistency matching of two-dimensional and three-dimensional object features is realized by the Hungarian algorithm, and weighted fusion and reverse mapping are carried out on this basis. This method not only guarantees the alignment quality of the two-dimensional and three-dimensional feature spaces, but also significantly improves the object integrity and consistency of the three-dimensional point cloud segmentation result after projection fusion, providing a more reliable data foundation for subsequent three-dimensional scene understanding tasks. In addition, by adaptively adjusting the neighborhood through the boundary feature saliency, a set of boundary points is extracted, which can improve the accuracy of boundary point extraction and the quality of the projection image obtained subsequently; in addition, the optimal projection view combination is selected by using an optimization algorithm, which can maximize the retention of boundary information while eliminating redundant projection views, significantly improving the segmentation efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a label-free three-dimensional point cloud segmentation method based on a visual large model. DETAILED DESCRIPTION

[0025] The embodiments of the present application disclose a label-free three-dimensional point cloud segmentation method based on a visual large model, referring to Figure 1 , including steps S1-S8, as follows:

[0026] S1: Obtain three-dimensional point cloud data and calculate the boundary feature saliency of each point in the point cloud data; adaptively adjust the neighborhood of each point according to the boundary feature saliency.

[0027] Calculate the overall density of the three-dimensional point cloud data according to the three-dimensional point cloud data; the overall density is equal to the total number of points in the three-dimensional point cloud divided by the volume of the space covered by the point cloud.

[0028] Select any point as a target point and calculate the local density of the target point under different neighborhood sizes.

[0029] The calculation method of local density includes fixed radius neighborhood method, k nearest neighbor distance method and kernel density estimation method, which are prior art and will not be described here.

[0030] Take a single point as the center, count the number of points in its fixed radius neighborhood, and select the neighborhood size with the closest local density to the overall density as the initial neighborhood size of the point.

[0031] The ratio of the curvature of the target point to the mean curvature of all points within the initial neighborhood is used as the first ratio. The exponential function value of the first ratio is calculated. The cosine similarity between the target point and all points within the initial neighborhood is negatively correlated and normalized, and then the mean is taken. The product of the exponential function value and the mean is calculated to obtain the boundary feature saliency of the target point within the initial neighborhood. Similarly, the boundary feature saliency of each point is obtained, and the product of the boundary feature saliency of the target point and the initial neighborhood size is calculated to obtain the adaptive adjusted neighborhood of the target point. Similarly, the adaptive adjusted neighborhood of each point is obtained. The adjusted boundary feature saliency of each point is calculated based on the adaptive adjusted neighborhood.

[0032] In one embodiment, the formula for calculating the adaptively adjusted neighborhood can be:

[0033]

[0034] k i =k base ×A i (k base )k i To adaptively adjust the neighborhood; calculate the overall density ρ0,k of the 3D point cloud data. base It is the initial neighborhood size of the target point i whose local density is closest to ρ0, and serves as the basis for adjustment. A i (k base ) is the size k of the initial neighborhood of target point i. base The saliency of boundary features within the cloud is calculated. The curvature and normal vector of each point in the 3D point cloud are calculated using the fitted surface method, where α... i It is the curvature of the target point i. Centered on target point i, with a size of k base The mean curvature of all points in the neighborhood. It can reflect the relative magnitude of the curvature of the target point i in its neighborhood. The larger the ratio, the greater the curvature of the target point is compared with other points in the initial neighborhood size. The more significant its boundary features are, the smaller the neighborhood range should be to preserve details.

[0035] cos(n i ,n i′ ) represents the target point i and the initial neighborhood size k. base The cosine similarity between the normal vectors of any two points reflects the degree of similarity in the directions of their normal vectors. It is the mean of the normalized cosine similarity between the target point i and the normal vectors of all points in its neighborhood. Normalization is to avoid the negative impact of cosine similarity on the calculation of boundary feature significance. The smaller this value is, the lower the similarity between the target point i and the normal vectors of other points in its neighborhood. This means that the probability of a sudden change in the normal vector of the target point i is higher, and its boundary feature significance is higher. The neighborhood range should be reduced to retain more detailed information.

[0036] The curvature and normal vector of each point in the point cloud data are calculated; the curvature and normal vector of each point in the three-dimensional point cloud are calculated by using a fitting surface method, and the prior art will not be described again.

[0037] The local density of each point in the three-dimensional point cloud is used to adaptively adjust the neighborhood in combination with the boundary feature saliency, which overcomes the problem of missing or misjudging the boundary points caused by uneven point cloud density, and effectively eliminates the misjudgment interference caused by background noise and isolated points in sparse areas. The extraction accuracy and integrity of the boundary points in the point cloud with complex density distribution are greatly improved.

[0038] S2: Extracting a boundary point set in combination with the curvature and normal vector mutation characteristics.

[0039] Specifically, points with high boundary feature saliency are selected as marker points to obtain a boundary point set.

[0040] In one embodiment, after the local density of each point in the three-dimensional point cloud is used to adaptively adjust the neighborhood in combination with the boundary feature saliency, the boundary feature saliency of each point in the adaptively adjusted neighborhood is calculated, and the threshold value generated by the OTSU (Otsu's Maximum Between-Class Variance Method) algorithm is used to screen out points with boundary feature saliency meeting the threshold value, which are marked as boundary points to obtain a boundary point set B for subsequent analysis.

[0041] In combination with the adaptively adjusted neighborhood, the boundary features of each point in its neighborhood are extracted to obtain a boundary point set, improve the boundary point extraction accuracy, improve the projection image quality in the subsequent projection view angle selection process for the boundary points, and greatly reduce the error.

[0042] S3: Clustering the boundary point set based on spatial position and normal vector direction.

[0043] In one embodiment, it is assumed that the world coordinate system of the three-dimensional point cloud is a right-handed coordinate system with the Z-axis upward, and the virtual camera faces the centroid (average value of the axis coordinates of all points) of the three-dimensional point cloud, and the distance between the virtual camera and the centroid is fixed, such as 2 times the radius of the entire three-dimensional point cloud. The point cloud radius refers to the distance from the centroid of the point cloud to the farthest point in the point cloud, and the specific position of the virtual camera (i.e., the view angle position of the photographed point cloud image) is selected by the corresponding projection effect.

[0044] After the boundary point set is extracted, an effective view angle direction needs to be generated based on the boundary point distribution, but since the boundary points are scattered, they need to be aggregated to form meaningful boundary region clusters, and then the projection view angle candidate set is generated based on this to ensure that the projection can fully capture the boundary details.

[0045] In one embodiment, a DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm is used to cluster based on the position and normal vector direction of each point in the set of boundary points, and in the clustering process, the clustering distance between each two points is calculated by the product of the normalized Euclidean distance and the cosine similarity between the normalized normal vector directions, avoiding dimensional conflicts. The clustering radius is set to 0.1, and the minimum number of points is 3. The boundary points with similar positions and normal vector directions in the set of boundary points are clustered into one class, and a plurality of clustering clusters and corresponding clustering centers are obtained. The normal vector direction of each clustering center point is taken as a candidate view angle, and the point with a distance of twice the radius of the three-dimensional point cloud from the centroid of the clustering center point in the normal vector direction is taken as a candidate view angle position. Here, the fixed distance between the virtual camera and the centroid of the point cloud is to ensure that the projection covers the entire three-dimensional point cloud, which can be determined according to the actual situation.

[0046] By clustering, the points on different surfaces can be avoided from being clustered into one class, and errors can be avoided.

[0047] S4: Obtain a set of candidate view angle positions according to the normal vector directions of the clustering results, traverse all combinations of the set of candidate projection view angles, and construct an objective function with the overall point cloud coverage rate and the boundary point projection clarity as the optimization targets.

[0048] In one embodiment, any candidate view angle position is taken as a target projection view angle position; the cosine similarity between the projection direction of the target projection view angle position and the normal vector of any boundary point is calculated as a first cosine similarity; the difference between 1 and the absolute value of the first cosine similarity is taken as a first difference value, the first normalized Euclidean distance between the target projection view angle position and any boundary point is calculated, the product of the first difference value and the first normalized Euclidean distance is calculated, and the ratio of the product to the number of boundary points in the target projection view angle position is obtained. Traverse the ratio of each boundary point in all target projection view angle positions to obtain the first clarity of the target projection view angle position, traverse the first clarity of all projection view angle combinations to obtain a first sum value, and calculate the product of the first sum value and the first difference value to obtain the objective function.

[0049] Specifically, the calculation method of the objective function is:

[0050]

[0051] wherein,

[0052]

[0053] In the formula, V(r) is the set of point clouds covered under the target projection view angle r, |P| is the number of complete point clouds, is the coverage rate of all the perspective pairs in the projection perspective combination R on the overall point cloud. In order to ensure that each projection perspective can completely cover the entire three-dimensional point cloud, it is necessary to constrain The smaller the value of S(R) is, the better the coverage of the projection perspective combination R is. The larger the value of S(R) is, the more redundant perspectives there are, which will cause resource waste and cause more algorithm loss for subsequent projection image segmentation.

[0054] C(r) = B∩V(r) indicates the intersection of the boundary point set B and the point cloud set covered by the perspective r, that is, the boundary point set covered by the perspective r, and N(C(r)) is the number of boundary points covered by the target projection perspective position r. |cos(r, n i )| is the absolute value of the cosine similarity between the projection direction of the target projection perspective position r and the normal vector n i of any boundary point i. If the projection direction is consistent or opposite to the direction of the normal vector of the boundary point, the projection clarity of the boundary point by the projection perspective is higher. If the projection direction is perpendicular to the direction of the normal vector of the boundary point, the projection clarity is lower. is the first normalized Euclidean distance between the target projection perspective position r and any boundary point i. The closer the distance is, the clearer the projection is.

[0055] S5: Use an optimization algorithm to solve the target function to select the optimal projection perspective combination.

[0056] Specifically, all possible combinations in the candidate perspective set are traversed, and an optimization algorithm is used to solve the target function to select the optimal projection perspective combination.

[0057] Specifically, the optimal projection perspective combination formula can be:

[0058] R' = arg min S(R)

[0059] In the formula, R' represents the optimal projection perspective combination, arg min S() represents the minimum target operator, and R represents the candidate projection perspective combination. The optimization algorithm uses a genetic algorithm.

[0060] Using an optimization algorithm to select the optimal projection perspective combination can maximize the retention of boundary information while reducing redundant projection perspectives, thereby improving the segmentation efficiency.

[0061] S6: Extract two-dimensional features of all two-dimensional projection images and three-dimensional features of three-dimensional point cloud data using a visual large model; and perform consistency matching on the two-dimensional features and the three-dimensional features through a matching algorithm to obtain multiple matching results.

[0062] In one embodiment, after obtaining the optimal projection view combination, the three-dimensional point cloud is projected from different views, one projection view corresponds to one two-dimensional projection image; the image segmentation function of SAM (Segment Anything Model) is used to segment the two-dimensional projection image, and the segmentation mask of the corresponding point cloud of each frame of image is obtained by using the pose and depth map, and the two-dimensional features of each object in the segmentation result are extracted; the three-dimensional point cloud convolution network is used to extract the three-dimensional features.

[0063] In order to ensure the consistency of the objects, the two-dimensional features and the three-dimensional features need to be matched for consistency by using the Hungarian algorithm, and a plurality of matching results are obtained.

[0064] The matching cost matrix is calculated based on the feature similarity, which is a prior art and will not be described here.

[0065] S7: obtaining all local fusion two-dimensional features based on the matching results.

[0066] Among them, any projection view in the optimal projection view combination is selected as the specified view, the projection clarity of the boundary point in the specified view is calculated as the clarity weight factor; the coverage rate of the specified view relative to the overall point cloud number is calculated as the coverage rate weight factor, and the importance weight of the specified view is calculated according to the clarity weight factor and the coverage rate weight factor; similarly, the importance weights of the specified views are obtained; according to the matching results, the corresponding two-dimensional features are weighted by the importance weights, and the local fusion two-dimensional features are obtained.

[0067] Among them, the segmentation results of each frame of two-dimensional image are fused into a unified expression based on the matching results, and then inversely mapped to the three-dimensional point cloud. However, since the two-dimensional features obtained by projection under different views have overlapping parts, conflicts may exist during the fusion process. In order to ensure that the fused features are more reliable and consistent, weights need to be assigned to different projection views.

[0068] Specifically, the second cosine similarity between the projection direction normal vector in the specified view and any boundary point normal vector is calculated, the difference between 1 and the absolute value of the second cosine similarity is taken as the second difference, the second normalized Euclidean distance between the specified view and any boundary point in the specified view is calculated, the ratio of the product of the second difference and the second normalized Euclidean distance to the number of boundary points in the specified view is calculated, the second clarity of the specified view is obtained by traversing the ratio of each boundary point in the specified view, and the clarity weight factor is obtained after normalization.

[0069] The ratio of the number of point clouds at the specified projection view position to the number of all point clouds is taken as the coverage rate weight factor.

[0070] The product of the clarity weight factor and the coverage rate weight factor is calculated to obtain the importance weight.

[0071] In one embodiment, the importance weight calculation formula of the r'th projection view angle included in the optimal projection view angle combination R' can be:

[0072]

[0073] i'∈C(r')

[0074] where |cos(r',n i′ )| is the absolute value of the cosine similarity between the projection direction of the specified view angle r' and the normal vector n i′ of any boundary point i' in the specified view angle, and the projection clarity of the projection view angle for the boundary point is higher when the projection direction is consistent or opposite to the normal vector direction of any boundary point. is the normalized Euclidean distance between the position of the specified view angle r' and any boundary point i', and the projection is clearer when the distance is closer.

[0075] N(C(r')) is the number of boundary points covered by the specified view angle r', and the ratio can reflect the projection clarity of the projection view angle r' for the boundary point, and the importance weight of the view angle is higher in the fusion process when the projection clarity is higher. is the coverage of the specified view angle r' relative to the overall three-dimensional point cloud, and the importance weight of the specified view angle is higher in the fusion process when the coverage is higher, indicating that the number of points covered by the specified view angle is larger.

[0076] Specifically, the weighted average of the two-dimensional features of the local object is obtained by calculating the importance weight.

[0077] In the fusion process, when the two-dimensional features of a local object under different view angles correspond to the same three-dimensional feature, they are fused, and the calculation formula of the local fusion two-dimensional feature can be:

[0078]

[0079] where F 2d is the local fusion two-dimensional feature, F 2d (r') is the two-dimensional feature under the specified view angle r', and ω(r') is the importance weight of the specified view angle r'.

[0080] Taking ∑ r′∈R′ ω(r') as the importance weight factor of F 2d (r'), the weighted average of the two-dimensional features under the specified view angle r' is calculated.

[0081] Repeat the above steps to obtain all local two-dimensional features of all objects.

[0082] Since the two-dimensional image segmentation results obtained by projection under different viewing angles have overlapping parts, the importance weight of the projection viewing angle is quantified by setting a projection clarity weight factor and a projection viewing angle coverage weight factor, the high-weight viewing angle features are enhanced, the low-weight viewing angle features are suppressed, in the process of two-dimensional and three-dimensional matching, the fused features pay more attention to the boundary region, the negative influence of the occluded or blurred boundary viewing angle is reduced, and the reliability of the overall feature expression is improved.

[0083] S8: All local fused two-dimensional features are used for inverse mapping to the three-dimensional space to realize three-dimensional point cloud unlabelled segmentation.

[0084] Specifically, according to the projection viewing angle position, all local fused two-dimensional features are combined with the depth map and pose back-projection to the three-dimensional point cloud during projection, and the three-dimensional point cloud segmentation result is output.

[0085] In one embodiment, it should be noted that the projection pose refers to the position (such as X, Y, Z axis coordinates) and attitude (orientation) of the virtual camera in the three-dimensional space, which is a key bridge connecting two-dimensional projection and three-dimensional space. The depth map is an image recording the distance information of each point in the three-dimensional point cloud from the virtual camera, and each pixel value represents the three-dimensional depth (i.e. the distance of the point to the virtual camera) of the corresponding position.

[0086] The three-dimensional point cloud can be automatically segmented without labeling, which greatly reduces the labor and time cost.

[0087] The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application, therefore: any equivalent changes made on the structure, shape, principle of the present application shall be covered within the protection scope of the present application.

Claims

1. A label-free 3D point cloud segmentation method based on a large visual model, characterized in that, include: Acquire 3D point cloud data and calculate the saliency of boundary features of each point in the point cloud data; adaptively adjust the neighborhood of each point based on the saliency of boundary features, and extract the set of boundary points by combining curvature and normal vector mutation features. The boundary point set is clustered based on its spatial location and normal vector direction. A candidate viewpoint location set is obtained based on the normal vector direction of the clustering results. All combinations of candidate projection viewpoint sets are traversed to construct an objective function with the overall point cloud coverage and the projection sharpness of the boundary points as optimization objectives. The optimal projection viewpoint combination is selected by solving the objective function using an optimization algorithm. The visual large model is used to extract the two-dimensional features of all two-dimensional projected images and the three-dimensional features of three-dimensional point cloud data; A matching algorithm is used to perform consistency matching on two-dimensional and three-dimensional features to obtain multiple matching results; based on the matching results, all locally fused two-dimensional features are obtained. Specifically, any projection viewpoint from the optimal combination of projection viewpoints is selected as the specified viewpoint. The projection sharpness of the specified viewpoint on the boundary points is calculated as the sharpness weight factor. The coverage of the specified viewpoint relative to the overall point cloud is calculated as the coverage weight factor. The importance weight of the specified viewpoint is calculated based on the sharpness weight factor and the coverage weight factor. Similarly, the importance weight of each specified viewpoint is obtained. Based on the matching results, the corresponding two-dimensional features are weighted by the importance weights to obtain the local fused two-dimensional features. Similarly, all the local fused two-dimensional features are used to inversely map to three-dimensional space, and the three-dimensional point cloud segmentation result is output.

2. The label-free 3D point cloud segmentation method based on a large visual model according to claim 1, characterized in that, The method for calculating the saliency of the boundary features is as follows: Calculate the overall density of the 3D point cloud data based on the 3D point cloud data; select any point as the target point, calculate the local density of the target point under different neighborhood sizes, select the neighborhood size whose local density is closest to the overall density as the initial neighborhood size of the point; calculate the curvature and normal vector of each point in the point cloud data; The ratio of the curvature of the target point to the mean curvature of all points within the initial neighborhood is used as the first ratio. The exponential function value of the first ratio is calculated. The cosine similarity between the target point and all points within the initial neighborhood is negatively correlated and normalized, and then the mean is taken. The product of the exponential function value and the mean is calculated to obtain the boundary feature saliency of the target point within the initial neighborhood. Similarly, the boundary feature saliency of each point is obtained.

3. The label-free 3D point cloud segmentation method based on a large visual model according to claim 1, characterized in that, The method for calculating the set of boundary points is as follows: The adaptive adjustment neighborhood of the target point is obtained by multiplying the boundary feature saliency of the target point by the initial neighborhood size; similarly, the adaptive adjustment neighborhood of each point is obtained; the adjusted boundary feature saliency of each point is calculated based on the adaptive adjustment neighborhood; points with boundary feature saliency greater than a preset threshold are selected as marker points to obtain the boundary point set.

4. The label-free 3D point cloud segmentation method based on a large visual model according to claim 1, characterized in that, The method for calculating the candidate viewpoint position set is as follows: Calculate the product of the normalized Euclidean distance and the normalized normal vector direction between any two points, and use it as the cluster distance between any two points to obtain multiple clusters and the cluster center of each cluster. Use the normal vector direction of the cluster center as the candidate viewpoint. Calculate the centroid of the 3D point cloud, and obtain the set of candidate viewpoint positions based on the distance between the candidate viewpoint, the cluster center, and the centroid of the 3D point cloud.

5. The label-free 3D point cloud segmentation method based on a large visual model according to claim 1, characterized in that, The objective function is calculated as follows: Take any candidate viewpoint as the target projection viewpoint; calculate the cosine similarity between the projection direction and the normal vector of any boundary point in the target projection viewpoint as the first cosine similarity; take the difference between 1 and the absolute value of the first cosine similarity as the first difference; calculate the first normalized Euclidean distance between the target projection viewpoint and any boundary point; calculate the ratio of the product of the first difference and the first normalized Euclidean distance to the number of boundary points in the target projection viewpoint; iterate through the ratio of each boundary point in all target projection viewpoints to obtain the first sharpness of the target projection viewpoint; iterate through the first sharpness of all combinations of projection viewpoints to obtain the first sum; calculate the product of the first sum and the first difference to obtain the objective function.

6. The label-free 3D point cloud segmentation method based on a large visual model according to claim 1, characterized in that, The method for calculating the sharpness weighting factor is as follows: Calculate the second cosine similarity between the projection direction normal vector and the normal vector of any boundary point in the specified viewpoint. Take the difference between 1 and the absolute value of the second cosine similarity as the second difference. Calculate the second normalized Euclidean distance between the specified viewpoint and any boundary point in the specified viewpoint. Calculate the ratio of the product of the second difference and the second normalized Euclidean distance to the number of boundary points in the specified viewpoint. Iterate through the ratio of each boundary point in the specified viewpoint to obtain the second sharpness of the specified viewpoint. After normalization, obtain the sharpness weight factor.

7. The label-free 3D point cloud segmentation method based on a large visual model according to claim 1, characterized in that, The method for calculating the importance weight is as follows: The ratio of the number of point clouds at a specified projection viewpoint to the total number of point clouds is used as the coverage weighting factor. The importance weight is obtained by multiplying the clarity weight factor and the coverage weight factor.

8. The label-free 3D point cloud segmentation method based on a large visual model according to claim 1, characterized in that, The calculation method for the local fusion two-dimensional features is as follows: the weighted average of the two-dimensional features of the local objects is calculated according to the importance weight to obtain the local fusion two-dimensional features.

9. The label-free 3D point cloud segmentation method based on a large visual model according to claim 3, characterized in that, The method for selecting points whose boundary feature significance is greater than a threshold as marker points is as follows: the threshold is calculated using the OTSU algorithm, and points whose boundary feature significance is greater than the threshold are selected.

Images