Construction method and system of an interactive segmentation network model and an interactive segmentation method

Abstract

The application provides a construction method and system of an interactive segmentation network model and an interactive segmentation method, relates to the technical field of computer vision, and comprises the following steps: using an unsupervised clustering algorithm to divide point cloud instance data of a 3D scene into multiple clusters, and performing simulated click interaction on each cluster as a target foreground region to generate multiple candidate partial masks; screening the candidate partial masks by using a semantic effectiveness discriminator with granularity information to obtain a target partial mask, calculating a corresponding granularity label according to the point number proportion of the target partial mask, and constructing a multi-granularity partial labeling dataset; and based on the multi-granularity partial labeling dataset, training an interactive segmentation network model with frozen backbone pre-training parameters by using a low-rank adaptive strategy to obtain a target interactive segmentation network model supporting any granularity. The target interactive segmentation network model supporting any granularity segmentation can be constructed by using the above method.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, system, and interactive segmentation method for constructing an interactive segmentation network model. Background Technology

[0002] With the development of 3D scanning technology and deep learning, a refined understanding of 3D point cloud scenes has become a research hotspot in the field of computer vision. Interactive segmentation (IS) technology allows users to quickly extract target objects of interest from complex 3D scenes through sparse interactions such as clicking and drawing frames, serving as an important bridge connecting human intent and machine perception.

[0003] However, existing interactive 3D point cloud segmentation techniques still face significant challenges in practical applications, with a severe shortage of fine-grained (part-level) annotation data. Training a model capable of distinguishing different granularities relies on massive amounts of labeled data containing part-to-whole hierarchical relationships. However, most existing mainstream 3D datasets (such as ScanNet, KITTI, and Waymo) only provide instance-level annotations, lacking fine-grained part-level annotations. Due to the complexity of 3D point cloud annotation, large-scale manual part-level annotation is extremely costly, becoming the biggest bottleneck restricting the development of multi-granularity 3D segmentation techniques. Summary of the Invention

[0004] In view of this, this application proposes a method, system, and interactive segmentation method for constructing an interactive segmentation network model.

[0005] Firstly, this application provides a method for constructing an interactive segmentation network model, including: Unsupervised clustering algorithm is used to segment point cloud instance data of 3D scene into multiple clusters, and each cluster is traversed as the target foreground region for simulated click interaction to generate multiple candidate partial masks; The candidate partial masks are filtered by a semantic validity discriminator that integrates granular information. The candidate partial masks with high confidence are retained as the target partial masks. The corresponding granular labels are calculated according to the proportion of points in the target partial mask, and a multi-granularity partial labeling dataset is constructed. Based on the multi-granularity partially labeled dataset, an interactive segmentation network model with frozen main intervention training parameters is trained using a low-rank adaptation strategy to learn to map the input granularity scalar to the corresponding geometric segmentation scale, thereby obtaining a target interactive segmentation network model that supports arbitrary granularity.

[0006] In one embodiment, the semantic validity discriminator includes: a local geometric flow branch, a global context flow branch, a granularity-aware flow branch, and a feature fusion discrimination module; The local geometric flow branch is used to crop the point cloud instance data according to the candidate partial mask to obtain local point cloud blocks, and extract the geometric features of the local point cloud blocks as local geometric features; The global context flow branch is used to concatenate the point cloud instance data with the candidate part mask, and extract the relative position and scale features of the candidate part corresponding to the candidate part mask in the overall object corresponding to the point cloud instance data as global context features. The granularity-aware flow branch is used to obtain the granularity scalar corresponding to the candidate partial mask, and project the granularity scalar into a high-dimensional embedding as a granularity-aware flow feature. The feature fusion and discrimination module is used to concatenate the local geometric features, the global context features, and the granularity-aware flow features, predict the semantic validity probability score of each candidate partial mask through a fully connected layer, and retain candidate masks with probability scores higher than a preset threshold.

[0007] In one embodiment, before filtering each candidate partial mask using a semantic validity discriminator that incorporates granular information, the method further includes: Samples with actual component annotations are used as positive samples; A random KNN cropping strategy is adopted to randomly select the center point on the object corresponding to the point cloud instance and expand the K nearest neighbor points to form geometric fragments without semantic logic as the first type of negative samples. Select the real object component mask, match the object component mask with the wrong granularity scalar, and form the granularity mismatch sample as the second type of negative sample; The semantic validity discriminator is trained under supervision using the positive samples, the first type of negative samples, and the second type of negative samples, so that the semantic validity discriminator has the ability to identify geometric noise and granularity misalignment.

[0008] In one embodiment, the traversal simulates click interactions on each cluster as a target foreground region, generating multiple candidate partial masks, including: Each cluster is traversed as the target foreground region. Points are randomly sampled from the target foreground region as positive clicks, and points are randomly sampled from the object instances corresponding to the point cloud instance data, except for the target foreground region, as negative clicks. The positive click, the negative click, and the point cloud of the object instance are input into a pre-trained instance segmentation model with frozen weights to obtain the mask of each candidate part.

[0009] In one embodiment, the interactive segmentation network model includes: a feature extractor and an interactive decoder; The feature extractor is used to extract deep point cloud features from 3D point cloud data, and the deep point cloud features include semantic features and geometric detail features; The interactive decoder receives input click coordinates and granularity scalars, performs position encoding on the click coordinates to generate geometric query features, encodes the granularity scalars using a multilayer perceptron to generate granular features, fuses the geometric query features and the granular features to generate fused query features, uses the fused query features as the query, and the deep point cloud features as the key and value, calculates attention weights using a multi-head cross-attention mechanism, performs weighted fusion of the deep point cloud features based on the attention weights, and combines the fused features to complete the segmentation inference and mask prediction of the target 3D point cloud region, outputting a 3D point cloud segmentation mask result that meets the user's granularity requirements.

[0010] In one embodiment, the feature extractor includes: a sparse convolutional encoder and a sparse convolutional decoder; The sparse convolutional encoder includes multiple downsampling layers and multiple encoding layers. The sparse convolutional encoder is used to downsample and extract features layer by layer from the input features to form deep encoded features containing multi-scale context information. The encoding layers and the corresponding decoding layers pass features through skip connections. The sparse convolutional decoder includes multiple upsampling layers and multiple decoding layers. The sparse convolutional decoder is used to perform layer-by-layer upsampling and feature reconstruction on the deep encoded features, and fuse the encoded features of the corresponding layers to generate the deep point cloud features.

[0011] In one embodiment, the interactive decoder includes: a granular coding module, a feature fusion module, stacked Transformer decoding layers, and a low-rank adaptation module; The granularity encoding module is used to perform high-dimensional feature encoding on the granularity scalar using a learnable multilayer perceptron to generate the granularity features; The feature fusion module is used to perform dimensional matching between the granular features and the geometric query features, and to perform deep fusion of the two types of features by summing element by element after adjusting with a learnable scaling factor, thereby generating the fused query features. The stacked Transformer decoding layer establishes a precise correlation between user interaction intent and the features of each location in the 3D point cloud through multi-level attention interactions and nonlinear feature transformations, gradually refining the feature representation of the target region. The Transformer decoding layer includes a multi-head cross-attention sublayer and a feedforward network sublayer. The multi-head cross-attention sublayer uses the fused query features as the query and the deep point cloud features as the key and value, calculating attention weights in parallel using multiple attention heads. The feedforward network sublayer performs nonlinear transformations and feature refinement on the features after attention interactions. The low-rank adaptation module includes two sets of trainable low-rank matrices. The low-rank adaptation module injects these matrices in parallel into the projection matrix of the multi-head cross-attention sub-layer. The low-rank adaptation module is used to perform fine residual modulation on the attention projection weights by updating the parameters of the low-rank matrices while freezing the pre-trained projection matrix weights, so as to learn the mapping relationship between the granular scalar and the geometric segmentation scale.

[0012] In one embodiment, training the interactive segmentation network model with frozen main intervention training parameters based on the multi-granularity partially labeled dataset using a low-rank adaptation strategy includes: The samples from the multi-granularity partially labeled dataset are input into an interactive segmentation network model with frozen main intervention training parameters, and a predicted mask probability map is obtained through forward propagation. The predicted mask probability map is compared point by point with the target partial mask, and the binary cross-entropy loss is calculated. The preset trainable parameters in the interactive decoder are updated and iteratively trained using the backpropagation algorithm until the binary cross-entropy loss converges.

[0013] Secondly, this application also provides an interactive 3D point cloud segmentation method, including: Receive user click information and granularity input parameters for the target 3D point cloud; The click information and granularity input parameters are input into the target interactive segmentation network model to obtain the target segmentation mask, wherein the target interactive segmentation network model is constructed using the interactive segmentation network model construction method described in the first aspect.

[0014] Thirdly, this application also provides a system for constructing an interactive segmentation network model, comprising: The simulation module is used to segment point cloud instance data of a 3D scene into multiple clusters using an unsupervised clustering algorithm, and then traverse each cluster as a target foreground region to simulate click interaction, generating multiple candidate partial masks. The filtering module is used to filter each candidate partial mask by a semantic validity discriminator that integrates granular information, retain the candidate partial masks with high confidence as the target partial mask, and calculate the corresponding granular labels according to the proportion of points in the target partial mask to construct a multi-granularity partial labeling dataset. The training module is used to train the interactive segmentation network model with frozen main intervention training parameters based on the multi-granularity partially labeled dataset using a low-rank adaptation strategy, so as to learn to map the input granularity scalar to the corresponding geometric segmentation scale, thereby obtaining a target interactive segmentation network model that supports arbitrary granularity.

[0015] The method for constructing an interactive segmentation network model proposed in this application has the following advantages over related technologies: 1. The interactive segmentation network model construction method of this application divides point cloud clusters through unsupervised clustering and generates candidate partial masks by simulating clicks. Then, the effective masks are filtered by a semantic validity discriminator that integrates granular information and granular labels are calculated. This method can automatically complete the construction of multi-granularity partial annotation datasets without the need for manual high-cost component-level annotation. It fundamentally solves the problem of the scarcity of fine-grained annotation data and significantly reduces the cost of training data acquisition for multi-granularity 3D segmentation models.

[0016] 2. Based on the automatically generated high-quality multi-granularity partially labeled dataset, a low-rank adaptation strategy is used to train the interactive segmentation network model with frozen main intervention training parameters. This enables the model to accurately learn the mapping relationship between granularity scalars and geometric segmentation scales, ultimately obtaining a target interactive segmentation network model that supports segmentation at arbitrary granularity. This achieves efficient parameter fine-tuning, retains the general segmentation capability of the pre-trained model, and eliminates the scale ambiguity of 3D interactive segmentation, effectively breaking through the development limitations of multi-granularity 3D segmentation technology. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the method for constructing an interactive segmentation network model in one embodiment of this application; Figure 2 This is a schematic diagram illustrating the construction process and granular segmentation process of an interactive segmentation network model in one embodiment of this application; Figure 3 This is a schematic diagram of the network architecture of a semantic validity discriminator in one embodiment of this application; Figure 4 This is a partial structural diagram of an interactive decoder in one embodiment of this application; Figure 5 This is a schematic diagram of the structure of an interactive segmentation network model in one embodiment of this application; Figure 6 This is a flowchart illustrating an embodiment of the interactive segmentation method for 3D point clouds in this application. Figure 7 This is a schematic diagram of the structure of an interactive segmentation network model construction system in one embodiment of this application. Detailed Implementation

[0019] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] As mentioned in the background section, with the development of 3D scanning technology and deep learning, a refined understanding of 3D point cloud scenes has become a research hotspot in the field of computer vision. Interactive segmentation (IS) technology allows users to quickly extract target objects of interest from complex 3D scenes through sparse interactions such as clicking and drawing frames, serving as an important bridge connecting human intent and machine perception.

[0021] However, existing interactive 3D point cloud segmentation technologies still face significant challenges in practical applications, mainly in the following two aspects: First, fine-grained (part-level) labeled data is extremely scarce. Training a model capable of distinguishing different granularities relies on massive amounts of labeled data containing part-to-whole hierarchical relationships. However, most existing mainstream 3D datasets (such as ScanNet, KITTI, and Waymo) only provide instance-level annotations, lacking fine-grained part-level annotations. Due to the complexity of 3D point cloud annotation, large-scale manual part-level annotation is extremely costly, which has become the biggest bottleneck restricting the development of multi-granularity 3D segmentation technology.

[0022] Second, scale ambiguity is a serious problem. Objects in 3D scenes often have complex hierarchical nested structures. For example, a "car" contains "doors," and the doors contain "door handles." Existing interactive 3D segmentation methods (such as InterObject3D and AGILE3D) typically only output results at a single granularity (usually complete object instances). When a user clicks on the "door handle," the model cannot determine whether the user wants to segment the "handle," the "door," or the "entire car." This uncertainty in intent often results in segmentation results that do not meet user expectations, requiring repeated interactive corrections, which is inefficient. In contrast, models such as Segment Anything (SAM) in the 2D image domain have attempted to solve the multi-granularity problem, but due to the sparsity, occlusion, and density variations of 3D point clouds, directly transferring 2D methods is extremely difficult.

[0023] Based on this, in some embodiments, such as Figure 1 and Figure 2 As shown, the present application provides a method for constructing an interactive segmentation network model, which includes the following steps S101 to S103.

[0024] S101: Use an unsupervised clustering algorithm to segment the point cloud instance data of the 3D scene into multiple clusters, and traverse each cluster as the target foreground region to simulate click interaction and generate multiple candidate partial masks.

[0025] Step S101 involves constructing an automated scene-level partial annotation data generation pipeline. First, an unsupervised clustering algorithm is used to automatically partition the point cloud instance data in the 3D scene, splitting the complete object point cloud instance into multiple independent local point cloud clusters, thus completing the initial division of the object's local regions. Then, each generated point cloud cluster is sequentially traversed, and it is designated as the target foreground region. Positive clicks are selected within the target foreground region and negative clicks are selected in non-target regions through simulated human interaction. The click information from the simulated interaction, along with the point cloud instance data, is input into a pre-trained segmentation model, ultimately automatically generating multiple candidate partial masks corresponding to different local regions.

[0026] S102: The candidate partial masks are filtered by a semantic validity discriminator that integrates granular information. The candidate partial masks with high confidence are retained as the target partial masks. The corresponding granular labels are calculated according to the proportion of points in the target partial mask, and a multi-granularity partial labeling dataset is constructed.

[0027] For each target portion mask that passes the screening Calculate its corresponding granularity label : Finally, constructing a system containing A multi-granularity partially labeled dataset.

[0028] It is understandable that by using a semantic validity discriminator that integrates granular information, all candidate partial masks generated in the early stages can be comprehensively and accurately judged and filtered for semantic validity. This discriminator can combine local geometric features, global contextual features, and granular features to identify and remove invalid noise masks that are geometrically fragmented, semantically incoherent, or granularly mismatched, retaining only high-confidence candidate partial masks with reasonable semantics and geometric integrity as target partial masks. Then, based on the proportion of the number of point clouds in the target partial mask to the total number of points in the corresponding 3D point cloud instance, a unique granular label for each target partial mask is automatically calculated. The filtered target partial masks, matched simulated click data, and corresponding granular labels are then integrated to automatically build a multi-granularity partial annotation dataset containing multi-scale component annotation information, completing the construction of high-quality fine-grained annotation data without manual intervention.

[0029] S103: Based on a multi-granularity partially labeled dataset, an interactive segmentation network model with frozen main intervention training parameters is trained using a low-rank adaptation strategy to learn to map the input granularity scalar to the corresponding geometric segmentation scale, thereby obtaining a target interactive segmentation network model that supports arbitrary granularity.

[0030] It is understandable that, using the already constructed multi-granularity partially labeled dataset as the training basis, a low-rank adaptive parameter fine-tuning strategy is adopted. This involves keeping the main intervention training parameters of the interactive segmentation network model frozen and preserving the model's original basic segmentation capabilities while conducting targeted training optimization. During training, the model is driven to learn autonomously and establish a precise correspondence between the input granularity scalar and the corresponding geometric segmentation scale. Lightweight parameter updates are used to adapt the model's granularity-awareness capability, ultimately resulting in a target model that can dynamically adjust the segmentation range based on the granularity scalar and supports interactive segmentation of 3D point clouds at any granularity.

[0031] This application presents a method for constructing an interactive segmentation network model. This method uses unsupervised clustering to divide point cloud clusters and simulates clicks to generate candidate partial masks. Then, a semantic validity discriminator that integrates granular information filters valid masks and calculates granular labels. This automated approach constructs a multi-granularity partial annotation dataset without requiring costly manual component-level annotation, fundamentally solving the problem of scarce fine-grained annotation data and significantly reducing the cost of acquiring training data for multi-granularity 3D segmentation models. Based on the automatically generated high-quality multi-granularity partial annotation dataset, a low-rank adaptation strategy is used to train the interactive segmentation network model with frozen main intervention training parameters. This allows the model to accurately learn the mapping relationship between granularity scalars and geometric segmentation scales, ultimately resulting in a target interactive segmentation network model that supports arbitrary granularity segmentation. This achieves efficient parameter fine-tuning, retains the general segmentation capabilities of the pre-trained model, and eliminates the scale ambiguity of 3D interactive segmentation, effectively overcoming the development limitations of multi-granularity 3D segmentation technology.

[0032] In some embodiments, such as Figure 3 As shown, the semantic validity discriminator includes: a local geometric flow branch, a global context flow branch, a granularity-aware flow branch, and a feature fusion discrimination module.

[0033] The local geometry flow branch is used to crop point cloud instance data based on candidate partial masks to obtain local point cloud patches and extract their geometric features as local geometric features. The global context flow branch is used to concatenate the point cloud instance data and candidate partial masks through channels, extracting the relative position and scale features of the candidate parts corresponding to the candidate partial masks within the overall object corresponding to the point cloud instance data, as global context features. The granularity-aware flow branch is used to obtain the granularity scalar corresponding to the candidate partial masks, project the granularity scalar into a high-dimensional embedding, and use it as granularity-aware flow features. The feature fusion and discrimination module is used to concatenate the local geometric features, global context features, and granularity-aware flow features, predict the semantic validity probability score of each candidate partial mask through a fully connected layer, and retain candidate masks with probability scores higher than a preset threshold.

[0034] Since the clustering process is unsupervised, the generated candidate masks may contain geometric noise or semantically incoherent fragments. This embodiment constructs and trains a dedicated Semantic Validity Selector (SVS) for filtering.

[0035] In the application, SVS adopts a three-stream network architecture based on PointNet, with the local geometric stream (Candidate Stream) branch based on the candidate mask. For objects Crop the data to obtain local point cloud patches. Local geometric features are extracted using PointNet. The global context stream branch will display the complete object. Point cloud and mask Perform channel splicing (input dimension is) The relative position and scale features of this part within the whole are extracted using PointNet. The granularity stream branch calculates the percentage of candidate mask points. This is then fed into a learnable granularity estimator (MLP structure) to obtain granular embedding features. .

[0036] The input to the granularity-aware flow branch is the candidate mask. Normalized granularity percentage The calculation formula is: ,in, The number of points within the candidate mask. The total number of points for object instances. .

[0037] Specifically, scalars can be... The input is processed by a learnable granularity estimator. This granularity estimator is composed of a multilayer perceptron (MLP). The granularity estimator first expands the scalar into a vector, maps it to an intermediate dimension through a first linear layer, passes it through a ReLU activation function, and then maps it to the target dimension through a second linear layer. The corresponding formula is as follows:

[0038] High-dimensional features of granularity-aware flow branch output It carries prior information about "how big this component should be." For example, when It displays the characteristics of a "slender cylinder (like a chair leg)," while When the feature "0.9% (like the main body)" is displayed, the network can detect the feature conflict between the two and thus determine that the candidate mask is invalid (corresponding to Type 2 negative sample).

[0039] Finally, the three features are concatenated by the feature fusion and discrimination module, and the semantic validity probability of the candidate mask is output by the multilayer perceptron (MLP) classification head. : Set confidence threshold (e.g., 0.7), only retain The candidate mask is used as the target partial mask. The solution adopted in this embodiment can combine local geometric features, global context features, and granular features to identify and eliminate invalid noise masks in the candidate masks that are geometrically fragmented, semantically incoherent, or granularly mismatched, and retain only the semantically reasonable and geometrically complete high-confidence candidate partial masks as the target partial masks.

[0040] In some embodiments, before the semantic validity discriminator, which incorporates granular information, filters each candidate part mask, the method for constructing the interactive segmentation network model further includes: using samples with real part annotations as positive samples; employing a random KNN pruning strategy to randomly select the center point on the object corresponding to the point cloud instance and expand it to K nearest neighbors, forming geometric fragments without semantic logic as first-type negative samples; selecting real object part masks and matching the object part masks with incorrect granularity scalars, forming granularity mismatch samples as second-type negative samples; and using positive samples, first-type negative samples, and second-type negative samples to supervise the training of the semantic validity discriminator, so that the semantic validity discriminator has the ability to identify geometric noise and granularity misalignment.

[0041] In applications, to enable SVS to discriminate, a binary classification dataset is constructed for training. Positive samples can be derived from real ground-truth part annotations in datasets such as ShapeNet-Part. Negative samples can be synthesized using two strategies: Type 1 (Semantic Inconsistency) employs Random KNN Cropping, randomly selecting a center point on the object and expanding it with K nearest neighbors to form geometrically continuous but semantically unrelated fragments; Type 2 (Granularity Mismatch) selects a real and valid part mask but forces it to be assigned an incorrect granular scalar input (e.g., assigning a "0.1" granularity ratio to the mask of "the whole chair"), forcing SVS to learn the correspondence between geometry and scale.

[0042] In this embodiment, real component annotation samples are used as positive samples to provide a standard learning basis for the semantic validity discriminator to obtain valid component masks. A random KNN pruning strategy is employed, randomly selecting the center point on the object corresponding to the point cloud instance and expanding outwards to K nearest neighbor points to generate geometric fragments without reasonable semantic logic. These fragments are used as the first type of negative samples to simulate geometric noise-type invalid masks. Real object component masks are selected and matched with incorrect granularity scalars to construct a second type of negative sample with mismatched granularity, used to simulate granularity misalignment-type invalid masks. Subsequently, the positive samples, the first type of negative samples, and the second type of negative samples are used together as supervisory data to perform complete supervised training on the semantic validity discriminator. This allows the discriminator to fully learn the feature differences between valid and invalid masks, ultimately possessing the core ability to accurately identify both geometric noise and granularity misalignment-type invalid masks.

[0043] In some embodiments, traversing each cluster as a target foreground region and simulating click interactions to generate multiple candidate partial masks includes: traversing each cluster as a target foreground region, randomly sampling points from the target foreground region as positive clicks, and randomly sampling points from the region other than the target foreground region in the object instance corresponding to the point cloud instance data as negative clicks; and inputting the point clouds of positive clicks, negative clicks, and object instances into a pre-trained instance segmentation model with frozen weights to obtain each candidate partial mask.

[0044] In the application, a 3D scene point cloud is first given. (in For points, (for feature dimensions), and an object instance in the scene. Then, the K-Means clustering algorithm is used to group the object instances. Divided into Different clusters To increase data diversity, The value is randomly selected within a certain range (e.g., 2 to 10).

[0045] Next, traverse each cluster. This serves as the target foreground area, simulating user click interactions. (Positive Clicks) From the current target cluster Random sampling points. Negative Clicks. From the object Except Union of other clusters Random sampling points in the middle.

[0046] Then, point clouds of objects. and simulated click collection Input into a pre-trained interactive instance segmentation model with frozen weights (For example, in AGILE3D). The model predicts the binary mask according to the following formula. :

[0047] The mask This is the initial "candidate partial mask".

[0048] In some embodiments, such as Figure 4 As shown, the interactive segmentation network model includes a feature extractor and an interactive decoder.

[0049] The feature extractor is used to extract deep point cloud features from 3D point cloud data. Deep point cloud features include semantic features and geometric detail features.

[0050] The feature extractor employs a 4-stage Sparse ResNet implemented using MinkowskiEngine or SpConv. It includes an Encoder Path and a Decoder Path, preserving geometric details through skip connections. The input can be... The coordinates and color information of each point can be output as follows: Feature vector of points (in (This refers to the number of feature channels, such as 32 or 64). It should be noted that the parameters of the feature extractor are completely frozen during the training phase of the interactive segmentation network model.

[0051] The interactive decoder receives the input click coordinates and granularity scalars, performs position encoding on the click coordinates to generate geometric query features, encodes the granularity scalars using a multilayer perceptron to generate granular features, fuses the geometric query features and granular features to generate fused query features, uses the fused query features as the query and the deep point cloud features as the key and value, calculates attention weights through a multi-head cross-attention mechanism, performs weighted fusion of the deep point cloud features based on the attention weights, and combines the fused features to complete the segmentation inference and mask prediction of the target 3D point cloud region, outputting a 3D point cloud segmentation mask result that meets the user's granularity requirements.

[0052] It is understandable that the interactive decoder receives two types of interactive information from the user: click coordinates and granular scalars. First, it encodes the click coordinates to generate geometric query features. Then, it encodes the granular scalars using a multilayer perceptron to obtain granular features. The geometric query features and granular features are then fused to generate fused query features. Next, using the fused query features as the query and the deep point cloud features as the key and value, a multi-head cross-attention mechanism is used to calculate the corresponding attention weights. Based on these weights, a weighted fusion operation is performed on the deep point cloud features. Finally, the weighted fused features are combined to complete the segmentation inference and mask prediction of the target 3D point cloud region, directly outputting a 3D point cloud segmentation mask result that meets the granularity requirements specified by the user.

[0053] In some embodiments, such as Figure 4 As shown, the feature extractor includes a sparse convolutional encoder and a sparse convolutional decoder.

[0054] A sparse convolutional encoder consists of multiple downsampling layers and multiple encoding layers. The sparse convolutional encoder is used to downsample and extract features layer by layer from the input features to form deep encoded features containing multi-scale contextual information. The features are transferred between each encoding layer and the corresponding decoding layer through skip connections.

[0055] The sparse convolutional decoder consists of multiple upsampling layers and multiple decoding layers. The sparse convolutional decoder is used to perform layer-by-layer upsampling and feature reconstruction of deep encoded features, and fuse the encoded features of the corresponding layers to generate deep point cloud features.

[0056] In this embodiment, the feature extractor adopts the classic symmetric architecture of Sparse Convolutional U-Net, comprising a sparse convolutional encoder and a sparse convolutional decoder. The sparse convolutional encoder consists of multiple downsampling layers and multiple encoding layers sequentially connected. Its core function is to perform layer-by-layer downsampling operations and depth feature extraction on the original features of the input 3D point cloud. Through hierarchical feature transformation, it gradually extracts deep encoded features containing multi-scale contextual information. To avoid the loss of geometric details during deep feature extraction, each encoding layer and the corresponding decoding layer in the sparse convolutional decoder achieves direct feature transfer through skip connections, ensuring that low-level detailed features can be passed upwards. The sparse convolutional decoder consists of multiple upsampling layers and multiple decoding layers. Its core function is to receive the deep encoded features output by the sparse convolutional encoder, complete the feature scale recovery and feature reconstruction through layer-by-layer upsampling, and fuse the corresponding level encoded features passed through skip connections during the reconstruction process. This fully integrates multi-scale contextual information with low-level geometric details, and finally generates deep point cloud features that have both semantic and geometric details, providing high-quality basic feature support for the segmentation and inference of the subsequent interactive decoder.

[0057] In some embodiments, such as Figure 4 and Figure 5 As shown, the interactive decoder includes: a granular encoding module, a feature fusion module, stacked Transformer decoding layers, and a low-rank adaptation module.

[0058] The granularity coding module is used to encode high-dimensional features of granular scalars using a learnable multilayer perceptron to generate granular features.

[0059] The granularity encoding module receives normalized granularity scalars input by the user (or from the training set). To enable the network to adapt to nonlinear distributions at the object scale, a learnable multilayer perceptron (MLP) is used. Mapped to 3D eigenvectors The calculation formula is as follows:

[0060] in, and These are the weights and biases of the first-level linear transformation; It is the ReLU activation function, used to introduce nonlinearity; and These are the weights and biases of the second-level linear transformation; Consistent with the channel dimension of the query vector in the interactive decoder.

[0061] The feature fusion module is used to perform dimensional matching between granular features and geometric query features, and after adjusting with a learnable scaling factor, it performs deep fusion of the two types of features by summing element by element to generate fused query features.

[0062] Encoded granular features Injected into the geometric query vector generated by user clicks In this embodiment, an element-wise summation strategy is employed, treating granular features as a conditional bias. The core objective of feature fusion in this embodiment is to construct a composite query vector that includes both "location intent (Where)" and "scale intent (Howbig)". Specifically, this involves element-wise summation of the following two feature vectors. The first addend is the geometric query vector generated by the user's click. One or more 3D coordinate points can be generated from user clicks in a 3D scene. To enable the Transformer network to understand these coordinates, sinusoidal positional encoding or Fourier feature mapping can be used to convert low-dimensional physical coordinates into their corresponding coordinates. The mapping is to a high-dimensional feature vector. The mapped vector is denoted as... Its dimensions are .in, This indicates the number of queries (usually corresponding to the number of clicks or the preset number of query points). This represents the hidden layer channel dimension (e.g., ...) of the Transformer decoder. ). In the feature space, it represents the user's geometric location intent, that is, telling the model "the center of the object I am interested in is here".

[0063] The second addend is the encoded granular feature. , by particle size scalar The features are obtained through the aforementioned multilayer perceptron (MLP) nonlinear mapping. The dimensions are strictly designed as Here Must with The channel dimensions are completely consistent to meet the mathematical requirements of element-wise operations. The feature space represents the user's scale semantic intent, which tells the model "the range of objects I am interested in is approximately this proportion".

[0064] because It contains location information, and This is globally shared scale information. This embodiment uses a broadcasting addition strategy, that is, the dimension is... Granularity characteristics Along the number of queries Copy or broadcast the dimension to expand it to The matrix, then with Perform element-wise addition: in, Indicates the first One query; Indicates the first One feature channel; This is a learnable scaling factor (initially 0).

[0065] In related technologies, traditional interactive decoders (such as AGILE3D) only contain a click location encoding module, and their output is directly used as the query vector. Input attention layer.

[0066] The interactive decoder in this embodiment constructs a dual-source query fusion structure. Compared to related technologies, it adds a parallel granular coding branch and a feature fusion node during the query generation stage, namely, a granular coding module and a feature fusion module. The granular coding module incorporates a multilayer perceptron (MLP), receiving granular scalar data sources at its input and connecting to the fusion node at its output. The feature fusion module incorporates an element-wise adder as the fusion node. Its first input receives geometric query features input by the user, and its second input connects to the output of the granular coding module. This "Y-shaped" convergence structure ensures that the query vector entering the attention layer is no longer merely a position index, but carries a dual constraint of "position + scale".

[0067] The stacked Transformer decoding layer establishes a precise correlation between user interaction intent and the features of various locations in the 3D point cloud through multi-level attention interactions and non-linear feature transformations, gradually refining the feature representation of the target region. The Transformer decoding layer includes a multi-head cross-attention sublayer and a feedforward network sublayer. The multi-head cross-attention sublayer is used to calculate attention weights in parallel with multiple attention heads, using fused query features as queries and deep point cloud features as keys and values. The feedforward network sublayer is used to perform non-linear transformations and feature refinement on the features after attention interactions.

[0068] It's understandable that by processing user interaction information through stacked Transformer decoding layers, an attention mechanism can be used to map click queries onto scene features. In the cross-attention layer, the query vectors are fused. As a query, scenario characteristics As Key and Value.

[0069] The low-rank adaptation module includes two sets of trainable low-rank matrices. The low-rank adaptation module is injected in parallel into the projection matrix of the multi-head cross attention sub-layer. The low-rank adaptation module is used to perform fine residual modulation on the attention projection weights by updating the parameters of the low-rank matrix while freezing the weights of the pre-trained projection matrix, so as to learn the mapping relationship between the granular scalar and the geometric segmentation scale.

[0070] In related technologies, in the traditional Transformer attention module, the generation of query, key, and value are all achieved through a single dense linear layer, whose weight matrix... As a whole.

[0071] This application structurally decomposes the linear projection layer. The original fully connected layer is retained as the main pathway, but its parameter state is locked as "non-trainable". A low-rank adaptation module is connected in parallel on both sides of the main pathway as a low-rank bypass. This bypass contains two cascaded small linear layers (matrix...). sum matrix ), the dimensions of the intermediate layer Much smaller than the input-output dimension (Right now The outputs of the main path and the bypass path are added together before entering the next layer. This parallel bypass structure physically isolates "general object segmentation knowledge" (stored in the main path) and "granular control knowledge" (stored in the bypass path), which not only significantly reduces the number of training parameters, but also structurally ensures that the model will not lose its original feature extraction ability due to fine-tuning.

[0072] To efficiently adapt to new tasks on limited generated data while preserving the original segmentation capabilities, this embodiment employs a low-rank adaptation (LoRA) strategy, freezing most parameters in the backbone network (such as Sparse U-Net) and Transformer decoder of the pre-trained segmentation model. Trainable parameters are injected only into the linear projection layers (specifically the Query and Value projection matrices) of the Attention Block in the Decoder.

[0073] Assume the pre-trained weight matrix is Its update is parameterized into a low-rank decomposition form: in, Specifically, this refers to the linear projection weight matrix in the cross-attention or self-attention layers of the Transformer decoder mentioned above. More specifically, in the standard attention calculation formula... In this context, the input vector needs to undergo a linear transformation first. For example, the projection of the query is calculated as follows: , here This is a typical example (Pre-trained weights); similarly, the projection of Value is calculated as follows: , here It can also be a target And here in the pre-trained (Right now Next to it, connect a bypass in parallel. Then, during forward propagation, the calculation formula becomes: It should be noted that, It can be a pre-trained, unmodifiable parameter matrix downloaded from models such as AGILE3D (Frozen). These are the newly initialized, trainable low-rank matrices used in this invention to adapt to multi-granularity tasks. , , It is rank and satisfies (For example or During training, the value of W0 is kept constant, and the output of W0 is corrected using low-rank decomposition matrices A and B, thereby injecting granular control capability without destroying the object segmentation capability of the original model.

[0074] In some embodiments, based on a multi-granularity partially labeled dataset, an interactive segmentation network model with frozen main intervention training parameters is trained using a low-rank adaptation strategy. This includes: inputting samples from the multi-granularity partially labeled dataset into the interactive segmentation network model with frozen main intervention training parameters, and obtaining a predicted mask probability map through forward propagation; comparing the predicted mask probability map with the target partial mask point by point, and calculating the binary cross-entropy loss; and updating and iteratively training the preset trainable parameters in the interactive decoder using a backpropagation algorithm until the binary cross-entropy loss converges.

[0075] The parameters kept frozen can include the sparse convolutional encoder parameters in the pre-trained segmentation model and the original parameters in the interactive decoder (including...). And the position encoding module parameters. Updated parameters include the low-rank adaptation matrix. and (Located in the interactive decoder), weights of the multilayer perceptron (MLP) and feature fusion scaling factor .

[0076] In the application, during the training of the model using a multi-granularity partially labeled dataset, the input data includes object point clouds ( Simulated clicks () ) and granularity labels ( ).

[0077] Object point cloud ( ): Dimensions are The point cloud data is input into the sparse convolutional encoder of the pre-trained segmentation model to extract scene features. .

[0078] Simulated clicks ( ): A set of three-dimensional coordinates containing user (or simulation algorithm) clicks. .

[0079] Granularity label ( ): The normalized scalar obtained from the calculation is used to supervise granularity control.

[0080] Forward propagation consists of four steps. The first step is the generation of the click query vector (ClickEncoding), which simulates the coordinates of the click. The input is fed into the Positional Encoding Module of the pre-trained segmentation model. The second step is the generation of granular features, which involves assigning granular labels... The input is fed into the newly added multilayer perceptron (MLP) of this invention to obtain granular embedded features. The third step is feature fusion, which uses element-wise summation under a broadcast mechanism to... injection , to obtain the fused query vector The fourth step is decoding and prediction. The input is fed into an interactive decoder infused with LoRA modules, which utilizes a cross-attention mechanism to... For querying, based on scene characteristics For keys and values; when calculating attention weights, the original pre-trained weight matrix... The low-rank matrix injected by this invention synergistic effect (i.e.) The decoder ultimately outputs a predicted mask probability map for the object. .

[0081] It should be noted that the formula for calculating the Binary Cross Entropy Loss (BCE) is as follows: Among them, the target candidate mask serves as the supervised ground truth. .

[0082] In some embodiments, such as Figure 6 As shown, this application also provides a 3D point cloud interactive segmentation method, including the following steps S601 and S602.

[0083] S601: Receives user click information and granularity input parameters for the target 3D point cloud.

[0084] S602: Input the click information and granularity input parameters into the target interactive segmentation network model to obtain the target segmentation mask. The target interactive segmentation network model is constructed using the interactive segmentation network model construction method of any of the above schemes.

[0085] In the application, users can load 3D scenes into a visual interface. For example, if a user wants to segment the "front left door" of a car, they can click once in the front left door area. Users can specify a granularity value using a slider or a numeric input box on the interface. If the user sets (Fine-grained), the model interprets this as segmenting "door handles"; if the user sets... (Medium granularity), the model interprets this as segmenting "car doors"; if the user sets... (Coarse-grained) The model interprets this as segmenting the "whole vehicle". The model receives clicks and... The value, after being processed by MLP encoding and LoRA modulation network inference, directly outputs a segmentation mask conforming to the user-specified scale. Users can drag the slider in real time to observe the segmentation boundaries as they change. The value is continuously changed until a satisfactory result is obtained.

[0086] Based on the same inventive concept, such as Figure 7 As shown, this application also provides an interactive segmentation network model construction system 70, including: a simulation module 71, a screening module 72, and a training module 73.

[0087] The simulation module 71 is used to segment the point cloud instance data of the 3D scene into multiple clusters using an unsupervised clustering algorithm, and traverse each cluster as the target foreground region to simulate click interaction and generate multiple candidate partial masks. The filtering module 72 is used to filter each candidate partial mask by a semantic validity discriminator that integrates granular information, retain the candidate partial masks with high confidence as the target partial mask, and calculate the corresponding granular label according to the proportion of points in the target partial mask to construct a multi-granularity partial labeling dataset. Training module 73 is used to train an interactive segmentation network model with frozen main intervention training parameters based on a multi-granularity partially labeled dataset and employs a low-rank adaptation strategy to learn to map the input granularity scalar to the corresponding geometric segmentation scale, thereby obtaining a target interactive segmentation network model that supports arbitrary granularity.

[0088] It should be noted that the interactive segmentation network model construction system 70 provided in this application embodiment and the interactive segmentation network model construction method provided in this application embodiment are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned interactive segmentation network model construction method, and the repeated parts will not be described again.

[0089] In some embodiments, an electronic device provided in this application includes a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the above-described method for constructing an interactive segmentation network model.

[0090] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0091] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0092] This application also provides a non-transitory computer storage medium storing a computer program thereon, which, when executed by a processor, implements the method for constructing the interactive segmentation network model described above. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments of this application.

[0093] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0094] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0095] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for constructing an interactive segmentation network model, characterized in that, include: Unsupervised clustering algorithm is used to segment point cloud instance data of 3D scene into multiple clusters, and each cluster is traversed as the target foreground region for simulated click interaction to generate multiple candidate partial masks; The candidate partial masks are filtered by a semantic validity discriminator that integrates granular information. The candidate partial masks with high confidence are retained as the target partial masks. The corresponding granular labels are calculated according to the proportion of points in the target partial mask, and a multi-granularity partial labeling dataset is constructed. Based on the multi-granularity partially labeled dataset, an interactive segmentation network model with frozen main intervention training parameters is trained using a low-rank adaptation strategy to learn to map the input granularity scalar to the corresponding geometric segmentation scale, thereby obtaining a target interactive segmentation network model that supports arbitrary granularity.

2. The method for constructing the interactive segmentation network model as described in claim 1, characterized in that, The semantic validity discriminator includes: a local geometric flow branch, a global context flow branch, a granularity-aware flow branch, and a feature fusion discrimination module; The local geometric flow branch is used to crop the point cloud instance data according to the candidate partial mask to obtain local point cloud blocks, and extract the geometric features of the local point cloud blocks as local geometric features; The global context flow branch is used to concatenate the point cloud instance data with the candidate part mask, and extract the relative position and scale features of the candidate part corresponding to the candidate part mask in the overall object corresponding to the point cloud instance data as global context features. The granularity-aware flow branch is used to obtain the granularity scalar corresponding to the candidate partial mask, and project the granularity scalar into a high-dimensional embedding as a granularity-aware flow feature. The feature fusion and discrimination module is used to concatenate the local geometric features, the global context features, and the granularity-aware flow features, predict the semantic validity probability score of each candidate partial mask through a fully connected layer, and retain candidate masks with probability scores higher than a preset threshold.

3. The method for constructing the interactive segmentation network model as described in claim 1, characterized in that, Before filtering each candidate partial mask using a semantic validity discriminator that integrates granular information, the method further includes: Samples with actual component annotations are used as positive samples; A random KNN cropping strategy is adopted to randomly select the center point on the object corresponding to the point cloud instance and expand the K nearest neighbor points to form geometric fragments without semantic logic as the first type of negative samples. Select the real object component mask, match the object component mask with the wrong granularity scalar, and form the granularity mismatch sample as the second type of negative sample; The semantic validity discriminator is trained under supervision using the positive samples, the first type of negative samples, and the second type of negative samples, so that the semantic validity discriminator has the ability to identify geometric noise and granularity misalignment.

4. The method for constructing the interactive segmentation network model as described in claim 1, characterized in that, The traversal treats each cluster as a target foreground region and simulates click interactions to generate multiple candidate partial masks, including: Each cluster is traversed as the target foreground region. Points are randomly sampled from the target foreground region as positive clicks, and points are randomly sampled from the object instances corresponding to the point cloud instance data, except for the target foreground region, as negative clicks. The positive click, the negative click, and the point cloud of the object instance are input into a pre-trained instance segmentation model with frozen weights to obtain the mask of each candidate part.

5. The method for constructing the interactive segmentation network model as described in claim 1, characterized in that, The interactive segmentation network model includes: a feature extractor and an interactive decoder; The feature extractor is used to extract deep point cloud features from 3D point cloud data, and the deep point cloud features include semantic features and geometric detail features; The interactive decoder receives input click coordinates and granularity scalars, performs position encoding on the click coordinates to generate geometric query features, encodes the granularity scalars using a multilayer perceptron to generate granular features, fuses the geometric query features and the granular features to generate fused query features, uses the fused query features as the query, and the deep point cloud features as the key and value, calculates attention weights using a multi-head cross-attention mechanism, performs weighted fusion of the deep point cloud features based on the attention weights, and combines the fused features to complete the segmentation inference and mask prediction of the target 3D point cloud region, outputting a 3D point cloud segmentation mask result that meets the user's granularity requirements.

6. The method for constructing the interactive segmentation network model as described in claim 5, characterized in that, The feature extractor includes: a sparse convolutional encoder and a sparse convolutional decoder; The sparse convolutional encoder includes multiple downsampling layers and multiple encoding layers. The sparse convolutional encoder is used to perform layer-by-layer downsampling and feature extraction on the input features to form deep encoded features containing multi-scale contextual information. The encoding layers and the corresponding decoding layers pass features through skip connections. The sparse convolutional decoder includes multiple upsampling layers and multiple decoding layers. The sparse convolutional decoder is used to perform layer-by-layer upsampling and feature reconstruction on the deep encoded features, and fuse the encoded features of the corresponding layers to generate the deep point cloud features.

7. The method for constructing the interactive segmentation network model as described in claim 5, characterized in that, The interactive decoder includes: a granular encoding module, a feature fusion module, stacked Transformer decoding layers, and a low-rank adaptation module; The granularity encoding module is used to perform high-dimensional feature encoding on the granularity scalar using a learnable multilayer perceptron to generate the granularity features; The feature fusion module is used to perform dimensional matching between the granular features and the geometric query features, and to perform deep fusion of the two types of features by summing element by element after adjusting with a learnable scaling factor, thereby generating the fused query features. The stacked Transformer decoding layer establishes a precise correlation between user interaction intent and the features of each location in the 3D point cloud through multi-level attention interactions and nonlinear feature transformations, gradually refining the feature representation of the target region. The Transformer decoding layer includes a multi-head cross-attention sublayer and a feedforward network sublayer. The multi-head cross-attention sublayer uses the fused query features as the query and the deep point cloud features as the key and value, calculating attention weights in parallel using multiple attention heads. The feedforward network sublayer performs nonlinear transformations and feature refinement on the features after attention interactions. The low-rank adaptation module includes two sets of trainable low-rank matrices. The low-rank adaptation module injects these matrices in parallel into the projection matrix of the multi-head cross-attention sub-layer. The low-rank adaptation module is used to perform fine residual modulation on the attention projection weights by updating the parameters of the low-rank matrices while freezing the pre-trained projection matrix weights, so as to learn the mapping relationship between the granular scalar and the geometric segmentation scale.

8. The method for constructing the interactive segmentation network model as described in claim 7, characterized in that, The step of training an interactive segmentation network model with frozen main intervention training parameters based on the multi-granularity partially labeled dataset using a low-rank adaptation strategy includes: The samples from the multi-granularity partially labeled dataset are input into an interactive segmentation network model with frozen main intervention training parameters, and a predicted mask probability map is obtained through forward propagation. The predicted mask probability map is compared point by point with the target partial mask, and the binary cross-entropy loss is calculated. The preset trainable parameters in the interactive decoder are updated and iteratively trained using the backpropagation algorithm until the binary cross-entropy loss converges.

9. A 3D point cloud interactive segmentation method, characterized in that, include: Receive user click information and granularity input parameters for the target 3D point cloud; The click information and granularity input parameters are input into the target interactive segmentation network model to obtain the target segmentation mask, wherein the target interactive segmentation network model is constructed using the construction method of the interactive segmentation network model as described in any one of claims 1 to 8.

10. A system for constructing an interactive segmentation network model, characterized in that, include: The simulation module is used to segment point cloud instance data of a 3D scene into multiple clusters using an unsupervised clustering algorithm, and then traverse each cluster as a target foreground region to simulate click interaction, generating multiple candidate partial masks. The filtering module is used to filter each candidate partial mask by a semantic validity discriminator that integrates granular information, retain the candidate partial masks with high confidence as the target partial mask, and calculate the corresponding granular labels according to the proportion of points in the target partial mask to construct a multi-granularity partial labeling dataset. The training module is used to train the interactive segmentation network model with frozen main intervention training parameters based on the multi-granularity partially labeled dataset using a low-rank adaptation strategy, so as to learn to map the input granularity scalar to the corresponding geometric segmentation scale, thereby obtaining a target interactive segmentation network model that supports arbitrary granularity.

Images