An open scene instance segmentation method based on decoupled distillation

Abstract

The application provides an open scene instance segmentation method based on decoupling distillation, generates decoupling features in the spatial domain and the frequency domain through multi-scale feature decomposition and two-stage wavelet decomposition, calculates semantic loss of a teacher model and a student model for distillation training, and focuses on aligning fine-grained features in the frequency domain; in order to expand the basic model to a three-dimensional scene, three-dimensional key points are selected and projected to two dimensions as prompt points, a student model is used for real-time segmentation, a mapping matrix is established between the instance label and the three-dimensional key points after the instance label is obtained, and the instance label is further optimized through clustering, so that efficient and accurate three-dimensional open scene instance segmentation is realized; the application can fully capture global and local information, improve the accuracy of instance segmentation, reduce label redundancy and errors, and ensure the accuracy of the instance label; finally, the application has the ability to efficiently process three-dimensional scenes and is suitable for fast instance segmentation tasks in any three-dimensional scene.

Description

Technical Field

[0001] This invention belongs to the field of instance segmentation technology, and particularly relates to an open scene instance segmentation method based on decoupled distillation. Background Technology

[0002] Traditional instance segmentation and semantic segmentation algorithms require training on a given target category. If the test samples contain new targets outside the set of categories, high-precision segmentation becomes difficult. With the development of large model technology, exploring zero-shot segmentation methods with open-set capabilities has become a new trend. Currently, large visual language models for 2D images are widely used in fields such as medical image processing and defect detection due to their superior cross-scene generalization ability; examples include SAM, CLIP, and DINO. However, large models for 2D images cannot directly handle 3D open scenes. Therefore, some studies convert 3D scenes into multi-view images and then map the segmentation results to 3D space.

[0003] However, the computational complexity of multi-view images is far higher than that of a single image. To achieve dense segmentation of a 3D scene, the SAM model is needed to segment the multi-view images frame by frame. However, the image encoding module of this model uses a ViT network with a large number of parameters, which reduces the algorithm's efficiency. Furthermore, multi-view images suffer from problems such as target occlusion and semantic ambiguity, leading to significant differences in feature distributions of the same object in images from different perspectives, thus reducing the accuracy of 3D segmentation. Therefore, exploring an efficient and high-precision 3D scene segmentation method is essential. The decoupled distillation proposed in this invention can reduce the number of parameters in the model's image encoding module, significantly improving the computational speed of multi-view image segmentation and facilitating the rapid implementation of 3D scene segmentation tasks.

[0004] Chinese patent application number 2021115338951, entitled "Semantic Segmentation Model Compression System and Method Based on Knowledge Aggregation and Decoupling Distillation," proposes a model compression method comprising a feature layer comprehensive knowledge extraction and encoding module, an auxiliary predictor, and a label decoupling distillation module. This method optimizes the student model to approximate the teacher's comprehensive knowledge graph, achieving distillation learning of various semantic information and contextual knowledge. Finally, the label decoupling distillation module obtains the soft label distillation at the output end and the feature knowledge distillation at the intermediate layer. While this method can achieve comprehensive knowledge learning of aggregated features, enabling the student model to simulate the multi-dimensional feature extraction process of the teacher model, it does not consider the extraction and learning of image detail features, which may reduce the segmentation accuracy of the student model for small samples and edge regions. Furthermore, this method only achieves semantic segmentation for closed-set labels, is not applicable to open scenarios, and lacks open-set capability. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides an open scene instance segmentation method based on decoupled distillation, which can comprehensively improve the efficiency and accuracy of open scene segmentation.

[0006] An open scene instance segmentation method based on decoupled distillation includes the following steps:

[0007] S1: Input image I into the initial teacher model M respectively t and the initial student model M s Image I is obtained in the initial teacher model M t Spatial domain decoupling characteristics σ t-spatial and frequency domain decoupling characteristics σ t-freq And in the initial student model M s Spatial domain decoupling characteristics σ s-spatial and frequency domain decoupling characteristics σ s-freq ;

[0008] S2: Based on σ t-spatial σ s-spatial Constructing spatial domain semantic loss L spatial Based on σ t-freq σ s-freq Constructing frequency domain semantic loss L freq and adopt L spatial and L freq For the initial teacher model M respectively t and the initial student model M s Perform backpropagation to adjust the initial teacher model M. t and the initial student model M s The network parameters, until L spatial and L freq All values ​​are below a set threshold and no longer decrease with increasing training epochs, resulting in the distilled student model.

[0009] S3: Using a student model Perform instance segmentation on the multi-view 2D image of the scene to be segmented to obtain the instance label corresponding to each pixel in the 2D image;

[0010] S4: Use the fast histogram sampling method to select K key points from the 3D space corresponding to the scene to be segmented, and use the instance labels corresponding to the projection points of the key points on the 2D image as the labels of each key point. Among them, there are more than three key points on any two planes in the 3D space where the difference between the directions of any two normal vectors is greater than a set value.

[0011] S5: Use the instance label corresponding to each key point as the instance label of each pixel in its plane to obtain the mapping matrix M between each pixel and the instance label in the three-dimensional space.

[0012] S6: Perform instance label clustering on the mapping matrix M to obtain the final instance label corresponding to each pixel in the three-dimensional space, thus completing the instance segmentation in the three-dimensional space.

[0013] Furthermore, the spatial domain decoupling characteristic σ t-spatial and σ s-spatial The method for obtaining it is as follows:

[0014]

[0015] in, The initial teacher model M is respectively t The feature maps with different resolutions output by the n1, n2, n3, and n4 layers of the network; The initial student model M is respectively s The n1, n2, n3, and n4 layers of the network output feature maps at different resolutions; where n1, n2, n3, and n4 are the layer numbers of the four-layer network that generates the decoder input features in the initial teacher model and the initial student model, respectively, and the network layer numbers that generate the decoder input features are the same in both models;

[0016] Frequency domain decoupling characteristics σ t-freq and σ s-freq Both consist of high-frequency features and low-frequency features, and the methods for obtaining the high-frequency features and low-frequency features are as follows:

[0017]

[0018] LL1,LH1,HL1,HH1=W2(LL)

[0019]

[0020] in, The value is and W1(·) represents the first-level wavelet decomposition, W2(·) represents the second-level wavelet decomposition, and LL represents the decomposition of wavelets. The low-frequency components in the horizontal and vertical directions obtained from the first-level wavelet decomposition, where LH represents the component map of low frequency in both directions. The horizontal low-frequency and vertical high-frequency component maps obtained by performing first-level wavelet decomposition, where HL represents the component map of the first-level wavelet decomposition. The component diagrams of high-frequency horizontal direction and low-frequency vertical direction obtained by performing first-level wavelet decomposition, where HH represents the component diagram of high-frequency horizontal direction and low-frequency vertical direction. The graphs represent the horizontal and vertical high-frequency components obtained from the first-level wavelet decomposition of LL, respectively. LL1 represents the horizontal and vertical low-frequency components obtained from the second-level wavelet decomposition of LL, LH1 represents the horizontal and vertical low-frequency components obtained from the second-level wavelet decomposition of LL, HL1 represents the horizontal and vertical high-frequency components obtained from the second-level wavelet decomposition of LL, and HH1 represents the horizontal and vertical high-frequency components obtained from the second-level wavelet decomposition of LL. The value is and and and These are the low-frequency features of the initial teacher network and the initial student network, respectively. The value is and and and These are the high-frequency features of the initial teacher network and the initial student network, respectively.

[0021] Furthermore, spatial domain semantic loss L spatial The calculation method is as follows:

[0022]

[0023] Wherein, λ1 is the weight corresponding to the mean square distance loss MSE<·>, and λ2 is the weight corresponding to the cosine similarity loss COS<·>;

[0024] Frequency domain semantic loss L freq The calculation method is as follows:

[0025]

[0026] Where ||·|| represents the L2 norm.

[0027] Furthermore, the method for clustering instance labels on the mapping matrix M to obtain the final instance labels corresponding to each pixel in the three-dimensional space is as follows:

[0028] S601: Denoise and classify each pixel in the three-dimensional space by DBSCAN clustering, and select the label with the highest probability as the cluster label corresponding to each pixel in the three-dimensional space. The type of cluster label is the type of label remaining after the instance label is filtered out for noise.

[0029] S602: Calculate the C values ​​of adjacent pixels that do not belong to the same cluster label. j and C k The intersection and union ratio of IoU(C) j C k ), and according to the intersection-union ratio IoU(C j Ck If the value is not less than a set threshold θ, it is determined whether two adjacent pixels belong to the same object, and the cluster labels are merged. If the determination result is yes, the two adjacent pixels are in the same cluster label, and the one with the higher probability is selected as the cluster label of the two adjacent pixels. If the determination result is no, the cluster labels of the two adjacent pixels remain unchanged.

[0030] S603: The merged cluster labels C merge The combined mapping matrix M is obtained by merging the combined mapping matrix M. updated ;

[0031] S604: For each plane in 3D space where the difference in the normal vector directions between any two planes is greater than a set value, calculate the mapping matrix M for all pixels within each plane. updated The label variance ξ of the corresponding instance labels in p 1. Variance of normals for all pixels Δn; 2. Number of pixels N p Then determine the ξ corresponding to each plane. p , △n, N p If the set threshold conditions are met simultaneously, and the judgment result of any plane is yes, then the instance label of all pixels in that plane is unified to the instance label that appears most frequently in that plane. If the judgment result of any plane is no, then the instance label of all pixels in that plane remains unchanged.

[0032] Furthermore, the set threshold condition is as follows:

[0033]

[0034] Wherein, ε1, ε2, and ε3 are the set threshold values.

[0035] Beneficial effects:

[0036] 1. This invention provides an open scene instance segmentation method based on decoupled distillation. Through multi-scale feature decomposition and two-level wavelet decomposition, decoupled features in the spatial and frequency domains are generated. The semantic loss of the teacher and student models is calculated for distillation training, with a focus on aligning fine-grained features in the frequency domain. To extend the base model to 3D scenes, 3D spatial keypoints are selected and projected onto a 2D model as cue points. Real-time segmentation is performed using the student model to obtain instance labels, which are then mapped to the 3D keypoints. Further optimization of instance labels through clustering achieves efficient and accurate 3D open scene instance segmentation. This invention can fully capture global and local information through multi-scale feature fusion and fine-grained feature alignment, improving the accuracy of instance segmentation. Simultaneously, it utilizes 3D keypoint projection and a lightweight student model for rapid segmentation, and reduces label redundancy and errors through clustering and optimization methods, ensuring the accuracy of instance labels. Finally, this invention has the ability to efficiently process 3D scenes and is suitable for fast instance segmentation tasks in any 3D scene.

[0037] 2. This invention provides an open scene instance segmentation method based on decoupled distillation. By aligning spatial and frequency domain features at multiple scales, the number of parameters in the student model is significantly reduced. Furthermore, this invention introduces point cloud clustering, nonmaximum suppression, and planar priors into the label optimization strategy, which can effectively reduce label redundancy and label errors, and is applicable to any point cloud data. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of decoupled distillation provided by the present invention;

[0039] Figure 2 This is a schematic diagram of 3D-2D label mapping provided by the present invention;

[0040] Figure 3 A flowchart illustrating the label optimization strategy provided by this invention. Detailed Implementation

[0041] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0042] like Figure 1 As shown, an open scene instance segmentation method based on decoupled distillation includes the following steps:

[0043] S1: Input image I into the initial teacher model M respectively t and the initial student model M s Image I is obtained in the initial teacher model M t Spatial domain decoupling characteristics σ t-spatial and frequency domain decoupling characteristics σ t-freqAnd in the initial student model M s Spatial domain decoupling characteristics σ s-spatial and frequency domain decoupling characteristics σ s-freq ;

[0044] It should be noted that this invention selects the two-dimensional segmentation basic models with high parameter count SAM2_Large and low parameter count SAM2_Tiny as the teacher model M, respectively. t And student model M s The comprehensive spatial domain features of the input image I are extracted, and then the features are subjected to multi-scale spatial decomposition and two-level wavelet decomposition to generate decoupled features in the spatial domain and frequency domain.

[0045] Spatial domain decoupling characteristics σ t-spatial and σ s-spatial The method for obtaining it is as follows:

[0046]

[0047] in, The initial teacher model M is respectively t The feature maps with different resolutions output by the n1, n2, n3, and n4 layers of the network; The initial student model M is respectively s The feature maps at different resolutions output by the n1, n2, n3, and n4 layers of the network; where n1, n2, n3, and n4 are the layer numbers of the four layers with the highest weights in the initial teacher model and the initial student model, respectively, and the layer numbers of the four layers with the highest weights in the two models are the same.

[0048] Furthermore, the frequency domain decoupling characteristic σ t-freq and σ s-freq Both consist of high-frequency and low-frequency features, while high-resolution feature maps... It can better capture details in images, such as edges, textures, and small objects. Therefore, for Wavelet decomposition can yield rich low-frequency information. and high-frequency information High-frequency information contains more image details, and the methods for obtaining high-frequency and low-frequency features are as follows:

[0049]

[0050] LL1,LH1,HL1,HH1=W2(LL)

[0051]

[0052] in, The value is and W1(·) represents the first-level wavelet decomposition, W2(·) represents the second-level wavelet decomposition, and LL represents the decomposition of wavelets. The low-frequency components in the horizontal and vertical directions obtained from the first-level wavelet decomposition, where LH represents the component map of low frequency in both directions. The horizontal low-frequency and vertical high-frequency component maps obtained by performing first-level wavelet decomposition, where HL represents the component map of the first-level wavelet decomposition. The component diagrams of high-frequency horizontal direction and low-frequency vertical direction obtained by performing first-level wavelet decomposition, where HH represents the component diagram of high-frequency horizontal direction and low-frequency vertical direction. The graphs represent the horizontal and vertical high-frequency components obtained from the first-level wavelet decomposition of LL, respectively. LL1 represents the horizontal and vertical low-frequency components obtained from the second-level wavelet decomposition of LL, LH1 represents the horizontal and vertical low-frequency components obtained from the second-level wavelet decomposition of LL, HL1 represents the horizontal and vertical high-frequency components obtained from the second-level wavelet decomposition of LL, and HH1 represents the horizontal and vertical high-frequency components obtained from the second-level wavelet decomposition of LL. The value is and and and These are the low-frequency features of the initial teacher network and the initial student network, respectively. The value is and and and These are the high-frequency features of the initial teacher network and the initial student network, respectively.

[0053] S2: Based on σ t-spatial σ s-spatial Constructing spatial domain semantic loss L spatial Based on σ t-freq σ s-freq Constructing frequency domain semantic loss L freq and adopt L spatial and L freq For the initial teacher model M respectively t and the initial student model M s Perform backpropagation to adjust the initial teacher model M. t and the initial student model M s The network parameters, until L spatial and L freq All values ​​are below a set threshold and no longer decrease with increasing training epochs, resulting in the distilled student model.

[0054] Spatial domain semantic loss L spatial As long as it is the spatial domain decoupling characteristic σ t-spatial and σ s-spatialThe mean square distance loss and cosine similarity loss are weighted and summed, and the specific calculation formula is as follows:

[0055]

[0056] Wherein, λ1 is the weight corresponding to the mean square distance loss MSE<·>, and λ2 is the weight corresponding to the cosine similarity loss COS<·>;

[0057] This invention primarily focuses on denoising high-frequency information generated by wavelet decomposition, filtering out image noise, and preserving key detail regions. Based on this, the frequency domain semantic loss L... freq The calculation formula is as follows:

[0058]

[0059] Where ||·|| represents the L2 norm.

[0060] It should be noted that the semantic loss of the decoupled features obtained by the teacher model and the student model in this invention is calculated separately. This loss takes into account both global and local information and focuses on aligning fine-grained features. Combining the two losses can achieve efficient model distillation.

[0061] S3: Using a student model Perform instance segmentation on the multi-view 2D image of the scene to be segmented to obtain the instance label corresponding to each pixel in the 2D image;

[0062] It should be noted that the distilled student model can only be used for 2D instance segmentation. To extend to 3D scenes, this invention selects K key points in 3D space, projects these key points onto a 2D image using camera intrinsic and extrinsic parameters, and uses these key points as cue points for segmentation. The cue point labels are then used as instance labels for the corresponding key points. See steps S4 to S6 for details. Figure 2 As shown.

[0063] S4: Use the fast histogram sampling method to select K key points from the 3D space corresponding to the scene to be segmented, and use the instance labels corresponding to the projection points of the key points on the 2D image as the labels of each key point. Among them, there are more than three key points on any two planes in the 3D space where the difference between the directions of any two normal vectors is greater than a set value, so that the object edges and texture details where the normal vectors change drastically will get more sampling points.

[0064] S5: Use the instance label corresponding to each key point as the instance label of each pixel in its plane to obtain the Boolean mapping matrix M between each pixel and the instance label in the three-dimensional space;

[0065] S6: Perform instance label clustering on the mapping matrix M to obtain the final instance label corresponding to each pixel in the three-dimensional space, thus completing the instance segmentation in the three-dimensional space;

[0066] In other words, after obtaining the mapping matrix M, this invention further utilizes DBSCAN clustering, nonmaximum suppression, and multiplane detection to optimize instance labels, reducing label redundancy and label errors, such as... Figure 3 As shown, the specific steps include:

[0067] S601: Denoise and classify each pixel in the three-dimensional space by DBSCAN clustering, and select the label with the highest probability as the cluster label corresponding to each pixel in the three-dimensional space. The type of cluster label is the same as the type of instance label.

[0068] S602: Calculate the C values ​​of adjacent pixels that do not belong to the same cluster label. j and C k The intersection and union ratio of IoU(C) j C k ), and according to the intersection-union ratio IoU(C j C k If the value is not less than a set threshold θ, it is determined whether two adjacent pixels belong to the same object, and the cluster labels are merged. If the determination result is yes, the two adjacent pixels are in the same cluster label, and the one with the higher probability is selected as the cluster label of the two adjacent pixels. If the determination result is no, the cluster labels of the two adjacent pixels remain unchanged.

[0069] The above process can be expressed by the following formula:

[0070] C merge =C j ∪C k if IoU(C j C k )≥θ

[0071] S603: The merged cluster labels C merge The combined mapping matrix M is obtained by merging the combined mapping matrix M. updated ;

[0072] The above process can be expressed by the following formula:

[0073] M updated =M∪C merge

[0074] S604: For each plane in 3D space where the difference in the normal vector directions between any two planes is greater than a set value, calculate the mapping matrix M for all pixels within each plane. updated The label variance ξ of the corresponding instance labels inp 1. Variance of normals for all pixels Δn; 2. Number of pixels N p Then determine the ξ corresponding to each plane. p , △n, N p If the set threshold conditions are met simultaneously, and the judgment result of any plane is yes, then the instance label of all pixels in that plane is unified to the instance label that appears most frequently in that plane. If the judgment result of any plane is no, then the instance label of all pixels in that plane remains unchanged.

[0075] The set threshold condition is:

[0076]

[0077] Wherein, ε1, ε2, and ε3 are the set threshold values.

[0078] Therefore, to address the label redundancy problem caused by surface texture structure, this invention further detects the planar structure present in pixels and calculates the coordinates of pixels within the plane and the label variance ξ. p Normal difference △n and number of pixels N p If the threshold condition is met, the labels of the plane pixels that meet the condition will be merged to reduce label redundancy.

[0079] In summary, this invention provides an open-scene instance segmentation method based on decoupled distillation. This method generates decoupled features in the spatial and frequency domains through multi-scale feature decomposition and two-level wavelet decomposition. Semantic losses of the teacher and student models are calculated for distillation training, with a focus on aligning fine-grained features in the frequency domain. To extend the base model to 3D scenes, 3D spatial keypoints are selected and projected onto a 2D plane as cue points. Real-time segmentation is performed using the student model to obtain instance labels, which are then mapped to the 3D keypoints. Further optimization of instance labels is achieved through DBSCAN clustering, non-maximum suppression, and multi-plane detection, resulting in efficient and accurate 3D open-scene instance segmentation. This method can fully capture global and local information through multi-scale feature fusion and fine-grained feature alignment, improving the accuracy of instance segmentation. Simultaneously, rapid segmentation is achieved using 3D keypoint projection and a lightweight student model, while clustering and optimization methods reduce label redundancy and errors, ensuring the accuracy of instance labels. This method has the ability to efficiently process 3D scenes and is suitable for fast instance segmentation tasks in any 3D scene.

[0080] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. An open scene instance segmentation method based on decoupled distillation, characterized in that, The method comprises the following steps: S1: Transfer image I Input the initial teacher model respectively and the initial student model To obtain the image I In the initial teacher model Spatial domain decoupling characteristics Frequency domain decoupling characteristics and in the initial student model Spatial domain decoupling characteristics Frequency domain decoupling characteristics ; S2: based on , constructing a spatial domain semantic loss , based on , constructing a frequency domain semantic loss , then adopting and to perform back propagation on the initial teacher model , while adopting and to perform back propagation on the initial student model , so as to adjust the network parameters of the initial teacher model and the initial student model , until and are both less than a set threshold and no longer decrease with the increase of training rounds, to obtain a distilled student model ; S3: Adopting student model Instance segmentation is performed on the multi-view two-dimensional images of the scene to be segmented to obtain instance labels corresponding to each pixel point of the two-dimensional images; S4: Select from the 3D space corresponding to the scene to be segmented using the fast histogram sampling method. K There are 1 key points, and the instance labels corresponding to the projection points of the key points on the 2D image are used as the labels of each key point. Among them, there are more than 3 key points on any plane in 3D space where the difference between any two normal vector directions is greater than a set value. S5: taking the instance label corresponding to each key point as the instance label of each pixel point in the plane to which the key point belongs, to obtain a mapping matrix between each pixel point and the instance label in the three-dimensional space M ; S6: mapping matrix M Instance label clustering is performed to obtain the final instance label corresponding to each pixel point in the three-dimensional space, and instance segmentation of the three-dimensional space is completed.

2. The open scene instance segmentation method based on decoupled distillation of claim 1, wherein, Spatial domain decoupling features And The acquisition method is: in, ~ The initial teacher model The Feature maps at different resolutions output by the layer network; ~ The initial student model The Feature maps at different resolutions output by the layer network; among them... These are the layer numbers of the four-layer network that generates the decoder input features in the initial teacher model and the initial student model, respectively, and the layer numbers of the network that generates the decoder input features are the same in both models; Frequency domain decoupling features And Both are composed of high frequency features and low frequency features, and the acquisition method of high frequency features and low frequency features is: wherein, the value of and , is a first level wavelet decomposition, is a second level wavelet decomposition, represents a component graph of a horizontal direction low frequency and a vertical direction low frequency obtained by performing the first level wavelet decomposition on , represents a component graph of a horizontal direction low frequency and a vertical direction high frequency obtained by performing the first level wavelet decomposition on , represents a component graph of a horizontal direction high frequency and a vertical direction low frequency obtained by performing the first level wavelet decomposition on , represents a component graph of a horizontal direction high frequency and a vertical direction high frequency obtained by performing the first level wavelet decomposition on , represents a component graph of a horizontal direction low frequency and a vertical direction low frequency obtained by performing the second level wavelet decomposition on , represents a component graph of a horizontal direction low frequency and a vertical direction high frequency obtained by performing the second level wavelet decomposition on , represents a component graph of a horizontal direction high frequency and a vertical direction low frequency obtained by performing the second level wavelet decomposition on , represents a component graph of a horizontal direction high frequency and a vertical direction high frequency obtained by performing the second level wavelet decomposition on , the value of and , and and are low frequency features of the initial teacher network and the initial student network, respectively, the value of and , and and are high frequency features of the initial teacher network and the initial student network, respectively.

3. The open scene instance segmentation method based on decoupled distillation of claim 2, wherein, Spatial domain semantic loss The calculation method is: wherein, is a squared Euclidean distance loss is a corresponding weight, is a cosine similarity loss is a corresponding weight; Frequency domain semantic loss The calculation method is: wherein denotes the two-norm.

4. The open scene instance segmentation method based on decoupled distillation of claim 1, wherein, The method for obtaining the final instance label corresponding to each pixel point in the three-dimensional space by performing instance label clustering on the mapping matrix M is specifically as follows: S601: Perform denoising and classification on each pixel point in the three-dimensional space through DBSCAN clustering, and select the label with the highest probability as the clustering label corresponding to each pixel in the three-dimensional space, wherein the type of the clustering label is the label type remaining after the instance label is filtered from noise points; S602: Calculate the adjacent pixel points not belonging to the same cluster label respectively and the intersection-over-union ratio , and determine whether the two adjacent pixel points belong to the same object according to whether the intersection-over-union ratio is not less than a set threshold , complete the merging of the cluster labels; wherein, if the determination result is yes, the two adjacent pixel points are the same cluster label, and the corresponding probability of one of the two cluster labels is selected as the cluster label of the two adjacent pixel points; if the determination result is no, the cluster labels of the two adjacent pixel points remain unchanged; S603: merging the cluster labels after completion and the mapping matrix M to obtain a merged mapping matrix ; S604: For each plane in which the normal vector direction difference between each other in the three-dimensional space is greater than a set value, respectively calculate the label variance of the corresponding instance label of all pixel points in the plane in the mapping matrix , , , , , , whether the corresponding plane satisfies the set threshold condition, if the judgment result of any plane is yes, then the instance label of all pixel points in the plane is unified as the instance label with the highest occurrence frequency in the plane, if the judgment result of any plane is no, then the instance label of all pixel points in the plane remains unchanged.

5. The open scene instance segmentation method based on decoupled distillation of claim 4, wherein, The set threshold condition is: wherein , , are set threshold values, respectively.

Images