An unsupervised learning-based interactive image segmentation method

Abstract

The application provides an interactive image segmentation method based on unsupervised learning. The method comprises the following steps: firstly, using a pre-trained neural network model as a feature extractor to extract features of images in an unlabeled image dataset, then clustering the extracted features, assigning labels according to the clustering results, and using a superpixel voting algorithm to smooth the edges of the labels, and finally filtering out background pixels to obtain pseudo-labels that can be used for interactive segmentation; an interactive segmentation network that can be iteratively executed is constructed, the pseudo-labels are input into the network for training, and the outer points are generated according to the boundary box of the labels; and then, according to the comparison between the initial segmentation result and the true value, the center of the maximum error segmentation area is simulated to simulate the interactive points to obtain an iterative unsupervised interactive segmentation model. The application also realizes adaptive optimization during testing, so that the model of the application can adapt to different application scenarios and fields.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image analysis, specifically to an interactive image segmentation method based on unsupervised learning. Background Technology

[0002] Interactive image segmentation is a technique that separates target objects from complex image backgrounds using minimal user interaction (such as clicking, providing bounding boxes, drawing, etc.). Traditional interactive image segmentation methods rely on the local similarity between image pixels and the abrupt changes in boundaries, using user-provided interactions to provide hard constraints and segment objects based on these features. However, traditional image methods only utilize local relationships between image pixels and cannot utilize the global structural information of objects in the image. Therefore, they often lack robustness for images with similar textures and colors or containing noise. In recent years, supervised interactive image segmentation methods based on deep learning have emerged. These methods use known labeled data to build and train accurate segmentation neural network models to predict the category of data with unknown labels. The segmentation results obtained by supervised methods are usually highly interpretable, but they still depend on labeled data. Pixel-level labels are dense annotations, and each pixel requires execution under extremely costly conditions. Undoubtedly, this cost is expensive, laborious, and time-consuming. Interactive image segmentation uses human annotators to annotate images in a simple and effective way, but its training network model itself also depends on the labeled images. Furthermore, it is also limited by the image domain; for example, images obtained from natural scene samples often perform poorly in medical images, requiring retraining. Therefore, unsupervised interactive segmentation that does not rely on labeled data is necessarily a novel solution. Summary of the Invention

[0003] The purpose of this invention is to provide an interactive image segmentation method based on unsupervised learning, so as to solve the problems of limited segmentation results and time-consuming and labor-intensive processes in the existing technology based on supervised methods.

[0004] This invention is implemented as follows: an interactive image segmentation method based on unsupervised learning, comprising the following steps:

[0005] a. Use a pre-trained neural network model as a feature extractor to extract features from images in an unlabeled image dataset. Then, cluster the extracted features to establish cross-image correspondences between pixels. Assign labels based on the clustering results. Use a superpixel voting algorithm to smooth the edges of these labels. Finally, filter out background pixels to obtain pseudo-labels that can be used for interactive segmentation.

[0006] b. Construct an interactive segmentation network that can be executed iteratively. Feed the pseudo-labels obtained in step a into the network for training. Generate outbound points with the bounding boxes of the labels. Then, based on the comparison between the initial segmentation results and the ground truth, obtain the center of the largest missegmented region and simulate the interactive points of the iteration to obtain an iterative unsupervised interactive segmentation model.

[0007] c. Treat human interaction clicks as positive samples, adjust network parameters during testing, optimize the model by constructing a sparse click loss function, and avoid the model forgetting the initial good predictions of the image by adding a cost function that changes the network parameters.

[0008] In the above scheme, step a is as follows:

[0009] a-1. Use a pre-trained neural network model as a feature extractor to extract features; this pre-trained model is obtained through image-level self-supervised contrastive learning; specifically, given a set of unlabeled images defined as X = {x1, x2, ... x...} i}, where x i Let f be the i-th sample image in the sample set. θ (·) represents the feature extractor of an existing unsupervised method. After extracting features from all image samples in the sample set, for image x... i The generated feature map f θ (x i Each pixel p of f will generate a feature representation. Define f θ (x i [p] represents the pixel-level feature representation. The K-means algorithm is used to cluster each pixel of the feature map generated for all image samples:

[0010]

[0011] Where y ip μ represents the cluster center label of the p-th pixel in the feature map generated from the i-th image. k It is the k-th cluster center. After clustering, for each image x... i Each pixel p in the generated feature map can be assigned to its corresponding label y. ip We upsample it to the size of the original image to obtain the image's label map;

[0012] a-2. The edge information of the obtained label map is optimized using a superpixel voting algorithm. Specifically, oversegmented superpixels are obtained through this algorithm. The superpixels are generated by treating the original image as an undirected graph G = (V, E), where V represents the set of vertices, E represents the set of edges, and each edge connects two adjacent pixels (i, j). The weight w of the edge represents the squared difference of the RGB values ​​between the two vertices. In the initial stage, each pixel compares its weights to its eight neighboring pixels, arranges them in ascending order, selects the edge with the smallest weight, and merges them into one region, thus generating many minimum spanning trees. If the weight of the edge connecting two minimum spanning trees is less than or equal to the minimum value of the edge with the largest weight in those two trees, then the two trees are merged. This rule is continuously updated on the undirected graph until no trees satisfy the merging condition is found. The set of pixels in these trees is the superpixel. After obtaining the oversegmented superpixels, the superpixel with the most labels is assigned to all pixels within it, aligning the image label boundaries with the image boundaries.

[0013] a-3. Filter out the most frequently occurring categories and treat them as background, and treat the remaining tags as foreground tags and utilize them.

[0014] In the above scheme, step b is as follows:

[0015] b-1. Obtain a bounding box around the label of each category based on its leftmost, rightmost, topmost, and bottommost pixels; then select the four corner points of the bounding box to simulate human clicking behavior.

[0016] b-2. Treat the four corner points mentioned above as background clicks, and generate 2D Gaussian kernels at the coordinates of these corner points to represent the clicks and generate heatmaps. In the input part of the neural network, the heatmaps generated by the background clicks and the three RGB channels of the image are connected together as the input of the neural network. Then, the results predicted by the neural network and the assigned labels are used to perform binary cross-entropy loss for training, and finally the model is obtained.

[0017] b-3. Optimize the network model using labeled data.

[0018] In step b-2, an iterative training process was added later, incorporating a foreground click heatmap in addition to the background click heatmap for iterative training. First, the network model of this invention requires two initial clicks (two diagonally opposite corner points) to determine a bounding box surrounding an object. If the user is not satisfied with the current segmentation output, they can continue clicking for optimization. Specifically, during training, a new click is added to the center of the largest segmentation error region. If this point is located in a foreground pixel, it is considered a foreground click; if it is located in a background pixel, it is considered a background click. The iteration is performed only once during training, and the loss is the sum of the losses from the initial segmentation and the corrected segmentation results.

[0019] In the above scheme, step c is as follows:

[0020] c-1. Treat human interaction clicks as correct sample labels, and correct network parameters during testing to achieve continuous learning;

[0021] c-2. Treat the corrected clicks as ground truth labels and optimize the model by minimizing the sparse click-correction-based loss function; the click-correction-based loss function is shown in Equation (2):

[0022]

[0023] Where 1[.] is the indicator function; c is a vector with values ​​{1,0,-1}, where 1 represents that the interacting pixels are corrected to foreground pixels, 0 represents that the interacting pixels are corrected to background pixels, and the remaining pixels are marked as -1; f(x; θ) represents the feature mapping of the neural network model under the condition of parameter θ, and |.| represents L1 normalization;

[0024] c-3. By adding a cost function that changes the network parameters, the model can be prevented from forgetting its initial good predictions of the image; the cost function based on the change of network parameters is shown in formula (3):

[0025] L F (θ)=Ω(θ-θ * ) ⊙2 (3)

[0026] Where θ * Let be the initial network parameters, and θ be the updated network parameters. ⊙2 Ω represents the element-wise square, and is the importance coefficient of each network parameter, obtained from the accumulated gradient of the image, as shown in formula (4):

[0027]

[0028] Where x iIt is the i-th image sample in the dataset, N is the total number of data points in the dataset, and g(.) is the gradient generated when the image sample passes through the neural network;

[0029] c-4. Finally, the loss function based on click correction and the loss function that penalizes changes in important network parameters are combined to obtain the loss function corrected during testing, as shown in formula (5):

[0030] L combine (x,c,θ)=L CE (x,c,θ)+λL F (θ) (5)

[0031] Where λ is the control parameter.

[0032] This invention proposes an unsupervised interactive segmentation method that can learn patterns from unlabeled data and perform image segmentation without relying on any image annotations. Furthermore, this segmentation model can perform additional click optimization when the user is dissatisfied with the initial segmentation result, iteratively obtaining better results. This invention also overcomes the limitation of existing methods in cross-domain compatibility by employing test-time adaptive optimization. Through user-corrected clicks, the network model parameters are optimized during testing, enabling continuous learning and allowing the model to adapt to specific objects, domains, etc.

[0033] This invention trains an interactive segmentation network by clustering unlabeled image datasets using deep clustering to obtain labels, achieving an interactive segmentation model without using any labeled data. This saves significant manpower and resources. Furthermore, unsupervised learning can learn more adaptive and richer features, enabling cross-domain image segmentation without retraining samples. This invention also implements adaptive optimization during testing, allowing the model to adapt to different application scenarios and domains.

[0034] This invention first uses a pre-trained neural network model as a feature extractor to extract features from an unlabeled image dataset. Then, these features are clustered to establish cross-image correspondences between pixels. Labels are assigned based on the clustering results, and a superpixel voting algorithm is used to smooth the edges of these labels. Finally, background pixels are filtered out to obtain pseudo-labels suitable for interactive segmentation. These pseudo-labels are then fed into the network for training. Outline points are generated using the label bounding boxes, and these are used as background clicks to interactively train the neural network, yielding initial segmentation results. Figure 3This invention demonstrates how the bounding boxes generated by two outliers are used to obtain the segmentation result on a natural image after interaction. Furthermore, the invention obtains an iterative unsupervised interactive segmentation network model by simulating the interaction points of the simulated interaction points to obtain the center of the largest mis-segmented region based on the comparison between the initial segmentation result and the ground truth. Optionally, a small amount of labeled data can be fed into the network for optimization. The invention also implements test-time adaptive optimization, treating human interaction clicks as positive samples. During testing, the network parameters are adjusted, the model is optimized by constructing a loss function that minimizes sparse clicks, and the model avoids forgetting its initial good predictions of the image by adding a cost function that changes the network parameters. Figure 4 This invention demonstrates the cross-domain segmentation results obtained on medical images using iterative clicks and adaptive optimization during testing. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating the method of the present invention.

[0036] Figure 2 This is a schematic diagram illustrating the steps of the method for generating pseudo-tags according to the present invention.

[0037] Figure 3 This is a comparison between the segmentation result image obtained by using the method of the present invention to determine the bounding box of a natural scene image with the ground truth image in an embodiment of the present invention.

[0038] Figure 4 This is a segmentation result image obtained by using the method of the present invention on a medical image with different numbers of clicks in an embodiment of the present invention. Detailed Implementation

[0039] The implementation of this invention includes the following steps:

[0040] 1) Use a pre-trained neural network model as a feature extractor to extract features from the unlabeled image dataset, then cluster these features to establish cross-image correspondences between pixels, assign labels based on the clustering results, use a superpixel voting algorithm to smooth the edges of these labels, and finally filter out background pixels to obtain pseudo-labels that can be used for interactive segmentation.

[0041] 2) Construct an interactive segmentation network that can be executed iteratively. Feed the pseudo-labels obtained in the previous step into the network for training, and generate outliers using the bounding boxes of the labels. Compare the initial segmentation results with the ground truth to obtain the region with the largest missegmentation, and use the center of this region as the simulated user interaction click position. Through iterative training, calculate the loss stages of the initial segmentation results and the corrected segmentation results respectively to obtain an iterative unsupervised interactive segmentation model. Optionally, this invention can also use a small amount of labeled data to feed into the network for optimization.

[0042] 3) Adaptive optimization during testing: Treat human interaction clicks as positive samples, adjust network parameters during testing, optimize the model by constructing a sparse click loss function, and avoid the model forgetting the initial good predictions of the image by adding a cost function that changes the network parameters.

[0043] Step 1) is as follows:

[0044] (1) First, a pre-trained neural network model is used as a feature extractor to extract features. This pre-trained model is obtained through image-level self-supervised contrastive learning, without any labeled data. Specifically, a set of unlabeled images is defined as X = {x1, x2, ... x...}. i}, where x i Let f be the i-th sample image in the sample set. θ (·) represents the feature extractor of an existing unsupervised method. After extracting features from all image samples in the sample set, for image x... i The generated feature map f θ (x i Each pixel p of f will generate a feature representation. Define f θ (x i [p] represents the pixel-level feature representation. The K-means algorithm is used to cluster each pixel of the feature map generated for all image samples:

[0045]

[0046] Where y ip μ represents the cluster center label of the p-th pixel in the feature map generated from the i-th image. k It is the kth cluster center.

[0047] Through feature clustering, each pixel is assigned to an appropriate category based on the distance between its features and the cluster center. This method can group pixels with similar semantic and visual features and establish cross-image correspondences between pixels.

[0048] (2) After completing the clustering, for each image x i Each pixel p in the generated feature map can be assigned to its corresponding label y. ipWe upsample the image to the size of the original image to obtain the label map. However, the feature space and resolution of the feature map are limited by the feature extractor, which may result in a noisy label map. Therefore, this invention uses a simple and effective superpixel voting algorithm (Felzenszwalb algorithm) to optimize the edge information of the obtained label map. The Felzenszwalb algorithm obtains oversegmented superpixels, where the superpixel generation is based on treating the original image as an undirected graph G = (V, E), where V represents the set of vertices, E represents the set of edges, each edge connects two adjacent pixels (i, j), and the weight w of the edge represents the squared difference of the RGB values ​​between the two vertices. In the initial stage, each pixel compares its weights to its eight neighboring pixels, arranges them in ascending order, selects the edge with the smallest weight, and merges them into one region, thus generating many minimum spanning trees. If the weight of the edge connecting two minimum spanning trees is less than or equal to the minimum value of the edge with the largest weight in those two trees, then the two trees are merged. This rule is continuously updated in the undirected graph until no tree satisfies the merging condition. The set of pixels in these trees is called a superpixel. After obtaining the oversegmented superpixels, the superpixel with the most labels is assigned to all pixels within that superpixel. This aligns the boundaries of the image labels with the boundaries of the image, resulting in smoother labeling.

[0049] (3) The labels of the image obtained at this time include pixels of the background category and pixels of the foreground category. Interactive segmentation usually extracts foreground information through user interaction. In images of everyday scenes, background pixels usually occupy most of the image, so the most frequently occurring category is filtered out and treated as the background, and the remaining labels are treated as foreground labels and utilized.

[0050] Step 2) is as follows:

[0051] (1) First, to simulate human interaction, a bounding box surrounding the label of each category is obtained based on its leftmost, rightmost, topmost, and bottommost pixels. Then, the four corner points of the bounding box are selected to simulate human clicking behavior. Because during the interaction, clicking two opposite corner points, such as the top left and bottom right corners, can automatically infer the other two points, so in real-time interaction, only two clicks are needed to determine a bounding box surrounding the object.

[0052] (2) Treat the four corner points as background clicks, and generate a 2D Gaussian kernel for the coordinates of these corner points as a representation of the clicks to produce a heatmap. In the input part of the neural network, the heatmap generated by the background clicks and the three RGB channels of the image are concatenated as the input of the neural network. Then, the results predicted by the neural network and the assigned labels are used to perform binary cross-entropy loss for training, and finally the model is obtained.

[0053] Two background clicks alone may not yield accurate segmentation results. Therefore, an iterative training process was added, incorporating a foreground click heatmap in addition to the background click heatmap for iterative training. First, the network model of this invention requires two initial clicks (two diagonally opposite corner points) to determine a bounding box surrounding the object. If the user is not satisfied with the current segmentation output, they can continue clicking for optimization. During training, a new click is added to the center of the largest segmentation error region. If this point is located in a foreground pixel, it is considered a foreground click; if it is located in a background pixel, it is considered a background click. The training iteration is performed only once, and the loss is the sum of the losses from the initial segmentation and the corrected segmentation results.

[0054] (3) Due to the limitations of coarse pseudo-labels, the image segmentation results obtained are somewhat different from those of supervised methods. It is advisable to optimize the network model using a small amount of labeled data.

[0055] Step 3) is as follows:

[0056] (1) First, human interactions can be considered as correct sample labels, and network parameters can be corrected during testing to achieve continuous learning. This method allows the model to adapt to specific objects and backgrounds, changes in the distribution of the test set, and even large domain changes.

[0057] (2) Treating the corrected clicks as ground truth labels, the model is optimized by minimizing the sparse click-correction-based loss function. The click-correction-based loss function is shown in Equation (2):

[0058]

[0059] Where 1[.] is the indicator function; c is a vector with values ​​{1, 0, -1}, where 1 represents that the interacting pixel is corrected to a foreground pixel, 0 represents that the interacting pixel is corrected to a background pixel, and the remaining pixels are marked as -1. f(x; θ) represents the feature map of the neural network model under the condition of parameter θ, and |.| represents L1 normalization.

[0060] (3) However, user clicks are sparse relative to the entire image, providing limited spatial information about objects. If too much focus is placed on user click corrections, the model will degenerate. Therefore, to prevent the model from focusing solely on user corrections and forgetting initial prior knowledge, the approach of combating catastrophic forgetting in continuous learning is employed. This involves setting a cost function that changes the network parameters to prevent the model from forgetting its initial good predictions about the image. The cost function based on changing the network parameters is shown in equation (3):

[0061] L F(θ)=Ω(θ-θ * ) ⊙2 (3)

[0062] Where θ * Let be the initial network parameters, and θ be the updated network parameters. ⊙2 Ω represents the element-wise square, and is the importance coefficient of each network parameter, obtained from the accumulated gradient of the image, as shown in formula (4):

[0063]

[0064] Where x i It is the i-th image sample in the dataset, N is the total number of data points in the dataset, and g(.) is the gradient generated when the image sample passes through the neural network.

[0065] (4) Finally, the loss function based on click correction and the loss function that penalizes changes in important network parameters are combined to obtain the loss function corrected during testing, as shown in formula (5):

[0066] L combine (x,c,θ)=L CE (x,c,θ)+λL F (θ) (5)

[0067] Where λ is the adjustment parameter, through this loss function, a network model that can continuously adapt to interaction can be obtained during testing, and catastrophic forgetting during the learning process can also be avoided.

[0068] This invention was completed by the Hebei Provincial Natural Science Foundation Project (F2019201451). The preferred embodiments of the invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the scope of the invention.

[0069] This invention provides an unsupervised interactive segmentation method, comprising the following steps: First, a pre-trained neural network model is used as a feature extractor to extract features; this pre-trained model is obtained by training ResNet-50 using the image-level self-supervised contrastive learning method MOCO; the model training requires no labeled data; when using ResNet-50, features are extracted before its final adaptive average pooling layer, and the obtained feature vectors are L2 normalized. Since the total number of features in the unlabeled dataset can be very large, this invention randomly and uniformly selects 5000 images from the training set images of the COCO 2014 natural scene image dataset for clustering. Using the FAISS library to implement mini-batch K-means on a GPU, clustering can be completed in approximately 20 minutes.

[0070] After clustering, a label is obtained for each pixel of each image. However, the feature space and resolution of the feature map are limited by the feature extractor, which may result in a noisy label map. Therefore, this invention employs a simple and effective superpixel voting algorithm to optimize the edge information of the obtained label map. The segmented superpixel map is obtained using the Felzenszwalb algorithm, and the superpixel with the most labels is assigned to all pixels within that superpixel. This aligns the boundaries of the image labels with the boundaries of the image, resulting in smoother labeling.

[0071] The resulting image labels include pixels belonging to both the background and foreground categories. Interactive segmentation typically extracts foreground information through user interaction. In everyday scene images, background pixels usually occupy the majority of the image; therefore, the most frequently occurring category is filtered out and treated as background. The remaining labels are then used as foreground labels. In this invention, the K-means parameter is 30, and the top 5 clusters with the highest number of pixels are filtered out.

[0072] After obtaining the pseudo-labels, we use them to construct an interactive segmentation model. First, we define the interaction paradigm. We use the four corner points of the bounding box surrounding the label as the initial interaction clicks. Since only two opposite corner points are needed to obtain a defined bounding box, we define clicking two opposite corner points to define a bounding box surrounding the object as the initial interaction, thus obtaining the initial segmentation result. Figure 3 This is the segmentation result image of the model of this invention after using only the initial two clicks to determine the bounding box.

[0073] The initial segmentation results may not meet the requirements. During training, a human interaction correction process is simulated. The regions with segmentation errors can be identified by XORing the initial segmentation results with the label values. Based on the distance of pixels within these regions to the boundaries, the center of the largest error region is easily found. This error center is used to simulate a human interaction correction click operation; if the point belongs to a background pixel, it is considered a background click; if it belongs to a foreground pixel, it is considered a foreground click. During training, new clicks are added before the pyramid scene parsing module of the network model and encoded into Gaussian heatmaps for iterative forward propagation. The binary cross-entropy loss is calculated using the initial and iterative segmentation results with the labels, and this loss is used to train the network model. Optionally, a small amount of labeled data can be used for optimization. We achieved a 12% improvement using only 5% of the PASCAL dataset, with just 73 sample images.

[0074] During the testing phase, a test-time adaptive modeling technique was introduced. A loss function that minimizes click-based corrections was designed, treating user clicks as positive sample labels, thus enabling the optimization of model parameters during testing. To prevent sparse clicks from causing network degradation and forgetting initial segmentation information, regularization was applied to key network parameters to avoid catastrophic forgetting.

[0075] The network model of this invention is trained on unlabeled natural scene images, but its unsupervised approach allows it to learn high-dimensional features, resulting in greater adaptability and richness. We validated the model on medical images spanning a wide domain. Figure 4 As can be seen, the method of the present invention can still achieve good results on CT and MRI images.

Claims

1. An interactive image segmentation method based on unsupervised learning, characterized in that, Includes the following steps: a. Use a pre-trained neural network model as a feature extractor to extract features from images in an unlabeled image dataset. Then, cluster the extracted features to establish cross-image correspondences between pixels. Assign labels based on the clustering results. Use a superpixel voting algorithm to smooth the edges of these labels. Finally, filter out background pixels to obtain pseudo-labels that can be used for interactive segmentation. b. Construct an interactive segmentation network that can be executed iteratively. Feed the pseudo-labels obtained in step a into the network for training. Generate outbound points with the bounding boxes of the labels. Then, based on the comparison between the initial segmentation results and the ground truth, obtain the center of the largest missegmented region and simulate the interactive points of the iteration to obtain an iterative unsupervised interactive segmentation model. c. Treat human interaction clicks as positive samples, adjust network parameters during testing, optimize the model by constructing a sparse click loss function, and avoid the model forgetting the initial good predictions of the image by setting a cost function that changes the network parameters. Step a is as follows: a-1. Use a pre-trained neural network model as a feature extractor to extract features; this pre-trained model is obtained through image-level self-supervised contrastive learning; specifically, given a set of unlabeled images defined as X = {x1, x2, ..., x...} i }, where x i Let f be the i-th sample image in the sample set; θ (·) represents the feature extractor of an existing unsupervised method. After extracting features from all image samples in the sample set, for image x... i The generated feature map f θ (x i Each pixel p generates a feature representation; defined f θ (x i [p] represents the pixel-level feature representation. The K-means algorithm is used to cluster each pixel of the feature map generated for all image samples: Where y ip μ represents the cluster center label of the p-th pixel in the feature map generated from the i-th image. k It is the k-th cluster center; after clustering, for each image x i Each pixel p in the generated feature map can be assigned to its corresponding label y. ip By upsampling it to the size of the original image, we obtain the label map of the image; a-2. The edge information of the obtained label image is optimized using a superpixel voting algorithm. Specifically, oversegmented superpixels are obtained through the superpixel voting algorithm. The superpixel generation is based on treating the original image as an undirected graph G = (V, E), where V represents the set of vertices, E represents the set of edges, each edge connects two adjacent pixels (i, j), and the weight w of the edge represents the squared difference of the RGB values ​​between the two vertices. In the initial stage, each pixel compares its weights to its eight neighboring pixels, arranges them in ascending order, selects the edge with the smallest weight, and merges them into one region to form several minimum spanning trees. If the weight of the edge connecting two minimum spanning trees is less than or equal to the minimum value of the edge with the largest weight in these two trees, then the two trees are merged. This rule is continuously updated in the undirected graph until there are no trees that satisfy the merging condition. The set of pixels in these trees is called a superpixel. After obtaining the oversegmented superpixel, the superpixel with the most labels is assigned to all pixels of the superpixel, so that the boundaries of the image labels are aligned with the boundaries of the image. a-3. Filter out the most frequently occurring categories and treat them as background, and treat the remaining tags as foreground tags and utilize them.

2. The interactive image segmentation method based on unsupervised learning according to claim 1, characterized in that, Step b is as follows: b-1. Obtain a bounding box around the label of each category based on its leftmost, rightmost, topmost, and bottommost pixels; then select the four corner points of the bounding box to simulate human clicking behavior. b-2. Treat the four corner points mentioned above as background clicks, and generate 2D Gaussian kernels at the coordinates of these corner points to represent the clicks and generate heatmaps. In the input part of the neural network, the heatmaps generated by the background clicks and the three RGB channels of the image are connected together as the input of the neural network. Then, the results predicted by the neural network and the assigned pseudo-labels are used to perform binary cross-entropy loss for training, and finally the model is obtained.

3. The interactive image segmentation method based on unsupervised learning according to claim 2, characterized in that, Step b also includes the following steps: b-3. Optimize the network model using labeled data.

4. The interactive image segmentation method based on unsupervised learning according to claim 2, characterized in that, During the training process in step b-2, a foreground click heatmap is added to the heatmap containing only background clicks, and iterative training is performed.

5. The interactive image segmentation method based on unsupervised learning according to claim 1, characterized in that, Step c is as follows: c-1. Treat human interaction clicks as correct sample labels, and correct network parameters during testing to achieve continuous learning; c-2. Treat the corrected clicks as ground truth labels and optimize the model by minimizing the sparse click-correction-based loss function; the click-correction-based loss function is shown in Equation (2): Where l[.] is the indicator function; c is a vector with values ​​{1, 0, -1}, where 1 represents that the pixel of the interaction point is corrected to a foreground pixel, 0 represents that the pixel of the interaction point is corrected to a background pixel, and the remaining pixels are marked as -1; f(x; θ) represents the feature map of the neural network model under the condition of parameter θ, and |.| represents L1 normalization; c-3. Avoid the model forgetting its initial good predictions of the image by setting a cost function that changes the network parameters; The cost function based on changes in network parameters is shown in equation (3): L F (θ)=Ω(θ-θ * ) ⊙2 (3) Where θ * Let be the initial network parameters, and θ be the updated network parameters. ⊙2 Ω represents the element-wise square, and is the importance coefficient of each network parameter, obtained from the accumulated gradient of the image, as shown in formula (4): Where x i It is the i-th image sample in the dataset, N is the total number of data points in the dataset, and g(.) is the gradient generated when the image sample passes through the neural network; c-4. Finally, the loss function based on click correction and the loss function that penalizes changes in important network parameters are combined to obtain the loss function corrected during testing, as shown in formula (5): THE combine (x,c,θ)=L CE (x,c,θ)+λL F (θ) (5) Where λ is the control parameter.

Images