A click-based interactive image segmentation system and method

Abstract

The application provides a click-based interactive image segmentation system and method, and belongs to the technical field of image segmentation based on computer vision.The system comprises a global hint module, a self-attention feature extraction module, a feature pyramid sampling, a nonlinear segmentation head and a local correction module; wherein the designed global hint module enhances the ability of the model to more effectively utilize the guiding clues from the interactive graph; subsequently, after obtaining the segmentation mask, the system carefully evaluates the differences between the generated mask and the previously generated mask, and then performs targeted local optimization based on the designed local correction module to minimize these differences.The application realizes high-quality image segmentation through a small amount of user clicks, and aims to solve the problems of insufficient utilization of interactive graph information and poor new click guidance effect in the prior art, and improve the segmentation accuracy and efficiency.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image segmentation based on computer vision, and particularly relates to an interactive image segmentation system and method based on clicking. BACKGROUND

[0002] Current deep learning still requires a large amount of data. The problem of data annotation becomes more and more serious, especially for pixel-level image annotation, pure manual annotation needs to consume a lot of labor and time cost. Therefore, interactive image segmentation has attracted widespread attention. The goal of interactive image segmentation is to obtain high-quality segmentation masks through a small amount of user interaction. Interactive image segmentation includes various interaction methods, such as clicking, scribbling, bounding box and their combination. Among them, the method based on clicking is favored by most researchers because it is simpler and easier to point to the target of interest in the interaction process. The interactive image segmentation process based on clicking is specifically that for an image, the user indicates his interest by clicking on the target object once, and the model guides the user to obtain an initial segmentation mask. In the subsequent process, the user continues to click according to the segmentation result of the last round, applies positive clicks to the part that the user wants to judge as foreground, and applies negative clicks to the part that the user wants to judge as background, to obtain the next segmentation mask. This process is iterated until the user is satisfied with the segmentation mask. It is worth noting that from the second round of interaction, the segmentation result of the last round of interaction will be fed back into the current process. This feedback is crucial to the model because it provides prior information for the segmentation model, which helps to promote convergence and improve segmentation accuracy.

[0003] Recent research on interactive segmentation based on clicking mainly focuses on obtaining detailed target masks. For example, SimpleClick adopts a standard Vision Transformer (ViT) backbone network to capture global information to achieve accurate segmentation. These methods usually generate a single mask based on user clicks, so they are called single-mask methods. SimpleClick simply adds the interaction map to the original image before inputting the image into the network, failing to fully utilize the guidance of the interaction map. In addition, in subsequent interactions, SimpleClick treats the current click and the previous click equally, limiting its ability to focus more effectively on the current click. Not using the previous segmentation result as prior information will lead to inconsistent segmentation results. In another case, although the previous segmentation result is used, it is simply added to the original image before inputting into the network. Although this method contains prior information, the amount of information is relatively small compared to the original image. Therefore, this simple addition method does not fully utilize the prior information provided by the interaction map. SUMMARY

[0004] To solve the above problems, the application provides an interactive image segmentation system based on clicks in a first aspect, which comprises a global hint module, a self-attention feature extraction module, a feature pyramid sampling, a nonlinear segmentation head and a local correction module.

[0005] The global hint module introduces a patch embedding layer symmetrical to the patch embedding layer in the backbone, encodes the user click information into a disk map and splices the last round of segmentation mask to form a three-channel interactive map, and embeds the three-channel interactive map into the feature extraction network, so that the network can learn the object boundary and region information indicated by the user clicks.

[0006] The self-attention feature extraction module uses ViT as the backbone network and adopts a single-scale feature map structure to avoid information loss caused by feature map down-sampling, learn local detail information of the image, and use Transformer to learn global context information of the image.

[0007] The feature pyramid sampling part uses the last feature map to construct a multi-scale feature map on the basis of a single-scale feature pyramid, so that the network can capture features of objects at different scales.

[0008] The nonlinear segmentation head module uses an MLP layer for feature transformation and fusion to convert the multi-scale feature map into a segmentation probability map, avoiding complex convolution operations.

[0009] The local correction module decomposes one heavy inference on the whole image into two light predictions on small blocks, uses historical information, and focuses on fine processing of the local area of the new click area.

[0010] Preferably, the global hint module specifically comprises:

[0011] A patch embedding layer symmetrical to the patch embedding layer in the backbone is introduced, followed by element-wise feature addition; the user clicks are encoded in a two-channel disk map, one for positive clicks and the other for negative clicks, the positive clicks are placed on the foreground, and the negative clicks are placed in the background; the segmentation mask and the two-channel click map are connected as a three-channel map in the visual transformer for patch embedding, the two symmetrical embedding layers operate on the image and the connected three-channel map respectively, the input is repaired, flattened and projected into two vector sequences of the same dimension, and then element-wise addition is performed before inputting into the self-attention feature extraction module.

[0012] Preferably, the self-attention feature extraction module specifically comprises:

[0013] The self-attention feature extraction module uses a traditional ViT as the backbone network, which only maintains a single scale feature map throughout the process. The patch embedding layer divides the input image into non-overlapping fixed-size blocks (e.g., 16x16 for VIT-B), each of which is flattened and linearly projected into a fixed-length vector (e.g., 768 for VIT-B). The generated vector sequence is fed into a queue of Transformer blocks (e.g., 12 for ViT-B) for self-attention. The backbone is pre-trained on ImageNet-1k as MAEs. A few global self-attention blocks (e.g., 2 for ViT-B) assist in non-shifting window attention to adjust the pre-trained backbone to higher resolution inputs during fine-tuning. Global hint information is also added, and user interactions are encoded into a two-channel disk map, with one channel assigned to positive clicks, representing the foreground, and the other to negative clicks, representing the background. The pre-existing segmentation mask and the two-channel click map are combined to create a three-channel interaction map. GHM processes this map by dividing it into non-overlapping blocks of the same size (16x16 pixels). Each block is then flattened and linearly projected into a fixed-length vector space with 768 dimensions. This projection is done in triplicate, and the resulting set of three vectors is superimposed with the outputs of the first, sixth, and twelfth transformation layers to enhance the self-attention mechanism, addressing the issue of dilution of interaction map information.

[0014] Preferably, the feature pyramid sampling uses the last feature map output by the self-attention feature extraction module, and uses four convolutional layers with different strides to downsample the last feature map to resolutions of 1 / 32, 1 / 16, 1 / 8, and 1 / 4, respectively, generating multi-scale feature maps to enable the network to better capture object features at different scales.

[0015] Preferably, the nonlinear segmentation head uses an MLP layer to implement a lightweight segmentation head. It employs a simple feature pyramid to generate a segmentation probability map with a scale of 1 / 4, and then performs an upsampling operation to restore the original resolution. Please note that this segmentation head avoids computationally demanding components and only accounts for 1% of the model parameters. Crucially, with a powerful pre-trained backbone, the lightweight segmentation head is sufficient for interactive segmentation. The proposed all-MLP segmentation head works in three steps. First, each feature map from the simple feature pyramid is passed through an MLP layer to convert it to the same channel dimension. Second, all feature maps are upsampled to the same resolution for concatenation. Third, the concatenated features are fused with another MLP layer to produce a single-channel feature map, which is then converted to a binary segmentation using the sigmoid function given a predefined threshold (i.e., 0.5).

[0016] Preferably, the local refinement module decomposes one heavy inference on the whole image into two light predictions on small patches; first, select the image patch around the target object, adjust it to small scale, and send it to the Backbone to predict the coarse mask; then, select a local region around the clicks, which needs to be refined, and feed the enlarged local patch into the local refinement module; finally, the progressive merge aligns the local predictions back to the full-size mask, and only a small local region is refined after each click, and all pixels of the final prediction are refined by assigning them to different rounds of computation.

[0017] Preferably, the target crop is used to filter out background information irrelevant to the target object; first, calculate the minimum outer box of the previous mask and the newly added clicks, and then expand it at a ratio r TC = 1.4, then crop the input tensor and adjust the scale size;

[0018] The focus crop is used to locate the region that the user intends to modify; first, compare the difference between the original segmentation result and the previous mask to obtain the difference mask Mxor, then calculate the maximum connected region of Mxor containing the new clicks, and generate an outer box for this maximum connected region, and expand it at a ratio r FC = 1.4; crop the local patch on the input image and click map;

[0019] The refiner is used to restore the details of the coarse prediction in the focus crop; first, use depth separable convolution to extract low-level features from the cropped tensor; at the same time, adjust the channel number of the region features and fuse them with the extracted low-level features, use two heads to predict the detail map M d and the boundary map M b , and calculate the refined prediction Mr by updating the boundary region of the coarse prediction M l .

[0020] The second aspect of the present application provides an interactive image segmentation method based on clicks, which uses the interactive image segmentation system as described in the first aspect, and includes the following processes:

[0021] First, determine the user clicks, the user clicks on the image to indicate the model to focus on the object, the first click determines the center of the segmentation target, and the subsequent clicks are in the center of the maximum error region;

[0022] Then encode the clicks, encode the clicks using disk encoding and form an interaction graph, represented as a tensor;

[0023] The interaction graph and the original image are input into a self-attention feature extraction module and the global prompt module is used to enhance the self-attention to extract the global information of the image and the interaction graph;

[0024] Then, the last feature map is processed by using a nonlinear segmentation head module, and up-sampling is performed to generate a coarse segmentation mask;

[0025] Meanwhile, the local region to be optimized is identified through a local correction module, and a lightweight network is used to generate a local fine segmentation mask.

[0026] The whole system is iteratively optimized, and the segmentation mask is formed through user clicks, and the segmentation mask accuracy is gradually improved through iterative optimization.

[0027] Compared with the prior art, the present application has the following beneficial effects:

[0028] The present application proposes an innovative Glclick framework based on the SimpleClick model, which significantly improves the boundary perception ability of interactive segmentation through the collaborative architecture of the global prompt module (GHM) and the local correction module (LCM), and explores the deep integration mechanism of global semantic guidance and local feature correction. The technical architecture of the present application comprises two core components: the global prompt module encodes the user click information into the Transformer backbone network, realizing the cross-modal fusion of interactive signals and visual features; and the local correction module optimizes the initial segmentation mask through a multi-scale feature optimization mechanism, and reconstructs the boundary in detail, so that the model has pixel-level precision control ability while maintaining efficient interaction.

[0029] The Glclick model of the present application exhibits excellent performance in complex scene applications such as remote sensing image analysis and medical image processing, and can still generate high-quality segmentation results through a small amount of interactive clicks even in the face of challenges such as target edge blur, multi-scale structure interweaving and texture background mixing. The technical breakthrough is that, on the basis of retaining the lightweight advantage of the classic interactive segmentation framework, through the global-local dual optimization mechanism, the model maintains the cross-domain generalization ability and realizes multi-level feature fusion of interactive information without significantly increasing the computational load. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to make the technical solutions of the present application or the prior art clearer, the drawings needed to be used in the embodiments or the prior art description will be briefly described below. Obviously, the following description is only one embodiment of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0031] Figure 1 The overall structure block diagram of the interactive image segmentation model of the present application.

[0032] Figure 2 The structure diagram of the local correction module.

[0033] Figure 3 The visualization experimental results of the embodiment of the present application;

[0034] Figure 4 The convergence analysis of the model trained by the embodiment of the present application on the SBD dataset

[0035] Figure 5 The convergence analysis of the model trained by the embodiment of the present application on the DAVIS dataset. DETAILED DESCRIPTION

[0036] The application will be further described below in combination with specific embodiments.

[0037] The purpose of the click-based interactive image segmentation is to obtain a pixel-level segmentation mask with only a small number of manual clicks. This method simplifies the pixel-level annotation and image editing process. Many studies have focused on this field. In particular, SimpleClick, which utilizes a visual transformer, realizes a simple and direct design and proves its effectiveness in interactive image segmentation. However, there are still two problems: first, this simple design does not fully utilize the global guidance provided by the interaction map. Second, all clicks are treated equally, and the guiding role of new clicks cannot be maximized in each iteration.

[0038] The present application proposes a network model GlClick that integrates global hint information and local detail correction, as shown in Figure 1 The present application solves the inherent limitations of click-based interactive image segmentation when using traditional visual transformers. Initially, GlClick contains a global hint module (GHM) that enhances the model's ability to more effectively utilize guiding clues from the interaction map. Subsequently, after obtaining the segmentation mask, the system carefully evaluates the differences between the generated mask and the previously generated mask, and then performs targeted local optimization to minimize these differences.

[0039] As shown in Figure 1As shown, first, the user can suggest the target of interest to the segmentation model by clicking. In the first interaction, the model obtains the center point of the segmentation target according to the ground truth, thereby guiding the model to perform the first segmentation; in the subsequent interaction, the user compares the ground truth and the segmentation result of the last round, and performs the next click at the center of the largest error area. Second, the labeled clicks are converted into a click map S ∈ R H×W×2 Each click is represented as a small radius disk. Finally, S and the segmentation result P of the last round are spliced at the channel level to obtain the interaction map M ∈ R H×W×3 In the first interaction, P is initialized as a full zero matrix.

[0040] The present application follows the SimpleClick to build the network architecture, and uses the standard ViT-B as the backbone network. Two patch embedding layers respectively segment the input image I and the interaction map M into fixed size (16x16) non-overlapping patches. Each image patch is projected into a fixed dimension (768) vector through linear transformation. The vector sequence generated from the image and the interaction map is element-wise added to obtain the final vector sequence. The sequence is then input into a series of Transformer blocks for self-attention processing. In the self-attention stage, GlClick superimposes the vector sequence enhanced by the global information extracted by the global hints module (Global Hints Module) into the self-attention mechanism. Since the last feature map has been processed by all attention blocks, it is considered to have the most comprehensive representation ability. Therefore, the present application only uses this last feature map to build a simple multi-scale feature pyramid. The pyramid is then optimized by an all-MLP segmentation head (All-MLP Segmentation Head) and generates a segmentation mask through upsampling. Finally, the local correction module (Local Correction Module) uses the image, the interaction map, the segmentation mask and the previous segmentation mask for local enhancement to achieve fine adjustment.

[0041] Regarding the global hints module GHM:

[0042] To fully utilize the global contextual information in the interaction graph and seamlessly integrate it with the traditional backbone network architecture, this invention develops a Global Hints Module (GHM). User interactions are encoded as a two-channel disk graph, with one channel for positive clicks (representing foreground) and the other for negative clicks (representing background). The existing segmentation mask is merged with the two-channel click graph to generate a three-channel interaction graph. GHM processes the graph by dividing it into non-overlapping blocks of uniform size (16×16 pixels). Each block is flattened and mapped linearly into a 768-dimensional fixed-length vector space. This projection process is repeated three times, and the resulting three sets of vectors are superimposed on the outputs of the Transformer blocks at layers 1, 6, and 12 of the backbone network, respectively, to enhance the effect of the self-attention mechanism.

[0043] Regarding the Local Correction Module (LCM):

[0044] To achieve better local optimization, such as Figure 2 As shown, the Local Modification Module (LCM) first needs to determine the local regions that need refinement. First, this invention compares the current segmentation result with the segmentation mask from the previous round to obtain a difference mask Mxor. Then, it calculates the largest connected region containing the new click in the Mxor and generates the bounding box of this region. This invention expands this region by a scale of r = 1.4. Observing that this region is the core region for focused cropping, it performs local cropping on the input image I to obtain I', and cropping on the interaction image M to obtain M'.

[0045] To maintain the network's lightweight and simplicity, this invention does not use a complex model to extract more features from I' and M'. Instead, it uses three convolutional layers for basic feature extraction to obtain feature Fs. However, to capture richer features, this invention utilizes the feature pyramid output by the backbone network. By aligning the scale and upsampling, more informative features are obtained. These features are cropped using RoiAlign to obtain Fc. Then, Fc and Fs are scale-aligned and element-wise added to generate a local mask. The ground truth is cropped using the same bounding box, and the loss L between the cropped ground truth and the local mask is calculated. loss Global loss G loss and local loss L loss Both use the binary cross-entropy loss function.

[0046] Specific embodiments based on the above interactive image segmentation system are as follows:

[0047] User Clicks: Users click on the image to indicate the object of interest for the model. On the first click, the model determines the center of the segmentation target based on ground truth to guide the first segmentation. In subsequent interactions, users compare the ground truth and the previous segmentation result and click on the center of the area with the largest error for the next click.

[0048] Click Encoding: The labeled clicks are converted into a click map S using a disc encoding method, represented as a tensor of R^HxWx2, where each click is represented by a small radius disc.

[0049] Global Hints Module (GHM): This module combines the interaction map and the previous segmentation result into a three-channel interaction map M and divides it into non-overlapping blocks of 16x16 pixels. Each block is flattened and linearly projected into a 768-dimensional vector space, then interleaved with the outputs of the first, sixth, and twelfth transformation layers in the backbone network to enhance the self-attention mechanism.

[0050] ViT-B Backbone Network: A standard ViT-B is used as the backbone network to extract global information from the image and interaction map.

[0051] All-MLP Segmentation Head: The last feature map is processed using an All-MLP segmentation head and up-sampled to generate the segmentation mask.

[0052] Local Corrections Module (LCM): This module first identifies the local area that needs to be optimized. It obtains the difference mask Mxor by comparing the segmentation result and the previous mask, and calculates the maximum connected region containing the new click, generates an external bounding box for it, and enlarges the region in proportion. Then, LCM crops the local patch from the input image I and the interaction map M, and uses three convolutional layers to extract basic features, obtaining features Fs. To capture more rich features, LCM also utilizes the feature pyramid output by the backbone network and uses RoiAlign for cropping, obtaining features Fc. Finally, Fc and Fs are summed element-wise to generate a local mask.

[0053] Iterative Optimization: Glclick improves the accuracy of the segmentation mask by iteratively optimizing user clicks and network. In each iteration, the model performs local optimization based on user clicks and the previous segmentation result to reduce errors and improve segmentation accuracy.

[0054] Experimental results show that:

[0055] Table 1 Fine-grained experimental results on four general-purpose datasets

[0056]

[0057] Method: method name;

[0058] Berkeley, DAVIS, COCO_MVal, SBD are all dataset names;

[0059] NoC85, NoC90, NoC95: the number of clicks needed to achieve 85%, 90%, 95% average intersection over union;

[0060] †: indicates training on the SBD dataset;

[0061] ‡: indicates training on the COCO_LVIM dataset;

[0062] As shown in Table 1, the number of clicks needed to achieve 85%, 90%, and 95% intersection over union of the method of the present application on the two training sets is lower than that of the previous method.

[0063] As shown in Figure 3 , the figure is divided into two columns, the left is a simple image and the right is a more detailed image, and it can be clearly seen that for the same number of clicks, the intersection over union of both types of images is higher than that of the Simple-Click model.

[0064] In addition, as shown in Figure 4 and Figure 5 , the convergence analysis of the model trained on the DAVIS dataset and the convergence analysis of the model trained on the SBD dataset, the horizontal coordinate is the number of clicks, and the vertical coordinate is the average intersection over union, Figure 4 and Figure 5 demonstrate the superiority and convergence of the method of the present application. Since the method of the present application and the SOTA method SimpleClick model share the same backbone and training dataset, a detailed comparison is made with the SimpleClick model to prove the superiority of the method of the present application. The solid yellow line represents the SimpleClick model and the solid blue line represents the method of the present application. From Figure 4 and Figure 5 it can be seen that the method of the present application is significantly superior to the SimpleClick model in terms of convergence speed and final accuracy.

[0065] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0066] The above describes the specific embodiments of the present application, but is not a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications or variations made on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A click-based interactive image segmentation system, characterized by: The global hint module, the self-attention feature extraction module, the feature pyramid sampling, the nonlinear segmentation head and the local correction module are included. The global hint module introduces a patch embedding layer symmetrical to the patch embedding layer in the backbone, encodes the user click information into a disk map and splices the last round of segmentation masks to form a three-channel interaction map, and embeds the three-channel interaction map into the feature extraction network, so that the network can learn the object boundary and region information indicated by the user clicks. The global hint module specifically includes: introducing a patch embedding layer symmetrical to the patch embedding layer in the backbone, and then performing element-wise feature addition; the user clicks are encoded in a two-channel disk map, one for positive clicks and the other for negative clicks, and the positive clicks are placed in the foreground while the negative clicks are placed in the background. The segmentation mask and the two-channel click map in the visual transformer are connected into a three-channel map for patch embedding, the two symmetrical embedding layers operate on the image and the connected three-channel map respectively, the input is patched, flattened and projected into two vector sequences of the same dimension, and then element-wise addition is performed before being input into the self-attention feature extraction module. The self-attention feature extraction module uses ViT as the backbone network and adopts a single-scale feature map structure to avoid information loss caused by feature map downsampling, learn local detail information of the image, and use Transformer to learn global context information of the image. The feature pyramid sampling part uses the last feature map to construct a multi-scale feature map based on the single-scale feature pyramid, so that the network can capture features of objects at different scales. The nonlinear segmentation head module uses an MLP layer for feature transformation and fusion to convert the multi-scale feature map into a segmentation probability map, avoiding complex convolution operations. The local correction module decomposes one heavy inference on the entire image into two light predictions on small blocks, uses historical information to focus on fine processing of the local area of the new clicked area. The local correction module decomposes one heavy inference on the entire image into two light predictions on small blocks; first, the image block patch around the target object is selected, adjusted to a small scale, and sent to the Backbone to predict a coarse mask; then, a local area around the click is selected, which needs to be refined and the enlarged local patch is fed into the local correction module; finally, the progressive merging aligns the local prediction to the full-size mask, only a small local area is refined after each click, and all pixels of the final prediction are refined by assigning different rounds of calculations.

2. A click-based interactive image segmentation system as claimed in claim 1, wherein: The self-attention feature extraction module specifically includes: The traditional ViT is used as the backbone network, and only single-scale feature maps are maintained throughout the process; the patch embedding layer divides the input image into non-overlapping fixed-size blocks, each block is flattened and linearly projected into a fixed-length vector, and the generated vector sequence is fed into the queue of Transformer blocks for self-attention, the backbone is pre-trained on ImageNet-1k as MAEs, some global self-attention blocks are used to assist the non-shifting window attention during the fine-tuning of the pre-trained backbone to higher resolution input, and global prompt information is added, the user's interaction is encoded into a two-channel disk map, one channel is assigned to positive clicks, indicating the foreground, and the other channel is used for negative clicks, indicating the background, and the pre-existing segmentation mask and the two-channel click map are combined to create a three-channel interaction map; the global prompt module processes this map by dividing it into non-overlapping blocks of the same size, then flattens and linearly projects each block into a fixed-length vector space with 768 dimensions, and this projection is done in triplicate, the resulting set of three vectors is superimposed on the outputs of the first, sixth, and twelfth transformation layers to enhance the self-attention mechanism.

3. The click-based interactive image segmentation system of claim 1, wherein: The feature pyramid sampling uses the last feature map output by the self-attention feature extraction module, uses four convolutional layers with different strides to downsample the last feature map to resolutions of 1 / 32, 1 / 16, 1 / 8, and 1 / 4, respectively, to generate multi-scale feature maps, making the network better capture object features at different scales.

4. The click-based interactive image segmentation system of claim 1, wherein: The nonlinear segmentation head uses MLP layers as lightweight segmentation heads, adopts a simple feature pyramid to generate a segmentation probability map with a scale of 1 / 4, and then performs upsampling to restore the original resolution; the MLP segmentation head works in three steps: first, each feature map from the simple feature pyramid is converted to the same channel dimension through an MLP layer; second, all feature maps are upsampled to the same resolution for connection; third, the connected features are fused with another MLP layer to produce a single-channel feature map, and then a sigmoid function is used to obtain a segmentation probability map, which is then converted to a binary segmentation given a predefined threshold.

5. The click-based interactive image segmentation system of claim 1, wherein: Target Crop, to filter out background information that is not relevant to the target object; first compute the minimum outer box of the previous mask and newly added clicks, and after that extend it by a ratio r TC =1.4 scale it, then crop the input tensor and resize it; Focus Crop, to locate the region the user intends to modify; first compare the difference between the original segmentation result and the previous mask to obtain the differential mask Mxor, then calculate the largest connected region of Mxor containing the new click, and generate an external box for this largest connected region, to r FC = 1.4 scale; crop the local patch on the input image and click map; Refiner is used to restore the details of the coarse prediction in focal cropping; first, low-level features are extracted from the cropping tensor using depthwise separable convolution; at the same time, the number of channels of the region features is adjusted, and they are fused with the extracted low-level features, and two heads are used to predict the detail map M d and the boundary map M b , and calculate the refined prediction Mr by updating the boundary region of the coarse prediction log M l .

6. A click-based interactive image segmentation method, characterized in that, The interactive image segmentation system according to any one of claims 1 to 5 is used, and the following processes are included: First, determine the user clicks, the user clicks on the image to indicate the model to focus on the object, the first click determines the center of the segmentation target, and the subsequent clicks are in the center of the maximum error area; Then encode the clicks, encode the clicks using disk encoding and form an interaction map, represented as a tensor; The interaction map and the original image are fed into the self-attention feature extraction module and the global prompt module to enhance the self-attention, extracting global information from the image and the interaction map; Subsequently, the last feature map is processed using the nonlinear segmentation head module, and up-sampling generates a coarse segmentation mask; At the same time, the local correction module identifies the local area that needs to be optimized, and a lightweight network is used to generate a local fine segmentation mask; The whole system iterative optimization, through the user click, form segmentation mask, click again, iterative optimization, gradually improve the segmentation mask accuracy.

Images