Image segmentation method and system based on multi-scale feature enhancement and prototype fusion

Abstract

This application discloses an image segmentation method and system based on multi-scale feature enhancement and prototype fusion, belonging to the field of image processing technology. The method includes: inputting a support image and a query image into a ResNet residual network to extract support features and query features respectively; inputting the support features and query features into a multi-scale feature enhancement module to obtain enhanced support features and enhanced query features; inputting the enhanced support features and enhanced query features into a prototype fusion module to obtain a final mask; and inputting the final mask into a segmentation head to segment the final mask to obtain the segmentation result of the query image. The method of this application, combining a multi-scale feature enhancement module and a prototype fusion module, solves the problem of large intra-class differences and small inter-class differences in semantic segmentation of small sample images.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image segmentation method and system based on multi-scale feature enhancement and prototype fusion. Background Technology

[0002] Image segmentation is the process of extracting the desired regions from an image for subsequent processing such as object recognition and classification.

[0003] Existing image semantic segmentation methods primarily rely on training with a large number of labeled images, utilizing deep neural networks to extract high-level semantic information and achieve pixel-level classification, demonstrating excellent performance in standard tasks. However, these methods face two major challenges: large intra-class variance, where the appearance of the target varies significantly due to the limited number of reference images, making it difficult to form a unified representation; and high inter-class similarity, where different classes have similar appearance features, easily leading to confusion due to background interference in complex scenes. Semantic segmentation of small sample images typically employs meta-learning strategies, calculating prototype features or feature matching through support sets and query sets for classification. However, existing fixed prototype or dense matching strategies still struggle to effectively address the aforementioned challenges, limiting their widespread application in real-world scenarios. Summary of the Invention

[0004] This application provides an image segmentation method and system based on multi-scale feature enhancement and prototype fusion to solve the problems of large intra-class differences, high inter-class similarity, and insufficient data annotation in the semantic segmentation of small sample images in the prior art.

[0005] On the one hand, embodiments of this application provide an image segmentation method based on multi-scale feature enhancement and prototype fusion, including:

[0006] The support image and query image are input into the ResNet residual network to extract support features and query features, respectively.

[0007] The supporting features and query features are input into the multi-scale feature enhancement module, which includes a multi-feature fusion unit and a feature enhancement unit. The multi-feature fusion unit includes a parallel serial convolution branch and a feature selection branch. The serial convolution branch extracts multi-scale spatial information of the supporting features and query features through multiple cascaded convolutional blocks. The feature selection branch performs attention modeling on key regions in the supporting features and query features through spatial partitioning and a feedforward network. The results of multi-scale spatial information and attention modeling are added together to form the supporting fusion feature and the query fusion feature. The feature enhancement unit adaptively adjusts the weights of the supporting fusion feature and the query fusion feature through a channel-level attention control mechanism to obtain enhanced supporting features and enhanced query features, respectively.

[0008] The enhanced support features and enhanced query features are input into the prototype fusion module. The prototype fusion module first calculates the initial prototype based on the enhanced support features, then generates the initial segmentation mask based on the initial prototype and enhanced query features, and then it iterates and updates the initial segmentation mask through multiple stages to obtain the final mask.

[0009] The final mask is input into the segmentation head, which segments the final mask to obtain the segmentation result of the query image.

[0010] On the other hand, embodiments of this application also provide an image segmentation system based on multi-scale feature enhancement and prototype fusion, including:

[0011] Residual network ResNet is used to extract support features and query features from the input support image and query image, respectively;

[0012] The multi-scale feature enhancement module includes a multi-feature fusion unit and a feature enhancement unit. The multi-feature fusion unit comprises a parallel serial convolution branch and a feature selection branch. The serial convolution branch extracts multi-scale spatial information of support features and query features through multiple cascaded convolutional blocks. The feature selection branch performs attention modeling on key regions in support features and query features through spatial partitioning and a feedforward network. The results of multi-scale spatial information and attention modeling are added together to form support fusion features and query fusion features. The feature enhancement unit adaptively adjusts the weights of support fusion features and query fusion features through a channel-level attention control mechanism to obtain enhanced support features and enhanced query features, respectively.

[0013] The prototype fusion module is used to first calculate the initial prototype based on the enhanced support features, then generate the initial segmentation mask based on the initial prototype and the enhanced query features, and then it updates the initial segmentation mask through multiple stages of iterative optimization to obtain the final mask.

[0014] The segmentation head is used to segment the final mask to obtain the segmentation result of the query image.

[0015] The image segmentation method and system based on multi-scale feature enhancement and prototype fusion in this application have the following advantages:

[0016] 1. Enhanced generalization: Dynamic fusion of support sets and query set prototypes can adapt to changes in the target's appearance.

[0017] 2. Higher precision: The multi-scale feature enhancement (MFE) module significantly enhances feature representation and improves the ability to recognize boundary details.

[0018] 3. Significant advantages in actual testing: In PASCAL- 5i (Pattern analysis, statistical modeling, and computational learning) and COCO- 20iOn the (common objects in context) dataset, it achieves mIoU (mean intersection-union ratio) of 73.76% and 51.6% respectively, which is better than existing methods. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is an architecture diagram of the image segmentation method based on multi-scale feature enhancement and prototype fusion provided in the embodiments of this application.

[0021] Figure 2 This is an architecture diagram of the multi-feature fusion unit provided in the embodiments of this application.

[0022] Figure 3 This is an architecture diagram of the feature enhancement unit provided in an embodiment of this application.

[0023] Figure 4 This is an architecture diagram of the prototype fusion module and the segmentation head provided in the embodiments of this application.

[0024] Figure 5 A comparison chart of visualization results provided for embodiments of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] Figure 1-4 This is an architecture diagram of the image segmentation method based on multi-scale feature enhancement and prototype fusion provided in the embodiments of this application. The embodiments of this application provide an image segmentation method based on multi-scale feature enhancement and prototype fusion, including:

[0027] S100 will support images I s and query images I q Input the residual network ResNet to extract support features and query features respectively.

[0028] For example, after acquiring the supporting and query images, preprocessing is performed first, including size normalization and pixel value normalization, to adapt to the input requirements of the model.

[0029] The ResNet residual network consists of a first layer (Layer 0), a second layer (Layer 1), a third layer (Layer 2), and a fourth layer (Layer 3). The first layer (Layer 0) extracts supporting features and query features.

[0030] S110: The supporting features and query features are input into the multi-scale feature enhancement module. The multi-scale feature enhancement module includes a multi-feature fusion unit and a feature enhancement unit. The multi-feature fusion unit includes a parallel serial convolution branch and a feature selection branch. The serial convolution branch extracts multi-scale spatial information of the supporting features and query features through multiple cascaded convolutional blocks. The feature selection branch performs attention modeling on key regions in the supporting features and query features through spatial partitioning and a feedforward network. The results of multi-scale spatial information and attention modeling are added together to form the supporting fusion feature and the query fusion feature. The feature enhancement unit adaptively adjusts the weights of the supporting fusion feature and the query fusion feature through a channel-level attention control mechanism to obtain enhanced supporting features and enhanced query features, respectively.

[0031] For example, the second layer, Layer 1, performs convolution processing on the support fusion feature and query fusion feature output by the multi-feature fusion unit, and inputs the result of the convolution processing into the first feature enhancement unit. The third layer, Layer 2, performs convolution processing on the enhanced support feature and enhanced query feature output by the first feature enhancement unit, and inputs the result of the convolution processing into the second feature enhancement unit. The fourth layer, Layer 3, performs convolution processing on the enhanced support feature and enhanced query feature output by the second feature enhancement unit, and inputs the result of the convolution processing into the prototype fusion module.

[0032] Furthermore, the weights of the first layer (Layer 0) and the second layer (Layer 1) remain frozen, while the third layer (Layer 2) and the fourth layer (Layer 3) undergo convolution processing according to the weights adaptively adjusted by the first and second feature enhancement units, respectively.

[0033] Furthermore, such as Figure 2 As shown, in the sequential convolution branch, a 3×3 convolutional block with three sequentially connected kernels is used to extract multi-scale spatial information; in the feature selection branch, the input features are sequentially subjected to pointwise convolution (PWConv), unfolding, reshaping, and averaging (PWConv). U & R & M Attention modeling of key regions is completed through processing of ) and feedforward network (FFN) processing, activation (Softmax) and selection.

[0034] Specifically, the selection process includes sequentially performing cosine similarity calculation, amplitude limiting, and reshaping the bilinear difference (…). R & I ) and pointwise convolution. In cosine similarity calculation, the activation result is compared with the learnable vector ( Prompt Cosine similarity is calculated, and the result is then subjected to amplitude limiting. Furthermore, the result of the cosine similarity calculation is multiplied pixel by pixel with the result after amplitude limiting, and then remodeled using bilinear interpolation.

[0035] After obtaining the multi-scale spatial information and the results of attention modeling and adding them together, the system will sequentially undergo dropout, batch normalization (BN), and ReLU activation to finally obtain the fused features.

[0036] Furthermore, such as Figure 3 As shown, the feature enhancement unit sequentially performs global average pooling on the input features ( GAP The process involves convolution (Conv), stacking (Stack), mean (Mean), activation (Sigmoid), and expansion (Expand). The result of the expansion process is multiplied pixel-by-pixel with the result of global mean pooling to form the enhanced feature.

[0037] Specifically, in the convolution process, the results of global average pooling will be convolved using convolutional blocks with kernels of 3×3 and 5×5 respectively, and the outputs of the two convolutional blocks will be stacked together.

[0038] The sequential convolutional branch in this application captures multi-scale spatial information, the feature selection branch achieves attention modeling of key regions through spatial partitioning and a feedforward network, and the feature enhancement unit adaptively adjusts feature weights through a channel-level attention control mechanism to highlight task-relevant regions and suppress background interference or non-target information, thereby generating more discriminative feature representations. These two units work synergistically to significantly improve the ability of image features to focus on target regions, providing high-quality input for subsequent prototype construction and mask prediction. Experimental results show that the introduction of this module can effectively improve segmentation accuracy on different baseline models and significantly enhance the overall performance of the few-sample image semantic segmentation system.

[0039] S120, the enhanced support features and enhanced query features are input into the prototype fusion module. The prototype fusion module first calculates the initial prototype based on the enhanced support features, then generates the initial segmentation mask based on the initial prototype and enhanced query features, and then it iterates and updates the initial segmentation mask through multiple stages to obtain the final mask.

[0040] For example, such as Figure 4As shown, the prototype fusion module dynamically optimizes the prototype representation by utilizing the feature correspondence between the supporting image and the query image, thus overcoming the limitations of static prototypes in traditional methods.

[0041] In the embodiments of this application, when calculating the initial prototype and generating the initial segmentation mask, the prototype fusion module first uses the support mask to perform mask average pooling on the enhanced support features. MAP The initial prototypes of the foreground and background are obtained respectively, and then the initial segmentation mask is generated based on the enhanced query features and the cosine similarity between the initial prototypes of the foreground and background.

[0042] The initial prototype is represented as:

[0043]

[0044] in, and These represent the characteristics of the foreground and background, respectively. This indicates that the mask has undergone average pooling. F This represents the features extracted by the feature extraction network. M The mask that indicates the supported image. and These represent the initial prototypes of the foreground and background, respectively. Indicates average treatment. This indicates splicing / merging.

[0045] The initial segmentation mask can be represented as:

[0046]

[0047] in, Indicates the initial segmentation mask. This indicates the calculation of cosine similarity. This indicates enhanced query features.

[0048] Furthermore, during the iterative optimization and update of the initial segmentation mask, a first-stage query prototype is first generated based on the enhanced query features and the initial segmentation mask. The first-stage query prototype is weighted with the initial prototype to obtain a first-stage enhanced prototype. A first-stage segmentation mask is generated based on the cosine similarity between the enhanced query features and the first-stage enhanced prototype. Then, a second-stage query prototype is generated based on the enhanced query features and the first-stage segmentation mask. The initial prototype is weighted with both the first-stage enhanced prototype and the second-stage query prototype to obtain a second-stage enhanced prototype. The first-stage segmentation mask and the second-stage segmentation mask are weighted and combined to generate a second-stage segmentation mask, thereby enhancing the contribution of the most relevant regions. Next, a third-stage query prototype is generated based on the enhanced query features and the second-stage segmentation mask. The initial prototype, the first-stage enhanced prototype, the second-stage enhanced prototype, and the third-stage query prototype are weighted to obtain a third-stage enhanced prototype. A prediction mask is generated based on the cosine similarity between the enhanced query features and the third-stage enhanced prototype. Finally, the first-stage segmentation mask, the second-stage segmentation mask, and the prediction mask are weighted and summed to obtain the final mask.

[0049] The first-stage prototype is represented as:

[0050]

[0051] in, and These represent the first-stage prototypes of the foreground and background, respectively. , , and These are the weights of each item. and The first-stage query prototypes represent the foreground and background, respectively.

[0052] The two-stage prototype is represented as:

[0053]

[0054] in, and The two-stage prototypes represent the foreground and background, respectively. and These are the weights of each item. and A two-stage query prototype representing the foreground and background, respectively.

[0055] The two-stage segmentation mask can be represented as:

[0056]

[0057] in, and These represent the two-stage segmentation mask and the one-stage segmentation mask, respectively. and These are hyperparameters used to control the contributions of supporting features and query features.

[0058] The three-stage prototype is represented as follows:

[0059]

[0060] in, and The two-stage prototypes represent the foreground and background, respectively. and These are the weights of each item. and A two-stage query prototype representing the foreground and background, respectively.

[0061] The final mask is represented as:

[0062]

[0063] in, Indicates the final mask. This represents the prediction mask.

[0064] Through multi-stage iterative optimization of the prototype, the prototype fusion module enhances the ability to distinguish categories and adapts to the specific features of the query image. The dynamic prototype optimization and multi-stage supervised training strategy significantly improve the segmentation performance, especially showing good adaptability and robustness in complex backgrounds and diverse target shapes.

[0065] The hyperparameters and their weights are obtained by training the prototype fusion module. During the training of the prototype fusion module, the cross-entropy loss function is used, expressed as:

[0066]

[0067] in, This represents the loss function used in the training of the prototype fusion module. Represents cross-entropy loss, This represents the true mask of the query image in the training set. This loss function ensures consistent learning throughout the iterative process.

[0068] In training the multi-scale feature enhancement module, the cross-entropy loss function is also used, expressed as:

[0069]

[0070] in, This represents the loss function used by the multi-scale feature enhancement module during training. This represents the prototype generated by the enhanced query features produced by the multi-scale feature enhancement module. This loss function ensures the robustness of the network during feature extraction and prototype update. During training, samples from the support set and query set need to be preprocessed before being input into the model, and the model also needs to have its network parameters (including the weights of convolutional blocks, the parameters of the attention mechanism, and the hyperparameters of the optimizer) initialized before training can begin.

[0071] Therefore, the loss function for model training in this application is expressed as:

[0072]

[0073] This loss function optimizes the model's performance in small sample scenarios by using the supervision of multi-stage predictions and the self-supervised signals of query features, ensuring the consistency and accuracy of segmentation results under complex backgrounds and diverse object appearances.

[0074] S130: Input the final mask into the segmentation head. The segmentation head segments the final mask to obtain the segmentation result of the query image.

[0075] This application also provides an image segmentation system based on multi-scale feature enhancement and prototype fusion, including:

[0076] Residual network ResNet is used to extract support features and query features from the input support image and query image, respectively;

[0077] The multi-scale feature enhancement module includes a multi-feature fusion unit and a feature enhancement unit. The multi-feature fusion unit comprises a parallel serial convolution branch and a feature selection branch. The serial convolution branch extracts multi-scale spatial information of support features and query features through multiple cascaded convolutional blocks. The feature selection branch performs attention modeling on key regions in support features and query features through spatial partitioning and a feedforward network. The results of multi-scale spatial information and attention modeling are added together to form support fusion features and query fusion features. The feature enhancement unit adaptively adjusts the weights of support fusion features and query fusion features through a channel-level attention control mechanism to obtain enhanced support features and enhanced query features, respectively.

[0078] The prototype fusion module is used to first calculate the initial prototype based on the enhanced support features, then generate the initial segmentation mask based on the initial prototype and the enhanced query features, and then it updates the initial segmentation mask through multiple stages of iterative optimization to obtain the final mask.

[0079] The segmentation head is used to segment the final mask to obtain the segmentation result of the query image.

[0080] Experimental verification

[0081] The experiments were conducted using four NVIDIA GeForce GTX 3090 GPUs within the PyTorch framework. During model training, a stochastic gradient descent (SGD) optimizer was used with a momentum of 0.9, a weight decay factor of 5e-4, and an initial learning rate of 0.001. When using ResNet50 as the backbone, each training batch contained 16 support-query image pairs; when using ResNet101, each batch contained 8 image pairs. In PASCAL- 5i and COCO- 20i The evaluation was conducted using a dataset of 5 samples and a ResNet101 backbone network, yielding mIoU scores of 73.76 and 51.6, respectively, as shown in Table 1.

[0082] Table 1. Quantitative comparison between the method in this application and the optimal model.

[0083]

[0084] Among them, Rife (Relevant Intrinsic Feature Enhancement Network), DCP (Divide and Conquer Agent), DRNet (Decoupled Recombination Network), and DBM (Dual-Branch Multi-Level Network) were all the best-performing models for few-shot segmentation in 2024. Rife is mentioned in the paper "X. Bao, J. Qin, S. Sun, X. Wang, and Y. Zheng. Relevant intrinsic feature enhancement network for few-shot semantic segmentation". Proceedings of the AAAI Conference on Artificial Intelligence The DCP was proposed in the paper "C. Lang, G. Cheng, B. Tu, and J. Han. Few-shot segmentation via divide-and-conquerproxies". International Journal of Computer Vision DRNet was proposed in the paper “Z.Chang, X. Gao, N. Li, H. Zhou, and Y. Lu. Drnet: Disentanglement and recombination network for few-shot semantic segmentation”. IEEE Transactions on Circuits and Systems for Video TechnologyThe concept of DBM was proposed in the paper “Y. Chen, R. Jiang, Y. Zheng, B. Sheng, Z.-X. Yang, and E. Wu. Dual branch multi-levelsemantic learning for few-shot segmentation”. IEEE Transactions on Image Processing It was proposed in 2024.

[0085] 1-shot and 5-shot represent the experimental results when the number of labeled samples is 1 and 5, respectively. Fold-0, Fold-1, Fold-2, and Fold-3 represent the dataset being evenly divided into 4 subsets according to the categories. In the 5-shot case, the proposed method outperforms existing methods in the mIoU evaluation metric, demonstrating the superiority of the proposed method.

[0086] like Figure 5 The visualization results shown compare the method of this application with the baseline method. The baseline method does not use the multi-scale feature enhancement module of this application and lacks a prototype fusion module of one stage. The rest of the structure is the same as that of this application. Figure 5 The true mask and segmentation results are represented by red areas. The visualization results clearly reflect that the method proposed in this application has better performance in both target segmentation and background consistency.

[0087] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0088] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An image segmentation method based on multi-scale feature enhancement and prototype fusion, characterized in that, The method comprises the steps of: inputting a support image and a query image into a residual network Resnet to extract support features and query features, respectively; inputting the support features and the query features into a multi-scale feature enhancement module, the multi-scale feature enhancement module comprising a multi-feature fusion unit and a feature enhancement unit, the multi-feature fusion unit comprising a parallel serial convolution branch and a feature selection branch, the serial convolution branch extracting multi-scale spatial information of the support features and the query features through a plurality of serial convolution blocks, the feature selection branch modeling attention of key regions in the support features and the query features through spatial block division and a feedforward network, and the multi-scale spatial information and the attention modeling results being added to form support fusion features and query fusion features; the feature enhancement unit adaptively adjusting weights of the support fusion features and the query fusion features through a channel-level attention regulation mechanism to obtain enhanced support features and enhanced query features, respectively; inputting the enhanced support features and the enhanced query features into a prototype fusion module, the prototype fusion module first calculating an initial prototype according to the enhanced support features, then generating an initial segmentation mask according to the initial prototype and the enhanced query features, and then updating the initial segmentation mask through iterative optimization in multiple stages to obtain a final mask; inputting the final mask into a segmentation head, the segmentation head segmenting the final mask to obtain a segmentation result of the query image; wherein, when the initial segmentation mask is updated through iterative optimization, a one-stage query prototype is first generated based on the enhanced query features and the initial segmentation mask, the one-stage query prototype and the initial prototype are weighted to obtain a one-stage enhanced prototype, and a one-stage segmentation mask is generated based on the cosine similarity between the enhanced query features and the one-stage enhanced prototype; then, a two-stage query prototype is generated based on the enhanced query features and the one-stage segmentation mask, the initial prototype, the one-stage enhanced prototype and the two-stage query prototype are weighted to obtain a two-stage enhanced prototype, and a two-stage segmentation mask is generated by weighting and combining the one-stage segmentation mask and the two-stage segmentation mask; then, a three-stage query prototype is generated based on the enhanced query features and the two-stage segmentation mask, the initial prototype, the one-stage enhanced prototype, the two-stage enhanced prototype and the three-stage query prototype are weighted to obtain a three-stage enhanced prototype, and a prediction mask is generated based on the cosine similarity between the enhanced query features and the three-stage enhanced prototype; and the one-stage segmentation mask, the two-stage segmentation mask and the prediction mask are weighted and summed to obtain the final mask.

2. The image segmentation method based on multi-scale feature enhancement and prototype fusion according to claim 1, characterized in that, The residual network Resnet includes a first layer Layer 0, a second layer Layer 1, a third layer Layer 2 and a fourth layer Layer 3, the first layer Layer 0 extracts the support feature and the query feature, the second layer Layer 1 performs convolution processing on the support fusion feature and the query fusion feature output by the multi-feature fusion unit, inputs the convolution processing result into the first feature enhancement unit, the third layer Layer 2 performs convolution processing on the enhanced support feature and the enhanced query feature output by the first feature enhancement unit, inputs the convolution processing result into the second feature enhancement unit, and the fourth layer Layer 3 performs convolution processing on the enhanced support feature and the enhanced query feature output by the second feature enhancement unit, and inputs the convolution processing result into the prototype fusion module.

3. The image segmentation method based on multi-scale feature enhancement and prototype fusion according to claim 2, characterized in that, The weights of the first layer Layer 0 and the second layer Layer 1 remain frozen, and the third layer Layer 2 and the fourth layer Layer 3 perform convolution processing according to the weights of the first feature enhancement unit and the second feature enhancement unit respectively.

4. The image segmentation method based on multi-scale feature enhancement and prototype fusion according to claim 1, characterized in that, In the serial convolution branch, three convolution kernels 3x3 connected in sequence are used to extract the multi-scale spatial information; in the feature selection branch, point-by-point convolution, unfolding and reshaping average processing, feedforward network processing, activation and selection processing are sequentially performed on the input features to complete the attention modeling of the key region.

5. The image segmentation method based on multi-scale feature enhancement and prototype fusion according to claim 4, characterized in that, The selection processing includes sequentially performing cosine similarity calculation, reshaping bilinear interpolation and point-by-point convolution.

6. The image segmentation method based on multi-scale feature enhancement and prototype fusion according to claim 1, characterized in that, The feature enhancement unit sequentially performs global average pooling, convolution, stacking, averaging, activation and expansion on the input features, and the expansion processing result is multiplied pixel by pixel with the result of global average pooling to form the enhanced features.

7. The image segmentation method based on multi-scale feature enhancement and prototype fusion according to claim 1, characterized in that, In calculating the initial prototype and generating the initial segmentation mask, the prototype fusion module first performs mask average pooling on the enhanced support features using the support mask to obtain the initial prototype of the foreground and the background, and then generates the initial segmentation mask based on the cosine similarity between the enhanced query features and the initial prototypes of the foreground and the background.

8. The system for image segmentation based on multi-scale feature enhancement and prototype fusion according to any one of claims 1-7, characterized in that, Comprise: a residual network Resnet for extracting support features and query features from input support images and query images respectively; a multi-scale feature enhancement module, the multi-scale feature enhancement module includes a multi-feature fusion unit and a feature enhancement unit, the multi-feature fusion unit includes a serial convolution branch and a feature selection branch in parallel, the serial convolution branch extracts multi-scale spatial information of the support features and the query features through a plurality of serial convolution blocks, the feature selection branch models attention of key regions in the support features and the query features through spatial block division and feedforward network, and the multi-scale spatial information and the attention modeling result are added to form support fusion features and query fusion features; The feature enhancement unit adaptively adjusts the weights of the support fusion feature and the query fusion feature through a channel-level attention regulation mechanism, and obtains enhanced support features and enhanced query features, respectively; The prototype fusion module is configured to calculate an initial prototype according to the enhanced support features, generate an initial segmentation mask according to the initial prototype and the enhanced query features, and then update the initial segmentation mask through iterative optimization in multiple stages to obtain a final mask; The segmentation head is configured to segment the final mask to obtain a segmentation result of the query image.

Images