Image segmentation system based on multi-scale feature fusion and reverse attention mechanism

Abstract

The application discloses an image segmentation system based on multi-scale feature fusion and reverse attention mechanism, wherein the system comprises: inputting a to-be-segmented image into a trained image segmentation model to obtain an image segmentation result; wherein the model comprises a patch embedding layer, an encoder, a decoder and a final projection layer connected in sequence, a plurality of skip connections are arranged between the encoder and the decoder, and a reverse attention mechanism layer is arranged at each skip connection; the reverse attention mechanism layer dynamically screens features through a prediction guide mask mechanism; in the decoder, an LK-SS2D module is arranged to capture local edges and texture details of a lesion area; a multi-scale affinity feature fusion module is further arranged between the encoder and the final projection layer, adaptive fusion of cross-scale features is realized through feature alignment and affinity weight calculation, and feature expression of the lesion area is enhanced.

Description

Technical Field

[0001] This application relates to the field of medical image segmentation technology, and in particular to an image segmentation system based on multi-scale feature fusion and inverse attention mechanism. Background Technology

[0002] The rapid development of deep learning has driven the widespread application of computer vision technology in the field of medical image segmentation, which is one of the key technologies for assisting clinical diagnosis and improving patient care efficiency. Current mainstream medical image segmentation methods are mainly based on two fundamental architectures: Convolutional Neural Networks (CNNs) and Visual Transformers. However, both architectures have inherent limitations and are difficult to meet the demands of fine-grained, high-precision medical segmentation scenarios.

[0003] For CNN-based methods, the limited local receptive field severely restricts their ability to capture long-range contextual information. Even with improvements using large convolutional kernels, comprehensive feature extraction is difficult to achieve, often failing to adequately represent scattered or irregularly shaped targets (such as lesions and anatomical structures) in medical images. While visual Transformers were originally developed to address the shortcomings of CNNs in long-range modeling, these methods still have significant drawbacks: although innovative models like Data-efficient Image Transformers (DeiT) optimize data efficiency, the quadratic computational complexity of their self-attention mechanism still places a heavy burden on memory and computational resources—a particularly pronounced issue in high-resolution medical image tasks requiring dense pixel-level prediction.

[0004] Gu et al. (2021) proposed the State-Space Model (SSM), which excels at modeling long-range dependencies while maintaining computational complexity linear with the input size. It also supports efficient parallel training, significantly reducing the model's parameter overhead. Modern SSM variants, such as the Linear State-Space Layer (LSSL) and the Structured State-Space Sequence Model (S4), evolved from the classic state-space framework and have demonstrated excellent capabilities in capturing long-range feature associations. Building on previous work, Gu et al. introduced S6, a variant incorporating selective scanning, which has been applied to neural networks in the form of Mamba networks (Gu & Dao, 2023). SSM-based Mamba models achieve linear computational complexity while maintaining strong long-range dependencies. Compared to Transformer-based methods, the MAMBA method commonly used in U-shaped networks reduces computational complexity.

[0005] However, despite the fact that SSM-based visual models have demonstrated superior global modeling potential compared to traditional CNNs and Transformers in medical image segmentation in recent years, segmentation networks relying solely on SSM still have significant limitations. On the one hand, SSM excels at modeling long-range spatial dependencies but lacks the ability to characterize local textures, boundary details, and fine-grained structural changes. On the other hand, its interaction and utilization of channel-dimensional information are still insufficient, and this type of information is particularly crucial for distinguishing subtle differences in color, texture, and shape in medical images, especially in segmentation tasks with complex boundaries and low contrast, such as melanoma. This problem further limits the model's discriminative ability.

[0006] As a typical pure SSM-driven U-shaped segmentation framework, VM-UNet introduces varying numbers of VSS blocks in both the encoder and decoder, achieving relatively robust segmentation performance while retaining the advantages of global modeling. However, VM-UNet still inherits several key shortcomings of the pure SSM structure in medical image segmentation. First, its skip connections employ a direct fusion of encoder and decoder features, lacking effective feature selection and guidance mechanisms. This easily leads to redundant background responses being passed to the decoder, weakening the discriminative power of lesion regions and boundary features. Second, the VSS blocks themselves focus more on global dependency modeling, limiting their ability to represent local textures, fine-grained boundaries, and targets with significant scale changes. Therefore, when facing lesions with blurred boundaries, small targets, or irregular shapes, they are prone to detail loss and inaccurate segmentation. Third, the lack of sufficient alignment and adaptive fusion mechanisms between features at different levels makes it difficult for shallow detail information and deep semantic information to achieve efficient collaboration, thus limiting the model's overall representation ability of complex lesion regions. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this application provides an image segmentation system based on multi-scale feature fusion and reverse attention mechanism; On the one hand, an image segmentation system based on multi-scale feature fusion and reverse attention mechanism is provided, including: The acquisition module is configured to acquire the image to be segmented. The image segmentation module is configured to input the image to be segmented into the trained image segmentation model and obtain the image segmentation result. The trained image segmentation model includes a patch embedding layer, an encoder, a decoder, and a final projection layer connected in sequence. Several skip connections are set between the encoder and the decoder, and each skip connection is equipped with a reverse attention mechanism layer. The reverse attention mechanism layer dynamically filters features through a prediction-guided masking mechanism to suppress background noise and enhance the response of lesion areas and boundaries, thereby improving the discriminative power of feature fusion.

[0008] The above technical solution has the following advantages or beneficial effects: Based on the VM-UNet framework, this application systematically improves the encoder-decoder structure and skip connection mechanism to address key issues in medical image segmentation, such as blurred boundaries, significant scale changes, and strong background interference. While maintaining the global modeling advantages of the SSM model, it introduces multi-module collaborative optimization to improve segmentation performance.

[0009] To alleviate the discriminative insufficiency caused by redundant background features mixed in skip connections, this application introduces a reverse attention mechanism at the skip connection, which dynamically filters encoder features based on the prediction results, thereby enhancing the response of lesion areas and boundaries and suppressing irrelevant background noise. To compensate for the shortcomings of pure SSM structures in modeling local details and multi-scale spatial dependencies, this application designs a multi-branch long kernel convolution module in the decoding stage. By combining standard convolution and asymmetric long kernel convolution, it effectively captures short-range texture information and medium-to-long-range structural dependencies, thereby improving the representation ability of small targets, blurred boundaries and irregular lesions. Furthermore, to address the limitations of single-scale feature representation and insufficient utilization of complementary information from different levels of features, this application proposes a multi-scale affinity feature fusion module. This module achieves adaptive fusion through cross-layer feature alignment and affinity weighting, thereby reducing redundant interference and enhancing discriminative representations related to lesions. Attached Figure Description

[0010] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0011] Figure 1 This is a schematic diagram of the internal structure of the image segmentation model in Example 1; Figure 2 This is a schematic diagram of the internal structure of the LK-SS2D block module of the decoder in Embodiment 1; Figure 3 This is a schematic diagram of the internal structure of the VSS block module in Example 1; Figure 4 This is the multi-scale asymmetric large kernel convolution branch module of Example 1; Figure 5 This is a schematic diagram of the internal structure of the reverse attention mechanism layer RA in the skip connection enhancement of Example 1; Figure 6This is a schematic diagram of the Multi-Scale Affinity Feature Fusion (MAFF) module structure in Example 1; Figure 7 This is a schematic diagram of the internal structure of the Off Block of the Multi-Scale Affinity Feature Fusion (MAFF) module in Example 1; Figure 8 This is a schematic diagram of the internal structure of the In Block of the Multi-Scale Affinity Feature Fusion (MAFF) module in Example 1. Detailed Implementation

[0012] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0013] Example 1 This embodiment provides an image segmentation system based on multi-scale feature fusion and reverse attention mechanism; Image segmentation systems based on multi-scale feature fusion and reverse attention mechanisms include: The acquisition module is configured to acquire the image to be segmented. The image segmentation module is configured to input the image to be segmented into the trained image segmentation model and obtain the image segmentation result. The trained image segmentation model includes a patch embedding layer, an encoder, a decoder, and a final projection layer connected in sequence. Several skip connections are set between the encoder and the decoder, and each skip connection is equipped with a reverse attention mechanism layer. The reverse attention mechanism layer dynamically filters features through a prediction-guided masking mechanism to suppress background noise and enhance the response of lesion areas and boundaries, thereby improving the discriminative power of feature fusion.

[0014] Furthermore, the system also includes: In the decoder, the LK-SS2D module is set up to capture the local edges and texture details of the lesion area; A multi-scale affinity feature fusion module is also set between the encoder and the final projection layer. Through feature alignment and affinity weight calculation, adaptive fusion of cross-scale features is achieved, which enhances the feature expression of the lesion area.

[0015] Furthermore, the patch embedding layer divides the input image into... The image is then processed by mapping the number of channels of the image to 96, generating an embedded image. Layer normalization is then used to normalize the embedded image, and the normalized image is then input into the encoder for feature extraction. The encoder consists of four coding layers. At the end of the first three coding layers, a patch merging operation is performed to reduce the height and width of the features while increasing the number of channels.

[0016] Similarly, the decoder consists of four decoding layers. At the beginning of the first, second, and third decoding layers, a patch expanding operation is performed to reduce the number of feature channels and increase the height and width of the features. The decoder is then connected to the final projection layer to restore the feature size to match the segmentation target.

[0017] Furthermore, the encoder includes a first coding layer, a second coding layer, a third coding layer, and a fourth coding layer connected in sequence; The first coding layer includes: a first VSS module, a second VSS module, and a first Patch Merging layer connected in sequence; wherein, the input of the first VSS module is the input of the first coding layer, and the output of the first Patch Merging layer is the output of the first coding layer. The second coding layer includes: a third VSS module, a fourth VSS module, and a second merging layer Patch Merging connected in sequence; wherein, the input of the third VSS module is the input of the second coding layer, and the output of the second merging layer Patch Merging is the output of the second coding layer. The third coding layer includes: a fifth VSS module, a sixth VSS module, and a third merging layer, Patch Merging, connected in sequence; wherein, the input of the fifth VSS module is the input of the third coding layer, and the output of the third merging layer, Patch Merging, is the output of the third coding layer. The fourth coding layer includes a seventh VSS module and an eighth VSS module connected in sequence; wherein the input of the seventh VSS module is the input of the fourth coding layer, and the output of the eighth VSS module is the output of the fourth coding layer.

[0018] Furthermore, such as Figure 3 As shown, the internal structure of the first VSS module to the eighth VSS module is consistent. The first VSS module includes: After layer normalization, the input is divided into two branches: The first branch is the first activation function layer, SiLU; In the second branch, the input is processed sequentially through the first depthwise separable convolutional layer DWConv and the second activation function layer SiLU, and then fed into the 2D Selective Scan (SS2D) module for further feature extraction. After that, the extracted features are performed element-wise with the output of the first branch to fuse the two paths. Finally, the fused features are processed using the LayerNorm module, and the processing results are further processed through a linear layer to obtain the output value of the first VSS module.

[0019] Furthermore, the two-dimensional selective scanning SS2D module includes: scan extension operation, Mamba's S6 module, and scan merging operation; The scan expansion operation unfolds the input image into a sequence along four different directions (top left to bottom right, bottom right to top left, top right to bottom left, bottom left to top right); The resulting sequence is then subjected to feature extraction by the S6 module to ensure a comprehensive scan of information from all directions, thereby capturing diverse features; Subsequently, the scan-merge operation sums and merges the sequences in four directions to obtain two-dimensional features.

[0020] The S6 module, derived from Mamba, introduces a selective mechanism on top of S4. By adjusting the parameters of the SSM based on the input, the model can distinguish and retain relevant information while filtering out irrelevant information.

[0021] Furthermore, the decoder includes a fourth decoding layer, a second adder, a third decoding layer, a first adder, a second decoding layer, and a first decoding layer connected in sequence; The fourth decoding layer includes: a first LK-SS2D module and a second LK-SS2D module connected in sequence; wherein, the input terminal of the first LK-SS2D module is the input terminal of the fourth decoding layer, and the output terminal of the second LK-SS2D module is the output terminal of the fourth decoding layer; The third decoding layer includes: a first patch expansion operation layer, a third LK-SS2D module, and a fourth LK-SS2D module connected in sequence; wherein, the input of the first patch expansion operation layer is the input of the third decoding layer, and the output of the fourth LK-SS2D module is the output of the third decoding layer. The second decoding layer includes: a second patch expansion operation layer, a fifth LK-SS2D module, and a sixth LK-SS2D module connected in sequence; wherein, the input of the second patch expansion operation layer is the input of the second decoding layer, and the output of the sixth LK-SS2D module is the output of the second decoding layer; The first decoding layer includes: a third patch expansion operation layer, a seventh LK-SS2D module, and an eighth LK-SS2D module connected in sequence; wherein, the input of the third patch expansion operation layer is the input of the first decoding layer, and the output of the eighth LK-SS2D module is the output of the first decoding layer.

[0022] Furthermore, the internal working processes of the first patch expanding operation layer, the second patch expanding operation layer, and the third patch expanding operation layer are consistent, all used to perform upsampling and channel adjustment operations on the input features; the first patch expanding operation layer specifically includes: First, the channel dimension of the input features is expanded through linear mapping; Then, a feature rearrangement operation is used to rearrange the feature information in the channel dimension to the spatial dimension, so that the height and width of the output feature are doubled to the input feature, and the number of channels of the output feature is adjusted to half the number of channels of the input feature. Subsequently, the rearranged features are subjected to layer normalization to obtain the patched expanded features, which are then input into the corresponding LK-SS2D module for feature extraction.

[0023] Through the patching and expansion operations described above, the decoder can restore the spatial resolution of the feature map step by step, and make the decoded features consistent with the skip connection features output by the corresponding coding layer in terms of spatial size and channel dimension, so as to achieve feature fusion.

[0024] Furthermore, such as Figure 2 As shown, the internal structures of the first LK-SS2D module to the eighth LK-SS2D module are consistent. The first LK-SS2D module includes: The input terminal of the first LK-SS2D module is connected to the input terminal of the first layer normalization module. The output terminal of the first layer normalization module is connected to the input terminal of the ninth VSS module and the input terminal of the ana module. The output terminals of the ninth VSS module and the ana module are both connected to the input terminal of the splicing unit Concat. The output terminal of the splicing unit Concat is connected to the input terminal of the fusion unit Fusion. The output terminal of the fusion unit Fusion is connected to the input terminal of the third adder. The output terminal of the third adder is the output terminal of the first LK-SS2D module, and the input terminal of the third adder is also connected to the input terminal of the first LK-SS2D module.

[0025] The internal structure of the ninth VSS module is consistent with that of the first VSS module. The concatenation unit (Concat) includes concatenated concatenation; the fusion unit (Fusion) includes sequentially connected convolutional layers, normalization layers, and activation function layers. The ana module is a multi-scale asymmetric large-kernel convolutional module used to extract local texture, edge contours, and structural information at different scales from the input features.

[0026] like Figure 4 As shown, the ana module includes: a depthwise separable convolutional layer with a 5×5 kernel, the input of which is the input of the ana module, and the output of which is connected to the input of the first asymmetric large kernel convolutional branch, the input of the second asymmetric large kernel convolutional branch, and the input of the third asymmetric large kernel convolutional branch, respectively. The first asymmetric large kernel convolution branch includes: a depthwise separable convolutional layer with a 1×7 kernel and a depthwise separable convolutional layer with a 7×1 kernel connected in sequence; the input end of the depthwise separable convolutional layer with a 1×7 kernel is the input end of the first asymmetric large kernel branch, and the output end of the depthwise separable convolutional layer with a 7×1 kernel is the output end of the first asymmetric large kernel convolution branch. The second asymmetric large kernel convolution branch includes: a depthwise separable convolutional layer with a 1×9 kernel and a depthwise separable convolutional layer with a 9×1 kernel connected in sequence; the input end of the depthwise separable convolutional layer with a 1×9 kernel is the input end of the second asymmetric large kernel convolution branch, and the output end of the depthwise separable convolutional layer with a 9×1 kernel is the output end of the second asymmetric large kernel convolution branch; The third asymmetric large kernel convolution branch includes: a depthwise separable convolutional layer with a 1×11 kernel and a depthwise separable convolutional layer with an 11×1 kernel connected in sequence; the input end of the depthwise separable convolutional layer with a 1×11 kernel is the input end of the third asymmetric large kernel convolution branch, and the output end of the depthwise separable convolutional layer with an 11×1 kernel is the output end of the third asymmetric large kernel convolution branch. The outputs of the first, second, and third asymmetric large kernel convolution branches are all connected to the input of the fourth adder. The output of the fourth adder is connected to the input of the multiplier, and the output of the multiplier is the output of the ana module. The input of the fourth adder is also connected to the output of a 5×5 convolutional layer, and the input of the multiplier is also connected to the input of a 5×5 convolutional layer.

[0027] Three asymmetric large kernel convolution branches are used to extract local texture, edge contour and structural features of lesion areas within different receptive fields.

[0028] Furthermore, the internal structures of the first LK-SS2D module to the eighth LK-SS2D module are consistent. The calculation process of the first LK-SS2D module can be expressed as follows:

[0029]

[0030]

[0031]

[0032] in, This describes the overall feature mapping process of the VSS module. The VSS module first extracts local basic features through linear mapping, depthwise separable convolution, and the SiLU activation function. Then, it uses a two-dimensional selective scanning core operation to scan the features along four spatial directions and perform selective state space processing. Finally, the processed features are multiplied element-wise with the output of the gated branch to obtain features containing long-distance spatial dependencies and global contextual information. .

[0033] This describes the overall feature mapping process of the ana module. The ana module first extracts local neighborhood features through depthwise separable convolutions. Then, it employs three asymmetric large-kernel convolution branches, consisting of 1×7 and 7×1, 1×9 and 9×1, and 1×11 and 11×1 depthwise convolutions, to extract local texture, edge contours, and structural information within different receptive fields. Features at different scales are then element-wise summed and processed through 1×1 convolutions to generate structure-aware weights. These structure-aware weights are then used to element-wise modulate the input features to obtain local structural features. .

[0034] Indicates channel-dimensional splicing. This represents a 1×1 convolution operation used for channel blending and dimension adjustment. This represents the fusion mapping consisting of batch normalization operations and SiLU activation operations in sequence, i.e. .

[0035] The fusion mapping is used to normalize and nonlinearly fuse the global context features output by the VSS module and the local structural features output by the ana module to obtain fused features. .

[0036] Lesion morphology in medical images typically exhibits significant multi-scale and irregular features: on the one hand, its overall contour requires a large receptive field for modeling; on the other hand, local edges and texture details rely on fine-grained local operators for capture. A single convolutional scale or a single modeling path often struggles to simultaneously address both of these needs.

[0037] Therefore, this application reconstructs the original decoder and designs a decoding module with both global and local structure modeling capabilities, divided into two branches: the SS2D branch, which inherits the state-space modeling capability of VM-UNet to capture long-distance spatial dependencies, thereby preserving global contextual information in deep features; and the Large-kernel convolution branch, which corresponds to the ana_block, such as... Figure 4 The multi-scale asymmetric large kernel convolution branch module ana.

[0038] Furthermore, for the ana module, let the normalized feature of the input ana module be... First, a depthwise separable convolution with a kernel size of 5×5 is used to aggregate initial local information from the input features, thus obtaining basic local features. The calculation process is expressed as follows:

[0039] in, This indicates a depthwise separable convolution operation with a kernel size of 5×5.

[0040] Subsequently, the basic local features Input three parallel asymmetric large kernel convolution branches respectively.

[0041] The first asymmetric large-kernel convolution branch uses depthwise separable convolutions with kernel sizes of 1×7 and 7×1 in sequence to extract texture and local edge features within the first distance range. The calculation process is expressed as follows:

[0042] The second asymmetric large-kernel convolution branch uses depthwise separable convolutions with kernel sizes of 1×9 and 9×1 respectively to extract contour and structural features within the second distance range. The calculation process is expressed as follows:

[0043] The third asymmetric large-kernel convolution branch employs depthwise separable convolutions with kernel sizes of 1×11 and 11×1, respectively, to extract contour and structural features within the third distance range. The calculation process is expressed as follows:

[0044] The basic local features Features output by three asymmetric large kernel convolution branches and Element-wise addition is performed, and feature mixing is completed through 1×1 convolution to obtain multi-scale structural response features. :

[0045] Utilizing the multi-scale structural response characteristics Normalized input features Element-wise modulation is performed to obtain the output features of the large kernel convolution branch:

[0046] Therefore, the overall calculation process of the ana module can be represented as follows:

[0047] in, and These represent features obtained using asymmetric large kernel convolutions with scales of 7, 9, and 11, respectively. and These represent the strip depthwise convolution operations in the horizontal and vertical directions, respectively; This represents an 11×1 convolution operation; This represents element-wise multiplication; This indicates the overall mapping process of the ana module.

[0048] Compared to directly using large-size 2D square convolutions of 7×7, 9×9, and 11×11, this embodiment decomposes them into concatenated forms of horizontal strip convolutions and vertical strip convolutions. While maintaining a large effective receptive field, it can reduce the number of convolution parameters and computational cost, and extract local texture, contour information, and long-distance edge structures of lesion regions from different scales.

[0049] After the outputs of the two branches, they are fused by channel concatenation and 1×1 convolution, then feature integration is completed by BatchNorm and SiLU activation, and finally the module output is obtained by residual connection.

[0050] This dual-branch design offers two significant advantages. First, the SS2D branch maintains the network's global modeling capability, enabling the decoder to go beyond a simple feature restorer and continue integrating semantic context. Second, the long-kernel convolution branch explicitly enhances the perception of lesion contours and edge structures, making it particularly suitable for handling morphologically complex targets such as skin lesions and organ boundaries. Through branch parallelism and lightweight deep convolution design, the module improves representational capabilities without introducing an excessive parameter burden.

[0051] Furthermore, the output of the second coding layer is connected to the input of the first reverse attention mechanism layer, and the output of the first reverse attention mechanism layer is connected to the input of the first adder. The output of the third coding layer is connected to the input of the second reverse attention mechanism layer, and the output of the second reverse attention mechanism layer is connected to the input of the second adder. The output of the fourth coding layer is connected to the input of the third reverse attention mechanism layer, and the output of the third reverse attention mechanism layer is connected to the input of the fourth decoding layer.

[0052] In traditional U-Net-like models, skip connections typically add or concatenate encoder and decoder features directly. While this approach can compensate for the loss of detail during decoding, it also easily transmits a large amount of background noise to the decoder. This is especially problematic in medical image scenarios, where lesion areas are often small with fine edges, while background areas constitute a large proportion of the image, making direct fusion prone to missegmentation. To address this issue, this application introduces a Reverse Attention mechanism layer at the skip connection location. The core idea is to generate a reverse attention mask using the prediction results from the current decoding stage, explicitly filtering encoder features to suppress background responses and enhance lesion and edge regions.

[0053] Furthermore, such as Figure 5 As shown, the internal structures of the first reverse attention mechanism layer, the second reverse attention mechanism layer, and the third reverse attention mechanism layer are identical. The first reverse attention mechanism layer includes: Let the first The prediction map output by each decoding layer is The skip connection feature corresponding to the output of the coding layer is Among them, the prediction map Used to generate inverse attention masks, skip connection features These are the high-level features to be screened.

[0054] First, the prediction graph The spatial dimensions are adjusted to match the spatial dimensions of the corresponding skip connection features to obtain the size-aligned predicted features. :

[0055] in, Indicates an upsampling operation. and They represent the first Output features of each encoding stage Height and width; when predicting the graph Features of skip connections When the spatial dimensions are consistent, the upsampling operation is an identity mapping.

[0056] Subsequently, the size-aligned prediction map is calibrated for response, and the foreground probability response is obtained through the Sigmoid activation function. The foreground probability response is then reverse-processed to obtain the inverse attention mask. :

[0057] in, This represents the Sigmoid activation function. Indicates the first The learnable scaling parameters in the reverse attention mechanism layer are used to adaptively adjust the magnitude of the predicted response. Indicates the relationship with the first The inverse attention mask corresponding to each jump connection.

[0058] In the inverse attention mask, regions with high prediction confidence in the current decoder correspond to larger foreground probability values ​​and receive smaller mask weights after inverse processing; regions with low or uncertain prediction responses receive larger mask weights. Therefore, the inverse attention mask can reduce the response of fully identified regions and relatively highlight lesion boundaries, fine structures, and under-identified regions.

[0059] Due to the reverse attention mask This represents a single-channel spatial weight map, while the encoder output shows skip connection features. Since it contains multiple channels, the inverse attention mask is extended along the channel dimension:

[0060] in, Indicates a channel expansion operation. The extended inverse attention mask represents the number of channels corresponding to the skip connection feature. satisfy:

[0061] Next, the extended inverse attention mask is used to perform element-wise weighting of the skip connection features output by the encoder, thereby achieving prediction-guided feature selection:

[0062] in, This represents element-wise multiplication. This represents the features after reverse attention weighting.

[0063] For locations with smaller mask weights, the corresponding feature responses are suppressed; for locations with larger mask weights, the corresponding boundary features, small target features, and uncertain region features are preserved and enhanced, thereby achieving prediction-guided feature selection.

[0064] Finally, the skip connection features weighted by reverse attention are sequentially input into the first and second convolutional layers for feature mapping, yielding the output of the reverse attention mechanism layer:

[0065] in, and These represent the first and second convolution operations in the reverse attention mechanism layer, respectively. This represents the output features obtained after reverse attention filtering and feature mapping.

[0066] Through the above processing, the inverse attention mechanism layer generates an inverse attention mask using the prediction results of the current decoding stage, dynamically adjusting the response intensity of the encoder's skip connection features at different spatial locations. Regions that have been identified with high confidence are assigned lower weights, while boundary regions, fine structures, and regions with uncertain predictions are assigned higher weights, thereby reducing background noise and redundant features from propagating to the decoder and enhancing the feature representation of lesion edges and difficult-to-distinguish regions.

[0067] Unlike complex self-attention mechanisms, the Reverse Attention module in this application employs a lightweight design, using only two 1×1 convolutions and one learnable scaling parameter to achieve feature mapping and mask correction, resulting in extremely low overhead. The Reverse Attention (RA) layer operates only at the three skip connection fusion positions in the decoding stage, rather than at all layers, thus further controlling complexity while maintaining effectiveness.

[0068] The introduction of the Reverse Attention mechanism layer essentially uses the "prediction result" in reverse for "feature selection," achieving a decoder-driven skip connection enhancement mechanism. This strategy is particularly suitable for medical image segmentation: when the lesion outline is unclear or the boundary region is difficult to distinguish, the Reverse Attention mechanism layer (RA) can reduce invalid texture interference by suppressing the background region, thereby improving the model's ability to recover lesion edges and fine structures.

[0069] Furthermore, the outputs of the first coding layer, the second coding layer, the third coding layer, and the fourth coding layer are all connected to the input of the multi-scale affinity feature fusion module. The output of the multi-scale affinity feature fusion module is connected to the input of the multiplier, and the output of the multiplier is connected to the input of the final projection layer.

[0070] Relying solely on single-scale features is insufficient to fully describe the lesion structure in medical images. Shallow features possess strong detail and edge information but have limited semantic expressive power; deep features offer stronger semantic abstraction but often lose local details. Therefore, it is necessary to achieve complementarity between features at different levels through a multi-scale fusion mechanism. To this end, this application introduces a multi-scale affinity feature fusion module (MAFF) to integrate features from different stages of the encoder across scales and adaptively weight them based on the affinity relationships between features, thereby enhancing the response of the lesion region and suppressing background noise and artifacts, such as... Figure 6 . Figure 7 This is a schematic diagram of the internal structure of the Off Block of the Multi-Scale Affinity Feature Fusion (MAFF) module in Example 1; Figure 8 This is a schematic diagram of the internal structure of the In Block of the Multi-Scale Affinity Feature Fusion (MAFF) module in Example 1.

[0071] Furthermore, such as Figure 6 As shown, the multi-scale affinity feature fusion module includes: Given the output features of each stage of the encoder :

[0072] in, It has the highest spatial resolution, enabling it to retain more boundary, texture, and local detail information. Therefore, this application will As a reference feature, it is used to guide the generation of affinity weights for other levels of features.

[0073] First, to eliminate spatial size differences between features at different encoding stages, bilinear interpolation is used to unify all features to the same size. Same space dimensions:

[0074] in, This represents the bilinear interpolation operation. Indicates reference feature The spatial dimensions. After this operation, all scale features are consistent in spatial dimensions. For the first layer features, this process is equivalent to an identity mapping, i.e. .

[0075] Because the number of channels for features differs at different stages, direct addition would result in a dimensionality mismatch. Therefore, for the ... For each scale feature, the number of channels is first adjusted to match the number of channels of the reference feature using a channel mapping operation. Consistency:

[0076] in, Including sequentially connected Convolutional and normalization layers are used to perform channel alignment between features across scales. After the channel alignment operation, With reference features They have the same spatial dimensions and channel dimensions.

[0077] Subsequently, reference features Features aligned with channels Element-wise summation is performed, and the cross-scale affinity response is obtained through the ReLU activation function:

[0078] This operation is used to establish the response relationship between shallow detail features and deep semantic features. When a region has a strong response in both shallow and deep features, it indicates that the region is more likely to be related to the lesion target or key boundary, and therefore should be given higher weight in the subsequent fusion process.

[0079] Next, the cross-scale affinity response will be discussed. Input the affinity mapping branch to generate a single-channel spatial weight map at the corresponding scale:

[0080] in, Composed of convolution and normalization operations, it is used to convert cross-scale affinity responses. It is mapped to a single-channel spatial response. This represents the Sigmoid activation function, used to constrain the spatial response value to [0,1]. After this operation, the result is... Used to measure the The importance of different spatial locations for segmentation tasks in each scale feature.

[0081] Then, the affinity weight map The feature is expanded to the same channel dimension as the corresponding feature through a broadcast mechanism, and then weighted element-wise for that scale feature.

[0082] in, This represents element-wise multiplication. This represents the multi-scale features after affinity weighting. Through this operation, MAFF can highlight the spatial response associated with the lesion region while attenuating background regions and irrelevant noise.

[0083] Finally, the reference feature F1 and the weighted multi-scale features are compared. The images are stitched together along the channel dimension and then fused using 1×1 convolution to generate a single-channel, multi-scale spatial guidance map.

[0084] in, This indicates a channel splicing operation. This represents a fusion mapping consisting of 1×1 convolutions, normalization, and activation operations, used to integrate the stitched multi-scale features and generate a multi-scale affinity guide map. This represents the final generated multi-scale affinity guide map, which integrates shallow boundary details and deep semantic context information, reflecting the importance of different spatial locations for lesion segmentation.

[0085] This fusion feature is then used to enhance the representational capabilities of the decoding stage, helping the network to fully utilize complementary information from different depths when recovering high-resolution predictions.

[0086] Furthermore, the loss function used during the training of the trained image segmentation model is: To ensure the simplicity and comparability of the training objectives, the BceDice loss, composed of binary cross-entropy loss and Dice loss, is used in the binary classification segmentation task. This application combines the two to simultaneously enhance pixel classification accuracy and region structure consistency, as defined below:

[0087] in, Represents the binary cross-entropy loss. This indicates Dice's loss. and These are used to adjust the contribution ratio of the two parts of the loss, respectively.

[0088] The binary cross-entropy loss is defined as:

[0089] in, This represents the total number of pixels in the image. Indicates the first The real label of each pixel This indicates the probability that the model predicts the pixel belongs to the foreground region.

[0090] Dice loss is defined as follows:

[0091] in, This is a smoothing term used to avoid numerical instability caused by a denominator of zero.

[0092] By jointly optimizing the BCE loss and Dice loss, the model can not only learn pixel-level classification information, but also maintain spatial consistency between the predicted region and the real lesion region, thereby improving the model's ability to segment lesions with blurred boundaries, small targets and irregular shapes.

[0093] Overall, this application uses VM-UNet as the basic framework and enhances the model's ability to collaboratively represent lesion boundaries, local structures, and global context by integrating reverse attention, long kernel convolutional local modeling, and multi-scale affinity feature fusion mechanisms, thereby effectively improving the accuracy and robustness of medical image segmentation.

[0094] To balance the high accuracy and lightweight requirements of medical image segmentation tasks, this application proposes an improved framework for lesion segmentation based on VM-UNet. The overall network still adopts the classic encoder-decoder structure: the input image first undergoes patch embedding to generate initial feature representations, then the encoder progressively expands the receptive field and enhances semantic expression through multi-stage feature extraction and downsampling; the decoder restores spatial resolution through progressive upsampling and combines skip connection information from the encoder to achieve detailed reconstruction. Unlike the original VM-UNet, this application systematically improves the network from three levels: skip connection-enhanced back attention, structure-aware multi-scale decoder, and multi-scale affinity feature fusion. For example... Figure 1 As shown.

[0095] Medical image segmentation is a fundamental task in medical image analysis, playing a crucial role in lesion detection, organ contour extraction, and clinical auxiliary diagnosis. With the development of deep learning, segmentation methods based on convolutional neural networks have made significant progress. Among them, U-Net, with its encoder-decoder structure and skip connection mechanism, has become the most representative foundational framework in the field of medical image segmentation. Researchers have proposed numerous improvements to U-Net, such as UNet++ which reduces the semantic gap between the encoder and decoder through nested skip connections, Attention U-Net which enhances the response of target regions through attention gating, and the ResUNet series which utilizes residual connections to improve feature propagation ability and training stability. Furthermore, methods such as DeepLab further enhance the segmentation capability for complex targets through dilated convolutions and multi-scale context modeling.

[0096] While the aforementioned methods have achieved good results in medical image segmentation, medical images typically exhibit characteristics such as blurred boundaries, large scale variations, and strong background interference. Relying solely on convolutional operations is insufficient to simultaneously address both local detail modeling and global context representation. Therefore, constructing a feature modeling framework that balances accuracy and efficiency remains a crucial research challenge in this field. The feature interaction method directly impacts the performance of medical image segmentation. Traditional CNNs rely on local receptive fields, excelling in edge, texture, and local structure modeling, but have limited ability to characterize long-distance dependencies. To overcome this limitation, Transformers have been gradually introduced into the field of medical image segmentation.

[0097] Representative methods such as TransUNet combine Transformer and U-Net to enhance global context modeling capabilities; Swin-Unet constructs a pure Transformer U-shaped segmentation framework based on hierarchical window attention; and methods such as UTNet and MISSFormer further improve segmentation performance from the perspectives of hybrid modeling and multi-scale interaction. However, Transformer methods are usually accompanied by high computational complexity, which can easily lead to large computational and memory overhead in high-resolution medical images.

[0098] Besides global modeling, skip connection enhancement and multi-scale fusion are also key issues in medical image segmentation. Existing methods typically introduce attention gating, boundary guidance, or prediction guidance mechanisms into skip connections to reduce the interference of shallow noise on the decoder; simultaneously, they achieve cross-scale feature fusion through feature pyramids, dilated convolutions, parallel branching, and attention weighting. Although these methods improve the quality of feature interactions to some extent, they are still insufficient in terms of background suppression, local boundary enhancement, and cross-scale relationship modeling in skip connections.

[0099] In recent years, state-space models have provided a new paradigm for vision tasks, distinct from CNNs and Transformers. Mamba achieves long-distance dependency modeling with low complexity through a selective scanning mechanism, offering advantages in both efficiency and performance. To adapt to 2D vision tasks, related research has proposed visual Mamba models such as ViM and VMamba, which enhance the ability to model 2D spatial dependencies through bidirectional or cross-scanning. Building on this, VM-UNet introduces visual Mamba into a U-shaped structure, validating the application potential of the pure SSM framework in medical image segmentation. Although Vision Mamba excels in efficient global modeling, existing Mamba / VM-UNet-like methods still primarily focus on scanning mechanisms and global dependency modeling, paying insufficient attention to local boundaries, fine-grained textures, and complementary relationships between multi-scale features. Especially in medical image scenarios, lesion edges are blurred, shapes are irregular, and scale differences are significant, making it difficult to fully meet the requirements for fine segmentation relying solely on state-space modeling.

[0100] Based on this, this application uses VM-UNet as a foundation and, while retaining its global modeling advantages, makes improvements in three aspects: skip connection enhancement, decoder local structure modeling, and multi-scale feature fusion. It introduces a reverse attention mechanism layer, a long kernel convolution dual-branch decoding structure, and a multi-scale affinity feature fusion module to further enhance the model's boundary awareness, multi-scale expressive ability, and segmentation robustness.

[0101] This application conducts comprehensive experiments on skin lesion and polyp segmentation tasks to verify the effectiveness and practical value of the improved model in medical image segmentation. Systematic evaluations were performed on the ISIC 2017, ISIC 2018, and CVC-ClinicDB datasets. Experimental results show that the proposed method achieves competitive performance on multiple evaluation metrics, particularly demonstrating significant advantages in lesion boundary delineation and small object detection. Unlike the pure SSM model used only as a baseline, this application's method, while maintaining structural efficiency, significantly improves segmentation accuracy and robustness by introducing a structure optimization module tailored to the characteristics of medical images.

[0102] The main contributions of this application can be summarized as follows: A reverse attention (RA) module is introduced at the skip connection, dynamically filtering features through a "prediction-guided mask" mechanism to suppress background noise and enhance the response of lesion regions and boundaries, significantly improving feature fusion discriminability with extremely low parameter overhead; a multi-branch long kernel convolution module is constructed in the decoding stage, combining 3×3 convolution with multi-scale asymmetric long kernel convolution (11, 9, 7) to effectively capture long, medium, and short distance dependency information, compensating for the shortcomings of pure SSM structures in local detail modeling; a multi-scale affinity feature fusion (MAFF) module is proposed, achieving adaptive fusion of cross-scale features through feature alignment and affinity weight calculation, reducing redundant information and enhancing the expression of lesion-related features.

[0103] The MRA-VMUNet model proposed in this application achieves state-of-the-art performance on three public datasets (ISIC 2017, ISIC 2018, and CVC-ClinicDB), with relative improvements in mIoU of 0.3%, 1.2%, and 1.7%, respectively. It achieves better performance compared to state-of-the-art methods.

[0104] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image segmentation system based on multi-scale feature fusion and reverse attention mechanism, characterized by: include: The acquisition module is configured to acquire the image to be segmented. The image segmentation module is configured to input the image to be segmented into the trained image segmentation model and obtain the image segmentation result. The trained image segmentation model includes a patch embedding layer, an encoder, a decoder, and a final projection layer connected in sequence. Several skip connections are set between the encoder and the decoder, and each skip connection is equipped with a reverse attention mechanism layer. The reverse attention mechanism layer dynamically filters features through a prediction-guided masking mechanism to suppress background noise and enhance the response of lesion areas and boundaries, thereby improving the discriminative power of feature fusion.

2. The image segmentation system based on multi-scale feature fusion and reverse attention mechanism as described in claim 1, characterized in that, The system also includes: In the decoder, the LK-SS2D module is set up to capture the local edges and texture details of the lesion area; A multi-scale affinity feature fusion module is also set between the encoder and the final projection layer. Through feature alignment and affinity weight calculation, adaptive fusion of cross-scale features is achieved, which enhances the feature expression of the lesion area.

3. The image segmentation system based on multi-scale feature fusion and reverse attention mechanism as described in claim 1, characterized in that, The encoder includes a first coding layer, a second coding layer, a third coding layer, and a fourth coding layer connected in sequence; The first coding layer includes: a first VSS module, a second VSS module, and a first merging layer PatchMerging connected in sequence; wherein, the input of the first VSS module is the input of the first coding layer, and the output of the first merging layer PatchMerging is the output of the first coding layer. The second coding layer includes: a third VSS module, a fourth VSS module, and a second merging layer PatchMerging connected in sequence; wherein, the input of the third VSS module is the input of the second coding layer, and the output of the second merging layer PatchMerging is the output of the second coding layer. The third coding layer includes: a fifth VSS module, a sixth VSS module, and a third merging layer PatchMerging connected in sequence; wherein, the input of the fifth VSS module is the input of the third coding layer, and the output of the third merging layer PatchMerging is the output of the third coding layer. The fourth coding layer includes a seventh VSS module and an eighth VSS module connected in sequence; wherein the input of the seventh VSS module is the input of the fourth coding layer, and the output of the eighth VSS module is the output of the fourth coding layer. The decoder includes a fourth decoding layer, a second adder, a third decoding layer, a first adder, a second decoding layer, and a first decoding layer connected in sequence. The fourth decoding layer includes: a first LK-SS2D module and a second LK-SS2D module connected in sequence; wherein, the input terminal of the first LK-SS2D module is the input terminal of the fourth decoding layer, and the output terminal of the second LK-SS2D module is the output terminal of the fourth decoding layer; The third decoding layer includes: a first patch expansion operation layer, a third LK-SS2D module, and a fourth LK-SS2D module connected in sequence; wherein, the input of the first patch expansion operation layer is the input of the third decoding layer, and the output of the fourth LK-SS2D module is the output of the third decoding layer. The second decoding layer includes: a second patch expansion operation layer, a fifth LK-SS2D module, and a sixth LK-SS2D module connected in sequence; wherein, the input of the second patch expansion operation layer is the input of the second decoding layer, and the output of the sixth LK-SS2D module is the output of the second decoding layer; The first decoding layer includes: a third patch expansion operation layer, a seventh LK-SS2D module, and an eighth LK-SS2D module connected in sequence; wherein, the input of the third patch expansion operation layer is the input of the first decoding layer, and the output of the eighth LK-SS2D module is the output of the first decoding layer.

4. The image segmentation system based on multi-scale feature fusion and reverse attention mechanism as described in claim 3, characterized in that, The internal structure of the first LK-SS2D module to the eighth LK-SS2D module is consistent. The first LK-SS2D module includes: The input terminal of the first LK-SS2D module is connected to the input terminal of the first layer normalization module. The output terminal of the first layer normalization module is connected to the input terminal of the ninth VSS module and the input terminal of the ana module. The output terminals of the ninth VSS module and the ana module are both connected to the input terminal of the splicing unit Concat. The output terminal of the splicing unit Concat is connected to the input terminal of the fusion unit Fusion. The output terminal of the fusion unit Fusion is connected to the input terminal of the third adder. The output terminal of the third adder is the output terminal of the first LK-SS2D module, and the input terminal of the third adder is also connected to the input terminal of the first LK-SS2D module.

5. The image segmentation system based on multi-scale feature fusion and reverse attention mechanism as described in claim 4, characterized in that, The ana module includes: a depthwise separable convolutional layer with a 5×5 kernel, wherein the input of the depthwise separable convolutional layer with a 5×5 kernel is the input of the ana module, and the output of the depthwise separable convolutional layer with a 5×5 kernel is connected to the input of the first asymmetric large kernel convolutional branch, the input of the second asymmetric large kernel convolutional branch, and the input of the third asymmetric large kernel convolutional branch, respectively. The first asymmetric large kernel convolution branch includes: a depthwise separable convolutional layer with a 1×7 kernel and a depthwise separable convolutional layer with a 7×1 kernel connected in sequence; the input end of the depthwise separable convolutional layer with a 1×7 kernel is the input end of the first asymmetric large kernel branch, and the output end of the depthwise separable convolutional layer with a 7×1 kernel is the output end of the first asymmetric large kernel convolution branch. The second asymmetric large kernel convolution branch includes: a depthwise separable convolutional layer with a 1×9 kernel and a depthwise separable convolutional layer with a 9×1 kernel connected in sequence; the input end of the depthwise separable convolutional layer with a 1×9 kernel is the input end of the second asymmetric large kernel convolution branch, and the output end of the depthwise separable convolutional layer with a 9×1 kernel is the output end of the second asymmetric large kernel convolution branch; The third asymmetric large kernel convolution branch includes: a depthwise separable convolutional layer with a 1×11 kernel and a depthwise separable convolutional layer with an 11×1 kernel connected in sequence; the input end of the depthwise separable convolutional layer with a 1×11 kernel is the input end of the third asymmetric large kernel convolution branch, and the output end of the depthwise separable convolutional layer with an 11×1 kernel is the output end of the third asymmetric large kernel convolution branch. The outputs of the first, second, and third asymmetric large kernel convolution branches are all connected to the input of the fourth adder. The output of the fourth adder is connected to the input of the multiplier, and the output of the multiplier is the output of the ana module. The input of the fourth adder is also connected to the output of a 5×5 convolutional layer, and the input of the multiplier is also connected to the input of a 5×5 convolutional layer.

6. The image segmentation system based on multi-scale feature fusion and reverse attention mechanism as described in claim 3, characterized in that, The internal structure of the first LK-SS2D module to the eighth LK-SS2D module is consistent. The calculation process of the first LK-SS2D module can be expressed as follows: in, This describes the overall feature mapping process of the VSS module. The VSS module first extracts local basic features through linear mapping, depthwise separable convolution, and the SiLU activation function. Then, it uses a two-dimensional selective scanning core operation to scan the features along four spatial directions and perform selective state space processing. Finally, it multiplies the processed features element-wise with the output of the gated branch to obtain features containing long-distance spatial dependencies and global contextual information. ; This describes the overall feature mapping process of the ana module. The ana module first extracts local neighborhood features through depthwise separable convolutions. Then, it employs three asymmetric large-kernel convolution branches, consisting of 1×7 and 7×1, 1×9 and 9×1, and 1×11 and 11×1 depthwise convolutions, to extract local texture, edge contours, and structural information within different receptive fields. Features at different scales are element-wise summed and then processed through 1×1 convolutions to generate structure-aware weights. These structure-aware weights are then used to element-wise modulate the input features to obtain local structural features. ; Indicates channel-dimensional splicing. This represents a 1×1 convolution operation used for channel blending and dimension adjustment. This represents a fusion mapping consisting of batch normalization operations and SiLU activation operations in sequence. By using fusion mapping, the global context features output by the VSS module and the local structural features output by the ana module are normalized and nonlinearly fused to obtain fused features. .

7. The image segmentation system based on multi-scale feature fusion and reverse attention mechanism as described in claim 6, characterized in that, The ana module, let the normalized feature of the input ana module be... First, a depthwise separable convolution with a kernel size of 5×5 is used to aggregate initial local information from the input features, thus obtaining basic local features. The calculation process is expressed as follows: in, This indicates a depthwise separable convolution operation with a kernel size of 5×5; Subsequently, the basic local features Input three parallel asymmetric large kernel convolutional branches respectively; The first asymmetric large-kernel convolution branch uses depthwise separable convolutions with kernel sizes of 1×7 and 7×1 in sequence to extract texture and local edge features within the first distance range. The calculation process is expressed as follows: The second asymmetric large-kernel convolution branch uses depthwise separable convolutions with kernel sizes of 1×9 and 9×1 respectively to extract contour and structural features within the second distance range. The calculation process is expressed as follows: The third asymmetric large-kernel convolution branch employs depthwise separable convolutions with kernel sizes of 1×11 and 11×1, respectively, to extract contour and structural features within the third distance range. The calculation process is expressed as follows: The basic local features Features output by three asymmetric large kernel convolution branches and Element-wise addition is performed, and feature mixing is completed through 1×1 convolution to obtain multi-scale structural response features. : Utilizing the multi-scale structural response characteristics Normalized input features Element-wise modulation is performed to obtain the output features of the large kernel convolution branch: Therefore, the overall calculation process of the ana module can be represented as follows: in, and These represent features obtained using asymmetric large kernel convolutions with scales of 7, 9, and 11, respectively. and These represent the strip depthwise convolution operations in the horizontal and vertical directions, respectively; This represents an 11×1 convolution operation; This represents element-wise multiplication; This indicates the overall mapping process of the ana module.

8. The image segmentation system based on multi-scale feature fusion and reverse attention mechanism as described in claim 3, characterized in that, The output of the second coding layer is connected to the input of the first reverse attention mechanism layer, and the output of the first reverse attention mechanism layer is connected to the input of the first adder. The output of the third coding layer is connected to the input of the second reverse attention mechanism layer, and the output of the second reverse attention mechanism layer is connected to the input of the second adder. The output of the fourth coding layer is connected to the input of the third reverse attention mechanism layer, and the output of the third reverse attention mechanism layer is connected to the input of the fourth decoding layer. The internal structures of the first, second, and third reverse attention mechanism layers are identical. The first reverse attention mechanism layer includes: Let the first The prediction map output by each decoding layer is The skip connection feature corresponding to the output of the coding layer is Among them, the prediction map Used to generate inverse attention masks, skip connection features These are the high-level features to be screened; First, the prediction graph The spatial dimensions are adjusted to match the spatial dimensions of the corresponding skip connection features to obtain the size-aligned predicted features. : in, Indicates an upsampling operation. and They represent the first Output features of each encoding stage Height and width; when predicting the graph Features of skip connections When the spatial dimensions are consistent, the upsampling operation is an identity mapping; Subsequently, the size-aligned prediction map is calibrated for response, and the foreground probability response is obtained through the Sigmoid activation function. The foreground probability response is then reverse-processed to obtain the inverse attention mask. : in, This represents the Sigmoid activation function. Indicates the first The learnable scaling parameters in the reverse attention mechanism layer are used to adaptively adjust the magnitude of the predicted response. Indicates the relationship with the first Each hop connection corresponds to a reverse attention mask; the reverse attention mask is then extended along the channel dimension: in, Indicates a channel expansion operation. The extended inverse attention mask represents the number of channels corresponding to the skip connection feature. satisfy: Next, the extended inverse attention mask is used to perform element-wise weighting of the skip connection features output by the encoder, thereby achieving prediction-guided feature selection: in, This represents element-wise multiplication. This represents the features after reverse attention weighting; Finally, the skip connection features weighted by reverse attention are sequentially input into the first and second convolutional layers for feature mapping, yielding the output of the reverse attention mechanism layer: in, and These represent the first and second convolution operations in the reverse attention mechanism layer, respectively. This represents the output features obtained after reverse attention filtering and feature mapping.

9. The image segmentation system based on multi-scale feature fusion and reverse attention mechanism as described in claim 3, characterized in that, The outputs of the first, second, third, and fourth coding layers are all connected to the input of the multi-scale affinity feature fusion module. The output of the multi-scale affinity feature fusion module is connected to the input of the multiplier, and the output of the multiplier is connected to the input of the final projection layer.

10. The image segmentation system based on multi-scale feature fusion and reverse attention mechanism as described in claim 9, characterized in that, The multi-scale affinity feature fusion module includes: Given the output features of each stage of the encoder : in, It has the highest spatial resolution, which can preserve more boundary, texture and local detail information; First, bilinear interpolation is used to unify all features to the same level. Same space dimensions: in, This represents the bilinear interpolation operation. Indicates reference feature Spatial dimensions; for the first For each scale feature, the number of channels is first adjusted to match the number of channels of the reference feature using a channel mapping operation. Consistency: in, Including sequentially connected Convolutional and normalization layers are used to perform channel alignment between features across scales; after the channel alignment operation, With reference features They have the same spatial dimensions and channel dimensions; Subsequently, reference features Features aligned with channels Element-wise summation is performed, and the cross-scale affinity response is obtained through the ReLU activation function: Next, the cross-scale affinity response will be discussed. Input the affinity mapping branch to generate a single-channel spatial weight map at the corresponding scale: in, Composed of convolution and normalization operations, it is used to convert cross-scale affinity responses. Mapped to a single-channel spatial response; This represents the Sigmoid activation function, used to constrain the spatial response value to [0,1]; after this operation, the result is... Used to measure the The importance of different spatial locations in each scale feature for the segmentation task; Then, the affinity weight map The feature is expanded to the same channel dimension as the corresponding feature through a broadcast mechanism, and then weighted element-wise for that scale feature. in, This represents element-wise multiplication. This represents the multi-scale features after affinity weighting. Finally, the reference feature F1 and the weighted multi-scale features are compared. The images are stitched together along the channel dimension and then fused using 1×1 convolution to generate a single-channel, multi-scale spatial guidance map. in, This indicates a channel splicing operation. This represents a fusion mapping consisting of 1×1 convolutions, normalization, and activation operations.

Images