Medical image segmentation method based on AFMHiFormer

Abstract

The invention provides a medical image segmentation method based on an AFMHiFormer. The method comprises the steps that firstly, a multiple data enhancement module is provided, and the data distribution diversity is improved while the enhancement stability is guaranteed; secondly, a segmentation model AFHiMFormer is constructed, and the model architecture adopts a double-branch encoder and a multi-scale decoder; thirdly, a feature enhancement module is provided to construct a dynamic complementation mechanism of semantic enhancement and boundary modeling; fourthly, a multi-scale feature fusion module is introduced, multi-scale context information is captured through parallel hole convolution with different expansion rates, and self-adaptive fusion of global and local features is achieved; and fifth, a cross-scale fusion module is designed in the multi-scale decoder, so that the deep layer branch and the shallow layer branch are efficiently fused in a multi-level feature space. According to the method, the advantages of CNN and Transform are combined, dynamic fusion of local and global features is realized by providing a new module, and a remarkable performance advantage is shown in a medical image segmentation task.

Description

Technical Field

[0001] This invention belongs to the field of image segmentation technology. Specifically, it designs a medical image segmentation method based on AFMHiFormer to achieve accurate segmentation of different organs. Background Technology

[0002] Medical image segmentation plays a crucial role in clinical diagnosis, preoperative planning, and disease progression analysis. Traditional medical image segmentation methods, such as thresholding, region growing, level sets, and graph-based segmentation techniques, typically rely on manually defined rules or low-level image features. These methods tend to fail when images are noisy, organ boundaries are blurred, or grayscale distributions are uneven, making them ill-suited for complex and variable medical imaging data. Subsequently, with the development of traditional machine learning methods, segmentation methods based on feature engineering and classifiers have emerged. However, their feature extraction capabilities are limited, making it difficult to capture high-order semantic information of target tissues, and they also lack good generalization ability.

[0003] Since the advent of deep learning, Convolutional Neural Networks (CNNs), especially the U-Net architecture, have become the mainstream method for medical image segmentation. U-Net achieves semantic extraction and spatial reconstruction through an encoder-decoder structure, relying on skip connections to incorporate shallow detail information, achieving remarkable results in organ and lesion segmentation tasks. However, standard CNN models have inherent limitations: the receptive field of the convolutional kernel is limited, making it difficult for the model to simultaneously understand global semantics and local details. Especially in medical image scenarios where multi-scale structures coexist, problems such as missing small organs and incomplete prediction of large organs are prone to occur.

[0004] To enhance the global modeling capabilities of models, the Transformer architecture has been introduced into medical segmentation tasks, such as TransUNet, Swin-UNet, and HiFormer. These methods acquire long-distance dependencies through self-attention mechanisms, significantly improving global context modeling capabilities. However, Transformer methods also face new challenges: their large-scale computational overhead limits their ability to learn local features, and pure Transformer models often fail to effectively capture fine-grained boundary structures in medical images. Furthermore, multi-scale feature fusion remains a challenge; the differences in spatial scale, channel distribution, and information density between deep semantic features and shallow detail features often make simple addition or concatenation insufficient to achieve adequate information complementarity.

[0005] On the other hand, medical images exhibit strong cross-modal variability (e.g., different MRI sequences, different CT scans from different hospitals), and annotation costs are high, resulting in limited available data. Therefore, models are prone to overfitting and insufficient generalization during training. Although common data augmentation strategies (such as rotation and flipping) can increase some sample diversity, they cannot effectively simulate structural deformation, intensity perturbations, and occlusion in real clinical scenarios.

[0006] In summary, existing medical image segmentation technologies still have limitations in key aspects such as multi-scale semantic modeling, structural edge refinement, cross-layer feature fusion, and clinical data adaptability, making it difficult to achieve accurate segmentation in complex anatomical scenarios.

[0007] Therefore, it is necessary to propose a novel medical image segmentation method that can simultaneously enhance local details and global semantic understanding, achieve efficient cross-scale feature fusion, improve the learning ability of intermediate layer features, and possess stronger generalization ability to meet the needs of real-world clinical applications. Based on this, this invention proposes a medical image segmentation method that combines feature enhancement, multi-scale context fusion, cross-scale feature fusion, and multiple data augmentations, effectively improving segmentation accuracy. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and propose a medical image segmentation method based on AFMHiFormer for accurate segmentation of medical images.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows.

[0010] The specific implementation steps of the medical image segmentation method based on AFMHiFormer are as follows:

[0011] Step 1: Before encoding, the medical image is subjected to twelve transformation strategies, including geometric and intensity transformations, to adapt to the medical image. This process enhances stability while increasing the diversity of data distribution before inputting the image into the encoder.

[0012] Step 2: This invention constructs a segmentation model, AFMHiFormer, whose model architecture adopts a hybrid encoder with two branches of CNN and Transformer and a multi-scale decoder.

[0013] Step 3: Input the encoded high-level and low-level features into the feature enhancement module. By introducing channels, spatial attention, and edge gradient operators, a dynamic complementary mechanism is established between semantic saliency enhancement and structural boundary modeling.

[0014] Step four: After the feature enhancement module, a multi-scale feature fusion module is introduced. In the parallel branch, multi-scale contextual information is captured through dilated convolutions with different dilation rates to achieve adaptive fusion of global and local features.

[0015] Step 5: A cross-scale fusion module is designed in the multi-scale decoder to realize feature interaction at multiple resolution stages, so that the semantic abstraction of deep branches and the spatial details of shallow branches can be efficiently fused in multi-level feature spaces.

[0016] In step one, the specific operations performed by the Multi-level Data Augmentation (MDA) module before medical image encoding are: basic geometric transformation, advanced deformation enhancement, intensity domain perturbation, and regularized occlusion. The aim is to ensure that the network maintains a stable feature response under different scales, viewpoints, and noise conditions.

[0017] In step two, the present invention constructs a segmentation model, AFMHiFormer, whose architecture employs a hybrid encoder combining CNN and Transformer branches, along with a multi-scale decoder. The AFMHiFormer dual-branch encoder includes a convolutional branch and a Transformer branch:

[0018] The convolutional branch is primarily responsible for extracting local structural features and texture information from the image, and is the core part of the entire two-branch coding framework for modeling fine-grained anatomical boundaries. In each convolutional layer, a 3×3 convolutional kernel is first used to extract local neighborhood features, and training is stabilized through batch normalization (BN), followed by a non-linear mapping using the GELU activation function. Each convolutional operation can be represented as:

[0019]

[0020] Where l represents the network layer, For the first The output features of the convolutional branches. This design enables the CNN branches to capture clear tissue boundaries and texture variations in medical images, providing precise spatial localization information for small organs and subtle lesions. Simultaneously, it complements the Transformer branches: the former focuses on local identification, while the latter emphasizes global dependencies; their fusion significantly improves segmentation accuracy and structural consistency.

[0021] The core objective of the Transformer branch is to achieve global semantic modeling and long-range dependency capture of images, compensating for the limited receptive field of convolutional networks. This study adopts a hierarchical Transformer branch based on the Swin Transformer, using the Window-based Self-Attention (W-MSA) mechanism as a foundation, and achieves efficient global feature representation through local computation and hierarchical recursion.

[0022] Specifically, the input features are first divided into non-overlapping windows, and self-attention computation is performed within each window:

[0023]

[0024] Where Q, K, and V represent the query, key, and value matrices, respectively. The feature dimension is used. This mechanism can capture semantic dependencies within the same window. However, to overcome the boundary fragmentation problem caused by window partitioning, the Swin Transformer introduces Shifted Window Multi-head Self-Attention (SW-MSA) in adjacent layers, which enables cross-window interaction through window offset, thereby enhancing the transmission of global information.

[0025] Each block in the Transformer branch consists of an attention module and a feedforward network (Multi-Layer Perceptron, MLP), and uses residual connections and layer normalization (LN) for stable training. The computational flow is as follows:

[0026]

[0027]

[0028] This hierarchical self-attention structure enables the model to gradually expand its receptive field while maintaining computational efficiency, and achieves semantic integration from local to global through multi-level token interactions.

[0029] Furthermore, to better adapt to structural features with significant scale differences in medical images, the Transformer branch employs a downsampling linear projection module (Patch Merging) across different stages to achieve feature dimensionality reduction and spatial compression. This strategy effectively reduces computational complexity and strengthens cross-layer semantic associations. By running in parallel with the CNN branch, the Transformer branch provides the overall network with powerful global context modeling capabilities, enabling the model to accurately distinguish spatially adjacent but semantically different structural regions in multi-organ segmentation.

[0030] In step three, the Feature Enhancement Module (FEM) establishes a dynamic complementary mechanism between semantic saliency enhancement and structural boundary modeling by introducing channels, spatial attention, and edge gradient operators.

[0031] To adaptively enhance features of important channels and spatial locations, this module first adaptively redistributes feature weights from both the "channel dimension" and the "spatial dimension" by sequentially embedding two sub-modules: Channel Attention and Spatial Attention. This allows the network to focus on key information channels while strengthening responses to significant spatial regions, thus making the network more attentive to anatomical structures that contribute to the segmentation results.

[0032] Channel attention aims to uncover the importance relationships between different feature channels. Given an input feature map... First, global channel statistics are obtained through Global Average Pooling (GAP) and Global Max Pooling (GMP) respectively:

[0033]

[0034]

[0035] Subsequently, these two statistical features are input into a weighted MLP, and after ReLU activation and Sigmoid function normalization, the channel attention weights are obtained:

[0036]

[0037] in, Represents the ReLU function. Activated for Sigmoid , This is the learnable parameter matrix.

[0038] The final channel-weighted features are:

[0039]

[0040] in This indicates a channel-wise multiplication operation, which enables the network to adaptively enhance or suppress the response on different channels.

[0041] Feature map after channel weighting Building upon this foundation, the module further captures salient regional positional relationships in the spatial dimension. The calculation process is as follows:

[0042]

[0043] After the two are spliced ​​together in the channel dimension, Convolution extracts spatial dependencies and generates spatial attention weights:

[0044]

[0045] Final output augmented features:

[0046]

[0047] Through this mechanism, the model can "learn" at the feature level which channels are more important and which spatial locations are more critical, thereby achieving a synergistic enhancement of global and local saliency in feature representation. The channel weighting stage is responsible for highlighting feature channels with high semantic value, while the spatial weighting stage further refines the structural distribution, enabling the model to achieve a balance between semantic focus and spatial localization.

[0048] Building upon saliency enhancement, FEM further introduces explicit structural boundary modeling to strengthen the recognition of organ contours and tissue transition regions. Specifically, the module first uses gradient operators to extract the edge responses of features and calculates gradient information in the horizontal and vertical directions:

[0049]

[0050] in and Sobel convolution kernels for the horizontal and vertical directions, respectively, and the convolution operator " "Indicates a local weighted summation operation:

[0051] ,

[0052] By combining gradient information from both directions, the edge intensity map is calculated. :

[0053]

[0054] This operation can capture regions in an image with significant intensity variations, highlighting structural boundaries and texture details. To suppress noise and improve edge continuity, the edge map... Feed the data into a learnable convolutional layer for non-linear refinement:

[0055]

[0056] Finally, the refined edge features Integrating with semantic features to achieve complementary enhancement of edges and semantics:

[0057]

[0058] in is a learnable fusion coefficient used to control the contribution of edge features in the final output, with an initial value set to 0.1.

[0059] FEM achieves a two-layer enhancement of feature representation through a combination of adaptive saliency modeling and explicit boundary reinforcement. This module effectively bridges the representational gap between semantic information and structural details, enabling the segmentation results to exhibit higher structural consistency and contour accuracy in small organs and blurred edge regions. FEM achieves progressive optimization from global semantics to local geometry, providing a more discriminative input representation for subsequent feature fusion and decoding.

[0060] In step four, the multi-scale feature fusion module (Atrous Fusion, AF) captures multi-scale contextual information through dilated convolutions with different dilation rates in the parallel branch, thereby achieving adaptive fusion of global and local features.

[0061] The AF module consists of five parallel branches, each with a different receptive field, used to extract contextual features at different scales. Given an input feature map... The specific definitions of the five branches are as follows:

[0062] First, the first branch, using Convolution extracts pixel-wise features, primarily preserving local fine structures.

[0063]

[0064] Secondly, the second, third, and fourth branches are multi-scale dilated convolution branches, which use different dilation rates. of Dilated convolution is used to capture contextual information across different receptive fields.

[0065]

[0066] Wherein, dilated convolution is defined as:

[0067]

[0068] in, For expansion rate, For output position, Using convolution kernels, the hollow structure can expand the receptive field without increasing parameters.

[0069] Finally, the fifth branch is responsible for capturing global contextual information. It captures the semantic information of the entire graph through global average pooling, and then... Convolution and upsampling restore the original resolution:

[0070]

[0071] The features output from the five branches are concatenated along the channel dimension to form a multi-scale feature set:

[0072]

[0073] Where "[;]" indicates a cascading operation along the channel direction, and the feature dimension after concatenation is... .

[0074] To reduce computational burden and achieve multi-scale information integration, 1×1 convolution is used for channel projection and nonlinear transformation:

[0075]

[0076] Obtain the fused feature map This module achieves multi-level feature capture from local to global by parallel convolutions at different dilation rates: small dilation rate branch ( Focusing on local information such as texture and edges; the medium dilation branch ( Perceiving mesoscale structures; high expansion rate branch ( The system models long-distance dependencies and uses a global average branch to capture global semantic context. Ultimately, the five scale features are fused to form a richly layered contextual representation, providing more complete semantic information for subsequent decoding stages.

[0077] The overall fusion features output by the AF module can be expressed as follows:

[0078]

[0079] This feature preserves both local edge details and global semantic structure, providing key feature support for subsequent multi-scale decoding.

[0080] Compared to traditional convolutional layers, the AF module expands the effective receptive field of the model without significantly increasing computational cost, and achieves global integration of context through a multi-scale branch parallel structure.

[0081] In step five, a cross-scale fusion module (CFM) is designed in the multi-scale decoder to realize feature interaction at multiple resolution stages, enabling the semantic abstraction of deep branches and the spatial details of shallow branches to be efficiently fused in multi-level feature spaces.

[0082] This decoder comprises two parallel branches: a small-scale semantic branch and a large-scale detail branch. The former gradually recovers the spatial structure from deep semantic features, while the latter extracts edge and detail information from shallow features. The two are fused step-by-step at medium and high resolution stages to balance the global consistency and boundary accuracy of the segmentation results. Traditional decoders typically employ simple weighted addition or cascading operations in the feature fusion stage, ignoring the differences in channel and spatial distribution of features from different sources, which can easily lead to the loss of details dominated by semantic information.

[0083] To address this issue, the CFM module introduces a channel-space dual saliency modeling mechanism, which adaptively selects important semantic channels and strengthens key spatial regions during the fusion process, thereby achieving synergistic enhancement of multi-scale features.

[0084] CFM performs cross-branch fusion in two stages: the first stage fuses deep semantics with mid-level details; the second stage fuses shallow edge information with decoded high-level features.

[0085] In the first stage, let the output of the small-scale semantic branch after the second upsampling be... The initial output of the large-scale detail branch is .

[0086] Firstly, through Convolutional alignment channel number:

[0087]

[0088] Then, element-by-element addition is performed to achieve initial fusion:

[0089]

[0090] To further enhance the discriminative power of the fused features, an adaptive channel-spatial reweighting mechanism is introduced to calculate the fused attention weights:

[0091]

[0092]

[0093]

[0094]

[0095] This process enables channel-level filtering and spatial-level enhancement of fused features, thereby achieving complementary integration of semantic and detailed information at the mesoscale stage.

[0096] In the second stage, let the output of the third upsampling of the small-scale semantic branch be... The final output of the large-scale branch is Similarly, channel alignment is performed first:

[0097]

[0098] Further integration and attention enhancement:

[0099]

[0100]

[0101] in and These are the channel and spatial attention diagrams, respectively, and the calculation process is the same as in stage 1.

[0102] The main goal of this stage is to integrate high-resolution semantics with shallow boundary features to ensure that the model can balance overall structural consistency and boundary continuity during prediction.

[0103] Finally, the features from the two stages are fused, and the formula is expressed as:

[0104]

[0105] The final fused output will be fed into the subsequent upsampling and prediction module to generate high-resolution segmentation results.

[0106] The method proposed in this invention has the following main advantages.

[0107] (1) This invention proposes multiple data augmentation modules to expand the data distribution from multiple dimensions of space, morphology and intensity, so that the model can maintain stable performance across patients, devices and different acquisition conditions.

[0108] (2) This invention proposes a novel feature enhancement module, which realizes information reweighting of channel and spatial dimensions through the feature enhancement module, and combined with the edge structure enhancement mechanism, effectively improves the recognition accuracy of organ boundaries and small targets.

[0109] (3) The present invention adopts an adaptive multi-scale fusion structure, which can capture local texture and global semantics at the same time, significantly improving the model’s adaptability to organs of different sizes and shapes.

[0110] (4) The cross-scale fusion module proposed in this invention based on the multi-scale decoder can effectively fuse deep semantics and shallow details, solve the semantic inconsistency problem of traditional decoders, and improve the structural integrity of the recovery. Attached Figure Description

[0111] Figure 1 This is a flowchart of the medical image segmentation method based on AFMHiFormer of the present invention.

[0112] Figure 2 This is a diagram of the Multi-level Data Augmentation (MDA) structure.

[0113] Figure 3 This is a diagram showing the overall structure of the AFMHiFormer network of this invention.

[0114] Figure 4 This is a structural diagram of the Feature Enhancement Module (FEM).

[0115] Figure 5 This is a structural diagram of Channel Attention.

[0116] Figure 6 This is a structural diagram of Spatial Attention.

[0117] Figure 7 This is a structural diagram of the Atrous Fusion Module (AFM).

[0118] Figure 8 This is a diagram of the Cross-scale Fusion Module (CFM).

[0119] Figure 9 This is a segmentation visualization of the Synapse dataset.

[0120] Figure 10 A visualization comparison of multi-model segmentation on the Synapse dataset. Detailed Implementation

[0121] The present invention will be further described below with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be noted that those skilled in the art can make some modifications to the present invention without departing from its core spirit, and these modifications all fall within the scope of protection of the present invention.

[0122] like Figure 1 The flowchart shown is for a medical image segmentation method based on AFMHiFormer, which specifically includes the following steps:

[0123] Step 1, as follows Figure 2 As shown, medical images are subjected to twelve transformation strategies, including geometric and intensity transformations, before encoding. This approach enhances data distribution diversity while maintaining stability before inputting the images into the encoder. The specific operation description is as follows:

[0124] This example uses the Synapse multi-organ segmentation dataset for validation. This dataset contains 30 abdominal CT scans from the MICCAI 2015 challenge. Each CT volume consists of 85–198 512×512 pixel slices, with detailed annotations for eight abdominal organ categories (aorta, gallbladder, spleen, left kidney, right kidney, liver, pancreas, spleen, and stomach). Following a standard segmentation scheme, 18 scans were used for training and 12 for testing.

[0125] The original image undergoes basic geometric transformations, advanced deformation enhancement, intensity domain perturbation, and regularized occlusion to complete the data preprocessing process required for model training. The aim is to ensure that the network maintains stable feature responses under different scales, viewpoints, and noise conditions.

[0126] Step two, as Figure 3 As shown, this invention constructs a segmentation model, AFMHiFormer. The model architecture employs a hybrid encoder combining CNN and Transformer branches, along with a multi-scale decoder; the AFMHiFormer dual-branch encoder includes a convolutional branch and a Transformer branch:

[0127] The convolutional branch is primarily responsible for extracting local structural features and texture information from the image, and is the core part of the entire dual-branch coding framework for modeling fine-grained anatomical boundaries. In each convolutional layer, a 3×3 convolutional kernel is first used to extract local neighborhood features, and then the training is stabilized through batch normalization. Finally, a non-linear mapping is achieved through the GELU activation function. Each convolutional operation can be represented as:

[0128]

[0129] in, Indicates network hierarchy, For the first The output features of the convolutional branch; this design enables the CNN branch to capture clear tissue boundaries and texture changes in medical images, providing accurate spatial localization information for small organs and subtle lesions; at the same time, it complements the Transformer branch: the former focuses on local identification, while the latter focuses on global dependence, and the fusion of the two significantly improves segmentation accuracy and structural consistency.

[0130] The core objective of the Transformer branch is to achieve global semantic modeling and long-range dependency capture of images, compensating for the limited receptive field of convolutional networks. This study adopts a hierarchical Transformer branch based on the Swin Transformer, using a window attention mechanism to achieve efficient global feature representation through local computation and hierarchical recursion. Specifically, the input features are first divided into non-overlapping windows, and self-attention computation is performed within each window.

[0131]

[0132] Where Q, K, and V represent the query, key, and value matrices, respectively. The feature dimension is used; this mechanism can capture semantic dependencies within the same window; however, to overcome the boundary fragmentation problem caused by window partitioning, the Swin Transformer introduces shifted window attention in adjacent layers, achieving cross-window interaction through window offset, thereby enhancing global information transmission; each block of the Transformer branch consists of an attention module and an MLP, and residual connections and layer normalization are used to stabilize training; its computation flow is as follows:

[0133]

[0134]

[0135] This hierarchical self-attention structure enables the model to gradually expand its receptive field while maintaining computational efficiency, and achieves semantic integration from local to global through multi-level token interaction. In addition, to further adapt to structural features with large scale differences in medical images, the Transformer branch uses a downsampling linear projection module between different stages to achieve feature dimensionality reduction and spatial compression. This strategy effectively reduces computational complexity and strengthens cross-layer semantic association. By running in parallel with the CNN branch, the Transformer branch provides the whole network with powerful global context modeling capabilities, enabling the model to accurately distinguish spatially adjacent but semantically different structural regions in multi-organ segmentation.

[0136] Step 3: The feature enhancement module establishes a dynamic complementary mechanism between semantic saliency enhancement and structural boundary modeling by introducing channels, spatial attention, and edge gradient operators. The construction method is as follows:

[0137] like Figure 4 As shown, the deep and shallow features of the image after dual-branch decoding are respectively fed into the feature enhancement module. In order to adaptively enhance the features of important channels and spatial locations, this module first adaptively redistributes feature weights from both the "channel dimension" and the "spatial dimension" by sequentially embedding two sub-modules: channel attention and spatial attention. This allows the network to focus on key information channels and strengthen the response of significant spatial regions, thus making the network pay more attention to the anatomical structures that contribute to the segmentation results.

[0138] like Figure 5 As shown, channel attention aims to uncover the importance relationships between different feature channels; given an input feature map... First, global channel statistics are obtained through GAP and GMP respectively:

[0139]

[0140]

[0141] Subsequently, these two statistical features are input into a weighted MLP, and after ReLU activation and Sigmoid function normalization, the channel attention weights are obtained:

[0142]

[0143] in, Represents the ReLU function. Activated for Sigmoid , The learnable parameter matrix is ​​used; the final channel-weighted features are:

[0144]

[0145] in This indicates a channel-wise multiplication operation, which enables the network to adaptively enhance or suppress the response on different channels;

[0146] Feature map after channel weighting Building upon this foundation, spatial attention further captures salient regional location relationships, such as Figure 6 As shown, the calculation process is as follows:

[0147]

[0148] After the two are spliced ​​together in the channel dimension, Convolution extracts spatial dependencies and generates spatial attention weights:

[0149]

[0150] Final output augmented features:

[0151]

[0152] Through this mechanism, the model can "learn" which channels are more important and which spatial locations are more critical at the feature level, thereby achieving synergistic enhancement of global and local saliency in feature representation; the channel weighting stage is responsible for highlighting feature channels with high semantic value, while the spatial weighting stage further refines the structural distribution, enabling the model to achieve a balance between semantic focus and spatial positioning.

[0153] Building upon saliency enhancement, FEM further introduces explicit structural boundary modeling to strengthen the ability to identify organ contours and tissue transition regions. Specifically, the module first uses gradient operators to extract the edge responses of features and calculates gradient information in the horizontal and vertical directions.

[0154]

[0155] in and Sobel convolution kernels for the horizontal and vertical directions, respectively, and the convolution operator " "Indicates a local weighted summation operation:

[0156] , ,

[0157] By combining gradient information from both directions, the edge intensity map is calculated. :

[0158]

[0159] This operation can capture regions with significant intensity changes in an image, highlighting structural boundaries and texture details; to suppress noise and improve edge continuity, the edge map... Feed the data into a learnable convolutional layer for non-linear refinement:

[0160]

[0161] Finally, the refined edge features Integrating with semantic features to achieve complementary enhancement of edges and semantics:

[0162]

[0163] in is a learnable fusion coefficient used to control the contribution of edge features to the final output, with an initial value set to 0.1;

[0164] Step four, as Figure 7 As shown, the multi-scale feature fusion module captures multi-scale contextual information through dilated convolutions with different dilation rates in the parallel branch, thereby achieving adaptive fusion of global and local features.

[0165] The AF module consists of five parallel branches, each with a different receptive field, used to extract contextual features at different scales; given the input feature map... The specific definitions of the five branches are as follows:

[0166] First, the first branch, using Convolution extracts pixel-wise features, primarily preserving local fine structures.

[0167]

[0168] Secondly, the second, third, and fourth branches are multi-scale dilated convolution branches, which use different dilation rates. of Dilated convolution is used to capture contextual information across different receptive fields.

[0169]

[0170] Wherein, dilated convolution is defined as:

[0171]

[0172] in, For expansion rate, For output position, As a convolution kernel, the hollow structure can expand the receptive field without increasing the parameters;

[0173] Finally, the fifth branch is responsible for capturing global contextual information. It captures the semantic information of the entire graph through global average pooling, and then... Convolution and upsampling restore the original resolution:

[0174]

[0175] The features output from the five branches are concatenated along the channel dimension to form a multi-scale feature set:

[0176]

[0177] Where "[;]" indicates a cascading operation along the channel direction, and the feature dimension after concatenation is... ;

[0178] To reduce computational burden and achieve multi-scale information integration, 1×1 convolution is used for channel projection and nonlinear transformation:

[0179]

[0180] Obtain the fused feature map This module achieves multi-level feature capture from local to global by parallel convolutions at different dilation rates: small dilation rate branch ( Focusing on local information such as texture and edges; the medium dilation branch ( Perceiving mesoscale structures; high expansion rate branch ( The modeling process involves long-distance dependencies; global average branching captures global semantic context; and finally, the five scale features are fused to form a rich and hierarchical contextual representation, providing more complete semantic information for the subsequent decoding stage.

[0181] The overall fusion features output by the AF module can be expressed as follows:

[0182]

[0183] This feature preserves both local edge details and global semantic structure, providing key feature support for subsequent multi-scale decoding.

[0184] Step 5: A cross-scale fusion module was designed in the multi-scale decoder. The specific steps are as follows: Figure 8 As shown, this module enables feature interaction at multiple resolution stages, allowing the semantic abstraction of deep branches and the spatial details of shallow branches to be efficiently integrated in multi-level feature spaces.

[0185] The decoder comprises two parallel branches: a small-scale semantic branch and a large-scale detail branch. The former gradually recovers the spatial structure from deep semantic features, while the latter extracts edge and detail information from shallow features. The two branches are fused step-by-step at medium and high resolution stages to balance the global consistency and boundary accuracy of the segmentation results. Traditional decoders typically employ simple weighted addition or cascading operations in the feature fusion stage, ignoring the differences in channel and spatial distribution of features from different sources, which can easily lead to the loss of details dominated by semantic information. To address this issue, the CFM module introduces a channel-space dual saliency modeling mechanism, which adaptively selects important semantic channels and strengthens key spatial regions during the fusion process, thereby achieving synergistic enhancement of multi-scale features.

[0186] like Figure 9 As shown in the figure, this invention presents a segmentation visualization on the Synapse dataset. The figure compares the input CT image, the ground truth label, and the prediction results of AFMHiFormer. The visualization demonstrates that AFMHiFormer accurately reconstructs the shape and boundaries of each organ, especially for organs with blurred outlines and slender structures such as the pancreas and gallbladder. The prediction results are highly consistent with the ground truth labels, with smooth edges and significantly reduced omissions and misclassifications, fully demonstrating the model's advantages in boundary modeling and detail restoration.

[0187] like Figure 10As shown, this invention presents a comparative visualization of multi-model segmentation on the Synapse dataset. In the multi-model comparative visualization (compared with methods such as Swin-UNet, TransUNet, and HiFormer), AFMHiFormer demonstrates superior organ segmentation performance. Compared to other models that often suffer from organ edge defects, organ adhesions, or small structural breaks, AFMHiFormer generates more complete and continuous organ regions. For example, at the junction of the spleen and stomach, this model can better distinguish between the two, and maintains a clear structural boundary in the area of ​​the right kidney with low contrast to surrounding tissues. Furthermore, compared to the original HiFormer, AFMHiFormer provides more accurate contour rendering of detailed areas, reducing edge blurring and over-smoothing issues.

Claims

1. A medical image segmentation method based on AFMHiFormer, the specific implementation steps of which are as follows: Step 1: Before encoding, the medical image is subjected to twelve transformation strategies, including geometric and intensity transformations, to adapt to the medical image. This process enhances stability while increasing the diversity of data distribution before inputting the image into the encoder. Step 2: This invention constructs a segmentation model, AFMHiFormer, whose model architecture employs a hybrid encoder combining a Convolutional Neural Network (CNN) and a Transformer, as well as a multi-scale decoder. Step 3: Input the encoded high-level and low-level features into the feature enhancement module. By introducing channels, spatial attention, and edge gradient operators, a dynamic complementary mechanism is established between semantic saliency enhancement and structural boundary modeling. Step four: After the feature enhancement module, a multi-scale feature fusion module is introduced. In the parallel branch, multi-scale contextual information is captured through dilated convolutions with different dilation rates to achieve adaptive fusion of global and local features. Step 5: A cross-scale fusion module is designed in the multi-scale decoder to realize feature interaction at multiple resolution stages, so that the semantic abstraction of deep branches and the spatial details of shallow branches can be efficiently fused in multi-level feature spaces.

2. The medical image segmentation method based on AFMHiFormer according to claim 1, characterized in that: The specific operations of the Multi-level Data Augmentation (MDA) module in step one before medical image encoding are: basic geometric transformation, advanced deformation enhancement, intensity domain perturbation and regularized occlusion. The purpose is to ensure that the network can maintain a stable feature response under different scales, viewpoints and noise conditions.

3. The medical image segmentation method based on AFMHiFormer according to claim 1, characterized in that: In step two, this invention constructs a segmentation model, AFMHiFormer, whose architecture employs a hybrid encoder combining CNN and Transformer branches, along with a multi-scale decoder. The AFMHiFormer dual-branch encoder includes a convolutional branch and a Transformer branch. The convolutional branch is primarily responsible for extracting local structural features and texture information from the image, and is the core part of the entire dual-branch coding framework for modeling fine-grained anatomical boundaries. In each convolutional layer, a 3×3 convolutional kernel is first used to extract local neighborhood features, and then the training is stabilized through batch normalization (BN), followed by non-linear mapping through the GELU activation function. Each convolutional operation can be represented as: Where l represents the network layer, The output features of the l-th convolutional branch are used; this design enables the CNN branch to capture clear tissue boundaries and texture changes in medical images, providing accurate spatial localization information for small organs and subtle lesions; at the same time, it complements the Transformer branch: the former focuses on local identification, while the latter focuses on global dependence, and the fusion of the two significantly improves segmentation accuracy and structural consistency. The core objective of the Transformer branch is to achieve global semantic modeling and long-range dependency capture of images, compensating for the limited receptive field of convolutional networks. This study adopts a hierarchical Transformer branch based on the Swin Transformer, using a window-based self-attention (W-MSA) mechanism to achieve efficient global feature representation through local computation and hierarchical recursion. Specifically, the input features are first divided into non-overlapping windows, and self-attention computation is performed within each window. Where Q, K, and V represent the query, key, and value matrices, respectively. The feature dimension is used; this mechanism can capture semantic dependencies within the same window; however, to overcome the boundary fragmentation problem caused by window partitioning, the Swin Transformer introduces Shifted Window Multi-head Self-Attention (SW-MSA) in adjacent layers, which realizes cross-window interaction through window offset, thereby enhancing the transmission of global information; each block of the Transformer branch consists of an attention module and a feedforward network (Multi-Layer Perceptron, MLP), and residual connections and layer normalization (LN) are used to stabilize training; its computation flow is as follows: This hierarchical self-attention structure enables the model to gradually expand its receptive field while maintaining computational efficiency, and achieves semantic integration from local to global through multi-level token interaction. In addition, to further adapt to structural features with large scale differences in medical images, the Transformer branch uses a downsampling linear projection module (PatchMerging) between different stages to achieve feature dimensionality reduction and spatial compression. This strategy effectively reduces computational complexity and strengthens cross-layer semantic association. By running in parallel with the CNN branch, the Transformer branch provides the whole network with powerful global context modeling capabilities, enabling the model to accurately distinguish spatially adjacent but semantically different structural regions in multi-organ segmentation.

4. The medical image segmentation method based on AFMHiFormer according to claim 1, characterized in that: In step three, the Feature Enhancement Module (FEM) establishes a dynamic complementary mechanism between semantic saliency enhancement and structural boundary modeling by introducing channels, spatial attention, and edge gradient operators. To adaptively enhance the features of important channels and spatial locations, this module first adaptively redistributes feature weights from both the "channel dimension" and the "spatial dimension" by sequentially embedding two sub-modules: Channel Attention and Spatial Attention. This allows the network to focus on key information channels and enhance responses in significant spatial regions, thus making the network pay more attention to anatomical structures that contribute to the segmentation results. Channel attention aims to uncover the importance relationships between different feature channels; given an input feature map... First, global channel statistics are obtained through Global Average Pooling (GAP) and Global Max Pooling (GMP), respectively: Subsequently, these two statistical features are input into a weighted MLP, and after ReLU activation and Sigmoid function normalization, the channel attention weights are obtained: in, Represents the ReLU function. Activated for Sigmoid , The learnable parameter matrix is ​​used; the final channel-weighted features are: in This represents a channel-wise multiplication operation, which allows the network to adaptively enhance or suppress the response on different channels; the feature map after channel weighting. Building upon this foundation, the module further captures the positional relationships of significant regions in the spatial dimension; its calculation process is as follows: After the two are spliced ​​together in the channel dimension, Convolution extracts spatial dependencies and generates spatial attention weights: Final output augmented features: Through this mechanism, the model can "learn" which channels are more important and which spatial locations are more critical at the feature level, thereby achieving synergistic enhancement of global and local saliency in feature representation; the channel weighting stage is responsible for highlighting feature channels with high semantic value, while the spatial weighting stage further refines the structural distribution, enabling the model to achieve a balance between semantic focus and spatial positioning. Building upon saliency enhancement, FEM further introduces explicit structural boundary modeling to strengthen the ability to identify organ contours and tissue transition regions. Specifically, the module first uses gradient operators to extract the edge responses of features and calculates gradient information in the horizontal and vertical directions. in and Sobel convolution kernels and convolution operators are used in the horizontal and vertical directions, respectively. "Indicates a local weighted summation operation: ， By combining gradient information from both directions, the edge intensity map is calculated. : This operation can capture regions with significant intensity changes in an image, highlighting structural boundaries and texture details; to suppress noise and improve edge continuity, the edge map... Feed the data into a learnable convolutional layer for non-linear refinement: Finally, the refined edge features Integrating with semantic features to achieve complementary enhancement of edges and semantics: in is a learnable fusion coefficient used to control the contribution of edge features to the final output, with an initial value set to 0.1; FEM achieves a two-layer enhancement of feature representation by combining adaptive saliency modeling and explicit boundary reinforcement. This module effectively bridges the expression gap between semantic information and structural details, enabling the segmentation results to exhibit higher structural consistency and contour accuracy in small organs and blurred edge regions. FEM achieves progressive optimization from global semantics to local geometry, providing a more discriminative input representation for subsequent feature fusion and decoding.

5. The medical image segmentation method based on AFMHiFormer according to claim 1, characterized in that: In step four, the multi-scale feature fusion module (Atrous Fusion, AF) captures multi-scale contextual information through dilated convolutions with different dilation rates in the parallel branch, thereby achieving adaptive fusion of global and local features. The AF module consists of five parallel branches, each with a different receptive field, used to extract contextual features at different scales; given the input feature map... The specific definitions of the five branches are as follows: First, the first branch, using Convolution extracts pixel-wise features, primarily preserving local fine structures. Secondly, the second, third, and fourth branches are multi-scale dilated convolution branches, which use different dilation rates. of Dilated convolution is used to capture contextual information across different receptive fields. Wherein, dilated convolution is defined as: in, For expansion rate, For output position, As a convolution kernel, the hollow structure can expand the receptive field without increasing the parameters; Finally, the fifth branch is responsible for capturing global contextual information. It captures the semantic information of the entire graph through global average pooling, and then... Convolution and upsampling restore the original resolution: The features output from the five branches are concatenated along the channel dimension to form a multi-scale feature set: Here, "[;]" represents a cascading operation along the channel direction, and the feature dimension after concatenation is... ; To reduce computational burden and achieve multi-scale information integration, 1×1 convolution is used for channel projection and nonlinear transformation: Obtain the fused feature map This module achieves multi-level feature capture from local to global by performing parallel convolutions at different dilation rates: small dilation rate branch ( Focusing on local information such as texture and edges; the medium dilation branch ( Perceiving mesoscale structures; high expansion rate branch ( The modeling process involves long-distance dependencies; global average branching captures global semantic context; and finally, the five scale features are fused to form a rich and hierarchical contextual representation, providing more complete semantic information for the subsequent decoding stage. The overall fusion features output by the AF module can be expressed as follows: This feature preserves both local edge details and global semantic structure, providing key feature support for subsequent multi-scale decoding; Compared to traditional convolutional layers, the AF module expands the effective receptive field of the model without significantly increasing computational cost, and achieves global integration of context through a multi-scale branch parallel structure.

6. The medical image segmentation method based on AFMHiFormer according to claim 1, characterized in that: In step five, a cross-scale fusion module (CFM) is designed in the multi-scale decoder to realize feature interaction at multiple resolution stages, so that the semantic abstraction of deep branches and the spatial details of shallow branches can be efficiently fused in multi-level feature spaces. The decoder consists of two parallel branches: a small-scale semantic branch and a large-scale detail branch; The former gradually recovers the spatial structure from deep semantic features, while the latter extracts edge and detail information from shallow features. The two are fused step by step at medium and high resolution stages to balance the global consistency and boundary accuracy of the segmentation results. Traditional decoders usually use simple weighted addition or cascaded operations in the feature fusion stage, ignoring the differences in channel and spatial distribution of features from different sources, which can easily lead to the loss of details dominated by semantic information. To solve this problem, the CFM module introduces a channel-space dual saliency modeling mechanism, which adaptively selects important semantic channels and strengthens key spatial regions during the fusion process, thereby achieving synergistic enhancement of multi-scale features. CFM performs cross-branch fusion in two phases: the first phase is to fuse deep semantics with mid-level details; The second stage involves fusing shallow edge information with decoded high-level features; In the first stage, let the output of the small-scale semantic branch after the second upsampling be... The initial output of the large-scale detail branch is ; Firstly, through Convolutional alignment channel number: Then, element-by-element addition is performed to achieve initial fusion: To further enhance the discriminative power of the fused features, an adaptive channel-spatial reweighting mechanism is introduced to calculate the fused attention weights: This process enables channel-level filtering and spatial-level enhancement of fused features, thereby achieving complementary integration of semantic and detailed information at the mesoscale stage. In the second stage, let the output of the third upsampling of the small-scale semantic branch be... The final output of the large-scale branch is Similarly, channel alignment is performed first: Further integration and attention enhancement: in and These are channel and spatial attention diagrams, respectively, and the calculation process is the same as in stage 1. The main goal of this stage is to integrate high-resolution semantics with shallow boundary features to ensure that the model can balance overall structural consistency and boundary continuity during prediction. Finally, the features from the two stages are fused, and the formula is expressed as: The final fused output will be fed into the subsequent upsampling and prediction module to generate high-resolution segmentation results.

Images