A medical image segmentation method based on double encoders and double path fusion decoder

Abstract

The present application relates to a kind of medical image segmentation method and system based on double encoder and double path fusion decoder.The method is respectively inputted two encoders to preprocessed image: local structure enhancement and feature extractor utilize SCEM to promote boundary contrast, organ structure characteristics are combined with MSDE capturer;Global anatomical context structure extractor is based on PVT modeling global semantic dependence.Subsequently, using HCAM to modulate the attention of double encoder characteristics and depth fusion, then realize cross-level global interaction by MDFR-HFM.Finally, DPF-Decoder is generated through the dual-path design of the two paths of gradual fusion and global direct fusion, local details and global semantics are considered, and the final segmentation map is generated.The present application significantly improves the accuracy, robustness and generalization ability of abdominal multi-organ segmentation.

Description

Technical Field

[0001] This invention relates to a deep learning-based medical image segmentation method and system. The method integrates salient contrast enhancement and orientation-sensitive multi-scale feature extraction techniques through an end-to-end neural network architecture, effectively achieving accurate and automated segmentation of multiple organs in abdominal CT or MRI images. This invention is applicable to clinical scenarios such as computer-aided diagnosis. Background Technology

[0002] Medical image segmentation is a core component of medical image processing, aiming to accurately distinguish regions of interest (ROIs) such as organs, tissues, and lesions from the background in images. It is a crucial support for the construction of computer-aided diagnostic (CAD) systems, surgical navigation, pathological analysis, and efficacy evaluation. With the rapid evolution of deep learning technology, fully convolutional networks (FCNs) and their classic variant U-Net have brought revolutionary breakthroughs. These models employ an encoder-decoder architecture, where the encoder extracts hierarchical features from the image layer by layer, and the decoder restores spatial resolution through upsampling to generate segmentation results. The skip connections between these models effectively fuse shallow details and deep semantic information, significantly improving segmentation accuracy and efficiency.

[0003] However, existing deep learning methods still face many challenges in abdominal multi-organ segmentation scenarios. Firstly, the low contrast problem is prominent. Some abdominal organs have very little grayscale difference and blurred boundaries with surrounding tissues. After multiple downsampling and nonlinear transformations, traditional convolutional neural networks (CNNs) can easily mask these subtle intensity differences, making it difficult for the model to accurately identify and locate boundaries. Secondly, multi-scale and directional processing is challenging. Abdominal images contain organs of varying scales, ranging from large organs like the liver and spleen to smaller organs like the pancreatic duct and gallbladder. Furthermore, the complex morphology of organs and the directional heterogeneity of their edges make it difficult for general CNN operations (such as standard convolution and pooling) to simultaneously consider both global context and fine local structure.

[0004] Third, there is insufficient collaborative modeling of local details and global semantics. CNNs excel at extracting local features and spatial details, but their receptive field is limited, making it difficult to establish global long-range semantic dependencies. While Transformer-based encoders (such as PVT) can model the global context through self-attention, their computational complexity is high, and they lack the translation invariance and locality priors of CNNs, easily ignoring subtle structures and weakening the discriminative power of channel-dimensional features. It is difficult to combine the advantages of both. Fourth, the feature fusion mechanism is simple. Encoder cross-level fusion and decoder skip connections often use direct concatenation or addition, without considering the semantic gap and scale differences between different features, and lacking an adaptive interaction mechanism, resulting in insufficient fusion feature representation capabilities, and difficulty in recovering small organ details during decoding. Fifth, information loss occurs during the decoding process. The upsampling and feature transformation of standard decoders easily introduce noise and blurred details. Small organs themselves have weak physical characteristics, and their information is easily masked by the strong features of large organs, resulting in missed segmentation or inaccurate boundaries.

[0005] To address the aforementioned issues, this invention proposes an end-to-end architecture of "dual encoder + DPF-Decoder". A salient contrast enhancement module (SCEM) amplifies the difference between the target and background, alleviating the low contrast problem; direction-sensitive multi-scale convolution (MSDE) captures organ morphology and edge characteristics to address multi-scale and directional challenges; a hybrid attention-convolution module (HCAM) fuses the local advantages of CNNs with the global capabilities of Transformers to solve the problem of insufficient collaborative modeling; a multi-dilution rate feature optimization module (MDFR) and a hierarchical fusion strategy construct a cross-level contextual modeling mechanism to improve fusion quality; and a dual-path decoder focuses on preserving small organ details and maintaining global semantics respectively, outputting accurate results through residual integration. Validation on publicly available datasets demonstrates that this method outperforms existing state-of-the-art techniques. Summary of the Invention

[0006] The purpose of this invention is to overcome the aforementioned deficiencies in existing medical image segmentation techniques and provide a deep learning segmentation method and system that can effectively enhance target saliency, fuse multi-scale directional features, coordinate local details and global context, and perform refined feature fusion and decoding. This method aims to significantly improve the accuracy, robustness, and generalization ability of multi-organ segmentation in medical images, particularly abdominal CT images.

[0007] To achieve the above objectives, the core concept of this invention lies in constructing an end-to-end network architecture called "dual encoder + DPF-Decoder". Its core technological innovation is mainly reflected in the following four closely related components:

[0008] 1. Local-global alignment fusion encoder:

[0009] Specifically, this module comprises two key sub-components: the LocalStructural Enhancement and Feature Extractor and the Global Anatomical Contextual Structure Extractor. The local component features a self-designed dual-enhancement structure, focusing on enhancing and representing pixel-level local features; the global component is based on an improved PyramidVision Transformer (PVT) architecture to capture cross-scale semantic contextual information.

[0010] In the following discussion, we will focus on the local encoder, and elaborate on the design motivation, structural composition and implementation principle of its two-stage enhancement mechanism, explaining how it can effectively improve the model's ability to preserve details and recognize structures in complex medical image scenarios through a hierarchical enhancement strategy.

[0011] Phase 1: The SCEM module simulates the sensitivity of the human visual system to local contrast, aiming to explicitly enhance the representational differences between foreground objects and background regions. This module employs... , , , Four different scales of LLSKM convolutional kernels are used to process the input feature map in parallel. By integrating multi-scale receptive fields in parallel, large-scale convolutional kernels are responsible for capturing broad contextual information, medium-scale convolutional kernels are used to extract detailed features, and small-scale convolutional kernels focus on fine local texture patterns, thereby ensuring that the network can comprehensively capture information at multiple granular levels from global context to local details.

[0012] For input medical image feature tensors First, perform multi-scale contrast convolution:

[0013] ,

[0014] ,

[0015] in, Indicates the kernel size as LLSKM convolution operation, , , , These correspond to the feature maps extracted by 9×9, 7×7, 5×5, and 3×3 convolutional kernels, respectively.

[0016] Subsequently, the contrast features extracted from these four different scales are stitched together along the channel dimension to integrate comprehensive structural and boundary information:

[0017]

[0018] in, This indicates that the splicing operation is performed at the channel level. It is the result of feature splicing.

[0019] Meanwhile, to effectively fuse multi-scale contrast information with the original input features, we adopted a fusion method based on attention weights. First, through a... Convolution pairs concatenate multi-scale features Channel compression and feature recombination are performed to map them to a more expressive space:

[0020]

[0021] in, This is a result of reducing the number of channels.

[0022] At the same time, for the original input features Also conduct one Convolution operations, acting as residual projection paths, aim to preserve the most basic and essential structural information of the image, preventing the loss of original information during the enhancement process.

[0023]

[0024] in, It is a 1×1 convolutional projection of the original input features, designed to preserve the most basic structural information of the image.

[0025] Next, the characteristics of the recombination were analyzed. By applying the Sigmoid activation function and compressing its numerical range to [0,1], a spatial attention weight map is generated. It clearly indicates which areas in the image are key target areas that need to be enhanced, and which are background or redundant areas that can be suppressed:

[0026]

[0027] in, This represents the Sigmoid activation function, which maps feature values ​​to the range [0,1] and generates spatial attention weights for feature enhancement.

[0028] Ultimately, by using this learned attention weight map Residual projection characteristics Element-wise multiplication (i.e., modulation) is performed to fuse and enhance multi-scale contrast information with the original basic features. Output features The target region was significantly enhanced, while background information was relatively suppressed.

[0029]

[0030] in, This indicates element-wise multiplication. This is the enhanced output feature of the SCEM module, which will serve as the input for the next stage.

[0031] The second stage involves direction-sensitive multi-scale feature extraction and fusion. Features enhanced by the SCEM module are then processed. Although saliency has been improved, the multi-scale characteristics and directional information of organ structures have not yet been fully captured. This phase aims to address this issue.

[0032] First, through a channel expansion block right This process enhances the network's representational capabilities, providing a richer feature base for subsequent parallel convolutional branches.

[0033]

[0034] in, This represents a channel expansion block, consisting of a 1×1 convolution, batch normalization, and a Hardswish activation function, used to expand the number of feature channels.

[0035] Then, four parallel convolutional branches are used to extract multi-scale directional features: two directional branches use and Convolutions capture horizontal and vertical edge features separately; a local context branch employs... Convolution extracts detailed information; a long-range dependency branch uses... Large kernel convolution captures global semantic information; the computation of the four branches can be represented as follows:

[0036] ,

[0037] ,

[0038] in, and Extract features in the horizontal and vertical directions respectively. Capture local context, Pay attention to long-range dependencies. This indicates batch normalization.

[0039] Next, the differential features extracted from these four branches are concatenated along the channel dimension to form a comprehensive feature representation:

[0040]

[0041] in, This indicates a channel splicing operation.

[0042] Next, a channel attention module (SE module) is introduced. This module obtains channel statistics through global average pooling, learns inter-channel dependencies using fully connected layers, generates weight vectors, normalizes them with a sigmoid function, and multiplies them with the input features to achieve adaptive channel recalibration.

[0043]

[0044] in, The Squeeze-and-Excitation module is used to enhance the expressive power of important feature channels.

[0045] Finally, to stabilize training and ensure smooth information flow, we employ a residual connection structure. Features that have undergone channel attention weighting... Through a Convolution adjusted to match SCEM output The dimensions that match the projection, and then with... of The convolution projection results are then summed element by element:

[0046]

[0047] in, The final output features include both enhanced saliency information and multi-scale and directional structural features, and avoid the gradient vanishing problem through residual connections.

[0048] 2. Hybrid Attention-Convolutional Module (HCAM)

[0049] This invention proposes a "CNN+PVT" dual encoder architecture and designs a Hybrid Attention-Convolutional Module (HCAM) to achieve deep feature fusion. This module introduces spatial attention in local branches to enhance details, applies channel attention in global branches to enhance semantic discriminativity, and retains the original feature concatenation as a supplement, thereby completing differentiated feature optimization and cross-domain fusion.

[0050] Spatial attention (for CNN features) Spatial attention aims to make the network focus on which points in a spatial location are more important (e.g., organ boundary points). First, it focuses on the features output by the CNN encoder. Max pooling and average pooling are performed separately along the channel dimension to obtain two spatial descriptive maps that highlight salient regions. These two descriptive maps are then concatenated along the channel dimension to form a 2-channel feature map. Then, a... The convolutional layers are fused together, and their large kernels help to integrate spatial information within a larger neighborhood. Finally, a spatial attention weight map is generated using the sigmoid function. Its value ranges from 0 to 1, with larger values ​​indicating more important positions.

[0051]

[0052] in, For CNN branch features, and These represent max pooling and average pooling in the channel dimension, respectively. This represents the Sigmoid activation function, which maps feature values ​​to the range [0,1].

[0053] Multiplying the spatial attention map by the original CNN features yields the spatially weighted features. In this process, features of important spatial locations (such as edges) are enhanced, while features of insignificant background regions are suppressed.

[0054]

[0055] in, This is a spatial attention weight map. The original feature map of the CNN branch. This indicates element-wise multiplication.

[0056] Channel attention (for PVT features) The goal of channel attention is to guide the network to focus on which feature channels are more important. This applies to the features output by the PVT encoder. Global average pooling (GAP) and global max pooling (GMP) are performed separately to obtain two different global channel descriptors. These two descriptors are then input into two SE subnetworks with a shared structure (containing fully connected layers and activation functions) to learn their respective channel weights. The two weight vectors are summed and then passed through a sigmoid function to obtain the final channel attention weight vector. .

[0057]

[0058] in, This is a PVT branching feature. and These represent global average pooling and global max pooling, respectively. For channel attention subnetwork, This represents the Sigmoid activation function, which maps feature values ​​to the range [0,1].

[0059] Multiplying the channel weight vector by the original PVT features yields the weighted features for each channel. Among them, important feature channels are enhanced.

[0060]

[0061] in, This is a spatial attention weight map. This is the original feature map of the PVT branch. This indicates element-wise multiplication.

[0062] Feature concatenation and deep fusion: To fully utilize all information, we integrate features from three paths. The first path is raw feature fusion: fusing raw CNN features that have not been attention-weighted. and PVT characteristics Direct splicing, and through Convolution performs dimensionality reduction and initial fusion, preserving the most original information exchange.

[0063]

[0064] in, and These are the original output features of the CNN branch and the PVT branch, respectively. This indicates that the splicing operation is performed at the channel level. It is a feature that integrates the original information from both branches.

[0065] The second and third paths are the features described above after spatial and channel attention modulation. and .

[0066] These three features are spliced ​​together along the channel dimension to form a composite feature block that integrates original information, sharpens spatial details, and enhances channel discriminability.

[0067]

[0068] in, These are CNN features modulated by spatial attention. It is a PVT feature modulated by channel attention. It is a feature that integrates the original information from both branches.

[0069] Finally, a 1×1 convolutional layer is used to concatenate the composite features. After fusion and channel dimension adjustments, the final output of the HCAM module is generated. This feature achieves efficient complementarity and deep integration of local and global, details and semantics, significantly enhancing the feature expression capability.

[0070]

[0071] in, It is the final output after passing through the HCAM module.

[0072] 3. MDFR Layered Fusion Module (MDFR-HFM)

[0073] This invention proposes a multi-scale differentiated feature optimization and hierarchical fusion module (MDFR-HFM), which adopts a hierarchical enhancement and global fusion mechanism: for features of different resolutions, the MDFR module is stacked differentially to perform multi-scale enhancement - high-resolution features enhance local details, medium-resolution features balance details and semantics, and low-resolution features focus on global distribution; the enhanced features are mapped to a high-dimensional space through a projection layer, and cross-level adaptive fusion is achieved through splicing and multi-head attention mechanism, and finally multi-scale enhanced features are output.

[0074] MDFR module (Multi-void ratio feature optimization module):

[0075] It is the core unit for feature enhancement. This module extracts multi-scale features through parallel multi-dilated convolutional branches:

[0076]

[0077]

[0078]

[0079] in, Indicates dilated convolution. They are respectively It is used to capture multi-scale features.

[0080] Then, the three branch outputs are concatenated along the channel dimension:

[0081]

[0082] in, This indicates that a splicing operation is performed at the channel level.

[0083] Next, channel compression and fusion are performed using 1×1 convolution:

[0084]

[0085] in, It is the result of feature concatenation. This is the result after reducing the number of channels.

[0086] Finally, a residual enhancement mechanism is used to output optimized features:

[0087]

[0088] in This indicates element-wise multiplication. It is the result of feature enhancement.

[0089] The enhanced features are projected and interact globally across layers with a lightweight Transformer. Let the set of preprocessed feature maps be denoted as . ,in Indicates the first Enhanced feature maps at various scales.

[0090] Unified modeling and global interaction of features across different levels:

[0091] After differential preprocessing, the features at each level have been optimally enhanced at their respective resolutions. Next, MDFR-HFM aims to deeply fuse these features at different scales within a unified framework.

[0092] First, each enhanced feature map Through an independent The convolutional layer is projected onto a unified feature dimension. This operation aims to map all features to the same high-dimensional semantic space, laying the foundation for subsequent fusion.

[0093]

[0094] in, Indicates the first Enhanced feature maps at various scales These are the features after projection.

[0095] Then, each projected feature map In its spatial dimension The sequence is flattened into a feature sequence. Then, these sequences from different levels are concatenated along the sequence length dimension to form a continuous input sequence that integrates multi-scale information across different levels. :

[0096]

[0097] in, This indicates that the feature map is in its spatial dimension Flattening is an operation on feature sequences. This indicates concatenation along the sequence length dimension.

[0098] This sequence uses a multi-head self-attention mechanism to perform cross-level global interaction, calculate the correlation between arbitrary positions, and achieve bidirectional adaptive enhancement and deep fusion of low-level details and high-level semantics.

[0099] Finally, the module's output sequence According to the dimensions of the original feature maps ( The data is then re-divided into its four corresponding parts. Each part is then processed by an independent... Convolutional layers, reducing their channel count from a uniform dimension Reprojecting back to the original number of channels The final fused feature map is obtained:

[0100]

[0101] in, This indicates the bulls' self-attention. This represents the output sequence after interaction through a multi-head self-attention mechanism. According to the original number The size of each feature map Re-segment back to the corresponding parts. This is the final fused feature map.

[0102] 4. Dual-Path Feature Fusion Decoder (DPF-Decoder)

[0103] After obtaining the MDFR-HFM fused features, the decoder needs to reconstruct the accurate segmentation map. This invention proposes a dual-path feature fusion decoder (DPF-Decoder), which uses a dual-path design of progressive fusion in each channel and direct global fusion to balance local details and global semantic consistency, effectively preventing the loss of small organ features during decoding.

[0104] Sub-channel progressive fusion path:

[0105] This approach addresses the issue of feature loss in small organs by innovatively employing a channel grouping strategy. By grouping features by channel and performing independent adaptive fusion, a protected gradient flow channel is provided for features related to small organs, effectively ensuring that features of fine structures are preserved during training.

[0106] For high-resolution feature maps from encoder jumps Compared with the low-resolution feature map from the previous stage of the decoder First, divide both into 4 groups along the channel dimension:

[0107]

[0108]

[0109] in, This represents a high-resolution feature map from the encoder's skip connections. This represents a low-resolution feature map from the previous level of the decoder.

[0110] This grouping method assumes that different channel groups may implicitly encode features of different organs or structures.

[0111] Subsequently, each corresponding feature (such as and Independent fusion is performed through an adaptive combiner. This combiner uses a learnable set of parameters. Through the Sigmoid function This is converted into weights between 0 and 1, used to control the fusion ratio of low-resolution features to high-resolution features:

[0112]

[0113] in, and These represent the low-resolution features from the previous stage of the decoder and the high-resolution features from the encoder jump connections, respectively. Group, For the Sigmoid function, These are learnable parameters.

[0114] Networks can learn parameters The dynamic decision-making process determines whether the current decoding stage and the current channel group rely more on the semantic information transmitted from the upper layer. It still relies more on the detailed information provided by the encoder. .

[0115] Ultimately, the fusion results of the four groups Then, the components are reassembled along the channel dimension to form the enhanced output features of that path. :

[0116]

[0117] in, This indicates that the splicing operation is performed at the channel level. It is the result of fusion and enhancement.

[0118] This approach is highly robust. When gradient vanishing or feature attenuation occurs in a channel group (such as the group encoding small organs), the fusion process of other groups remains unaffected and can still effectively transfer feature information, ensuring training stability and reliable preservation of small organ details.

[0119] Direct fusion path of global features:

[0120] This approach, as a supplement, focuses on the preservation and rapid fusion of global semantic information. It ensures the rationality of the final segmentation result in terms of overall structure, such as the overall shape and size of organs and the relative spatial relationships between different organs.

[0121] This approach is straightforward and efficient: it extracts the complete low-resolution feature map. With complete high-resolution encoder feature map Perform splicing directly on the channel dimension, and then immediately use one Convolutional layers perform channel reduction and fusion:

[0122]

[0123] in, and These represent the complete low-resolution feature map and the high-resolution encoder feature map, respectively. This indicates splicing at the channel dimension.

[0124] Dual-path feature integration:

[0125] The two paths, with different emphases, produced the fused features: (rich in detail) and (Semantic global). Finally, the output features of these two paths are added together through residual connections to achieve feature complementarity:

[0126]

[0127] in, It features rich, detailed channel fusion characteristics. It is a direct fusion feature that preserves global semantics. The characteristics are those after addition.

[0128] Next, LayerNorm is applied to the summed features. LayerNorm stabilizes the network training process, alleviates internal covariate bias, and eliminates potential differences in the distribution of the output features from the two paths, making the fused features more stable.

[0129]

[0130] in, Presentation layer normalization operation.

[0131] The final output feature map obtained in this way It combines local detail information from the sub-channel paths with contextual semantic information from the global paths, providing an optimal feature base for the final accurate segmentation prediction. This feature will be passed to the next level decoder for further upsampling and fusion, or used to generate the final segmentation map. Attached Figure Description

[0132] Figure 1 This is a schematic diagram of the overall network architecture proposed in this invention;

[0133] Figure 2 A detailed schematic diagram of the salient contrast enhancement module (SCEM) in the Local Structure Enhancement and Feature Extractor (LSE-FE);

[0134] Figure 3 This is a detailed schematic diagram of the Multi-Scale Oriented Convolutional Module (MSDE) in the Local Structure Enhancement and Feature Extractor (LSE-FE).

[0135] Figure 4 This is a schematic diagram of the Hybrid Attention-Convolutional Module (HCAM) structure.

[0136] Figure 5 This is a schematic diagram of the Multi-Void Ratio Feature Optimization Block (MDFR Block).

[0137] Figure 6 This is a flowchart illustrating the MDFR-HFM (MDFR Layered Fusion Module) process.

[0138] Figure 7 This is a schematic diagram of the dual-path feature fusion decoder (DPF-Decoder) structure; Detailed Implementation

[0139] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0140] Step 1: Preprocess the data used;

[0141] Step 2: Input the preprocessed data into the Local-Global Alignment Fusion Encoder (LGA-Encoder) for feature extraction;

[0142] Step 3: Fuse the output features of the corresponding layers of the dual encoders using a Hybrid Attention-Convolutional Module (HCAM);

[0143] Step 4: Input the fused multi-level features into the hierarchical fusion module (MDFR-HFM) for enhancement and unified modeling;

[0144] Step 5: Input the processed features into the dual-path feature fusion decoder (DPF-Decoder) for upsampling and reconstruction to obtain the final prediction map;

[0145] Furthermore, step 1 specifically includes the following sub-steps:

[0146] Step 1-1: Standardize Image Size. The images in the abdominal multi-organ dataset vary in size. To ensure consistency in model input, all medical images are uniformly cropped or scaled to a size of 224×224, with the expanded region set to 0.

[0147] Steps 1-2: Intensity Normalization and Enhancement. The image is normalized using the Z-score normalization method. Data augmentation operations are then performed, including random image flipping, random rotation, and brightness / contrast adjustment.

[0148] Furthermore, step 2 specifically includes the following sub-steps:

[0149] Step 2-1: As Figure 1 The diagram shows a medical image segmentation model based on dual encoders and dual-path fusion decoders. The implementation process of the Local Structural Enhancement and Feature Extractor is described in detail below. The input image first undergoes a salient contrast enhancement module (SCEM) to improve boundary contrast. The implementation process of the SCEM module is described in detail below. Figure 2 As shown, this module uses , , , Parallel extraction of multi-scale contrast features using LLSKM convolutional kernels of four scales After splicing Convolution Transformation And generate a spatial attention weight map Finally, the projection of the input features Multiply to obtain the enhanced features .

[0150]

[0151] in, This indicates element-wise multiplication. This is the enhanced output feature of the SCEM module, which will serve as the input for the next stage.

[0152] Step 2-2: Orientation-sensitive multi-scale feature extraction, details of the MSDE module are as follows: Figure 3As shown, the SCEM output features via channel expansion block The number of extended channels is obtained Then, four parallel convolutional branches are used to extract features: and Extract horizontal and vertical features. Capture local context, Focus on long-range dependencies. Output features are batch normalized (BatchNorm) and then concatenated. The final output features are then enhanced by a channel attention module (SE) and obtained via residual connections. :

[0153]

[0154] in, The final output features include both enhanced saliency information and multi-scale and directional structural features, and avoid the gradient vanishing problem through residual connections.

[0155] Steps 2-3: Global Anatomical Contextual Structure Extractor (GAS) processing. In parallel, the image is input into a Pyramid Visual Transformer (PVT). At each stage, two Transformer Blocks are stacked to model global semantic dependencies, outputting features. .

[0156] Furthermore, step 3 specifically includes the following sub-steps:

[0157] Step 3-1: Feature modulation, such as Figure 4 As shown, the features output by the CNN encoder Features of PVT encoder output At each level, interaction and fusion are achieved through Hybrid Attention-Convolutional Modules (HCAM). This applies to CNN features. Apply spatial attention to generate a spatial weighted graph. Multiplying this by the given value yields the spatial sharpening feature. .

[0158]

[0159] in, This is a spatial attention weight map. The original feature map of the CNN branch. This indicates element-wise multiplication.

[0160] PVT characteristics Apply channel attention to generate channel weight vectors Multiplying this by the channel enhancement feature yields the channel enhancement feature. .

[0161]

[0162] in, This is a spatial attention weight map. This is the original feature map of the PVT branch. This indicates element-wise multiplication.

[0163] Step 3-2: Deep Fusion. This involves combining the original features... and splicing and passing Convolutional fusion Finally, , , The three features are concatenated along the channel dimension and then... The convolutional layers are fused to generate the final output of the HCAM module. .

[0164]

[0165] in, It is the final output after passing through the HCAM module.

[0166] Furthermore, step 4 specifically includes the following sub-steps:

[0167] Step 4-1: Differentiated MDFR Enhancement, where the implementation process of MDFR is described in detail as follows: Figure 5 As shown. Enhancement is performed using the MDFR module based on feature map resolution differences: for high-resolution feature maps (such as...) Stack 4 MDFR modules; for medium-resolution feature maps (such as...) Configure two MDFR modules; for low-resolution feature maps (such as...) , Each uses one MDFR module. Each MDFR module is tested for porosity. Three parallel dilated convolution branches extract multi-scale features, which are then concatenated and fused for dimensionality reduction. Finally, the optimized features are output through the residual enhancement mechanism. .

[0168]

[0169] in This indicates element-wise multiplication. It is the result of fusion and dimensionality reduction. It is the result of feature enhancement.

[0170] The enhanced feature map set is ,in Indicates the first Enhanced feature maps at various scales.

[0171] Step 4-2: Cross-level global interaction, its implementation process is described in detail as follows: Figure 6 The latter half shows the enhanced feature maps at various scales. Through independent Convolutional layers projected to a unified high-dimensional space ,get All projected feature maps are flattened and concatenated along the sequence length dimension to form a cross-level feature sequence. A multi-head self-attention mechanism is used to model the global context. Finally, the output sequence after interaction is... Re-segment to the original dimensions and reproject back to the original number of channels. The final fused feature map is obtained. .

[0172]

[0173] in, This represents the output sequence after interaction through a multi-head self-attention mechanism. According to the original number The size of each feature map Re-segment back to the corresponding parts. This is the final fused feature map.

[0174] Furthermore, step 5 specifically includes the following sub-steps:

[0175] Step 5-1: Dual-path fusion decoding, such as Figure 7 As shown, the multi-level features output from the MDFR-HFM module are input into the dual-path feature fusion decoder (DPF-Decoder). For each decoding layer, high-resolution features from the encoder are hopped. Low-resolution features from the previous decoding layer To integrate.

[0176] Step 5-2: Sub-channel progressive fusion. (The text abruptly ends here.) and Features are divided into four groups along the channel dimension, and each group's features are independently fused using an adaptive combiner. The weights are determined by learnable parameters. via the Sigmoid function The generated features are then concatenated to form the output path. .

[0177]

[0178] in, This indicates that the splicing operation is performed at the channel level. It is the result of fusion and enhancement.

[0179] Step 5-3: Direct fusion of global features. This involves fusing the complete... and Direct splicing, and through a The convolutional layers are fused to obtain features that preserve global semantics. .

[0180]

[0181] in, and These represent the complete low-resolution feature map and the high-resolution encoder feature map, respectively. This indicates splicing at the channel dimension.

[0182] Step 5-4: Dual-path integration and prediction. The output features of the two paths are added together, and the training process is stabilized using layer normalization (LayerNorm) to obtain the final decoder output features. .

[0183]

[0184] in, Presentation layer normalization operation.

[0185] Step 5-5: This feature The image is then passed to the next level decoder or restored to its original resolution via the final upsampling layer. Convolution and the Softmax function generate the final segmentation prediction map.

[0186] To verify the effectiveness of the model, we conducted comparative experiments with several state-of-the-art methods on the Synapse multi-organ dataset. This dataset comes from the MICCAI 2015 Multi-Atlas Abdominal Annotation Challenge and includes 30 abdominal CT scans, totaling 3,779 512×512 axial slices with voxel sizes of ([0.54–0.54]×[0.98–0.98]×[2.5–5.0]) mm³, divided into 18 training samples and 12 test samples.

[0187] Experimental results on the Synapse dataset show that the proposed method outperforms the comparison methods in overall performance and most individual results across eight abdominal organ segmentation tasks, including classic and hybrid architectures such as U-Net, SwinUnet, TransUNet, ParaTransCNN, PVT-EMCAD, CSWin-UNet, and MSLAU-Net. Specific quantitative results are shown in Table I.

[0188] Table 1 Comparison of Synapse dataset segmentation metrics

[0189]

Claims

1. A medical image segmentation method based on dual encoders and dual-path fusion decoders, characterized in that: Includes the following steps: S1. Preprocess the medical image data used; S2. Input the preprocessed data into the dual encoder for feature extraction. This encoder is called the Local-Global Alignment Fusion Encoder. The dual encoder includes a Local Structural Enhancement and Feature Extractor and a Global Anatomical Contextual Structure Extractor. S3. The output features of the corresponding layer of the Local-Global Alignment Fusion Encoder are fused using the Hybrid Attention-Convolutional Module (HCAM). S4. Input the fused multi-level features into the hierarchical fusion module (MDFR-HFM) for enhancement and unified modeling; S5. Input the processed features into the dual-path feature fusion decoder (DPF-Decoder) for upsampling and reconstruction to obtain the final prediction map.

2. The medical image segmentation method based on dual encoders and dual-path fusion decoders according to claim 1, characterized in that, Step S1 includes the following sub-steps: S1.1 Crop or scale the input image uniformly to... Size, expansion area filled with 0; S1.2 Process the image Intensity normalization, and data augmentation operations including random flipping, rotation, brightness and contrast adjustments.

3. The medical image segmentation method based on dual encoders and dual-path fusion decoders according to claim 1, characterized in that, Step S2 includes the following sub-steps: The S2.1 Local-Global Alignment Fusion Encoder (Local Structural Enhancement and Feature Extractor) consists of two modules. First, the SCEM module extracts contrast features using multi-scale LLSKM convolutional kernels to generate a spatial attention weight map. and the projection of the input features Multiplication yields enhanced features. : ; in, This indicates element-wise multiplication. This is the enhanced output feature of the SCEM module, which will serve as the input for the next stage; S2.2 The results enhanced by the SCEM module are further processed by the MSDE module to enhance the features. Channel expansion is performed to obtain Features are extracted through multiple parallel convolutional branches, and then concatenated after batch normalization to obtain the final product. The final output features are then enhanced by a channel attention module (SE) and obtained via residual connections. : ; in, The final output features include both enhanced saliency information and multi-scale and directional structural features, and the gradient vanishing problem is avoided through residual connections. S2.3 In parallel, the image is input into the Global Anatomical Contextual Structure Extractor, which models global semantic dependencies by stacking Transformer Blocks and outputs global features. .

4. The medical image segmentation method based on dual encoders and dual-path fusion decoders according to claim 1, characterized in that, Step S3 includes the following sub-steps: S3.1 Local structure enhancement and feature extractor output features Apply spatial attention to generate a spatial weighted graph. Multiplying this by the given value yields the spatial sharpening feature. : ; in, This is a spatial attention weight map. The original feature map of the CNN branch. This indicates element-wise multiplication; S3.2 Output features of the global anatomical context structure extractor Apply channel attention to generate channel weight vectors Multiplying this by the channel enhancement feature yields the channel enhancement characteristic. : ; in, This is a spatial attention weight map. This is the original feature map of the PVT branch. This indicates element-wise multiplication; S3.3 Original features and The splicing and fusion using 1×1 convolution are obtained Ultimately , , The three features are concatenated along the channel dimension and then output as a fused feature through a 1×1 convolution. : ; in, It is the final output after passing through the HCAM module.

5. The medical image segmentation method based on dual encoders and dual-path fusion decoders according to claim 1, characterized in that, Step S4 includes the following sub-steps: S4.1 Enhancement is performed using the MDFR module based on feature map resolution differences: for high-resolution feature maps (such as...) Stack 4 MDFR modules; for medium-resolution feature maps (such as...) Configure two MDFR modules; for low-resolution feature maps (such as...) , Each uses one MDFR module, and each MDFR module is tested for void ratio. Three parallel dilated convolution branches extract multi-scale features, which are then concatenated and fused for dimensionality reduction. Finally, the optimized features are output through the residual enhancement mechanism. ; ; in This indicates element-wise multiplication. It is the result of fusion and dimensionality reduction. It is the result of feature enhancement; S4.2 Enhanced feature maps at various scales pass Convolutional projection to higher-dimensional space get Expand and splice them into cross-level feature sequences The input is a multi-head self-attention mechanism for global context modeling, and the output is a sequence. Re-segment to the original dimensions and reproject back to the original number of channels. The final fused feature map is obtained. : ; in, This represents the output sequence after interaction through a multi-head self-attention mechanism. According to the original number The size of each feature map Re-segment back to the corresponding parts. This is the final fused feature map.

6. The medical image segmentation method based on dual encoders and dual-path fusion decoders according to claim 1, characterized in that, Step S5 includes the following sub-steps: S5.1 inputs the multi-level features output from the MDFR-HFM module into the dual-path feature fusion decoder (DPF-Decoder). For each decoding layer, the high-resolution features from the encoder are... Low-resolution features from the previous decoding layer To merge; S5.2 will and The data is evenly divided into 4 groups along the channel dimension, and each group is independently fused by an adaptive combiner. The weights are determined by learnable parameters. via the Sigmoid function The generated and fused feature sets are then concatenated to form the output of this path. : ; in, This indicates that the splicing operation is performed at the channel level. It is the result of fusion enhancement; S5.3 will be complete and The features are directly concatenated and fused using 1×1 convolutions to obtain features that preserve global semantics. : ; in, and These represent the complete low-resolution feature map and the high-resolution encoder feature map, respectively. This indicates splicing at the channel level; S5.4 The output features of the two paths are added together, and the training process is stabilized by layer normalization (LayerNorm) to obtain the final decoder output features. : ; in, Presentation layer normalization operation; S5.5 will The image is passed to the next level decoder or restored to the original resolution through the final upsampling layer, and then the final segmentation prediction map is generated by 1×1 convolution and the Softmax function.

Images