A medical image segmentation method with dual-branch cross-layer cooperation and gradual boundary constraint

Abstract

The application relates to the technical field of medical image segmentation, and discloses a medical image segmentation method based on double-branch cross-layer cooperation and gradual boundary constraint. The method comprises the following steps: inputting a pretreated medical image into a CNN-KAN encoding branch and a hierarchical Swin Transformer encoding branch respectively; in a shallow layer stage, features are independently extracted by the double branches, and a learnable anti-aliasing process is performed before down-sampling of the CNN-KAN encoding branch; in a deep layer stage, local semantic features and global semantic features of the same scale are subjected to gate-weighted fusion. The fusion coding features in the deep layer are used as the starting features for decoding, boundary priori is injected by using a boundary learner and a boundary guiding module in the subsequent three decoding stages, and a segmentation prediction result is generated. In the training stage, a joint loss function is constructed based on a regional main body segmentation loss and a boundary positioning loss, and the weight of the boundary positioning loss is gradually increased, so that the segmentation accuracy and the boundary positioning stability are improved.

Description

Technical Field

[0001] This invention relates to the field of medical image segmentation technology, specifically to a medical image segmentation method with bi-branch cross-layer collaboration and progressive boundary constraints. Background Technology

[0002] Medical image segmentation is used to identify lesion regions, organ structures, or target tissues from medical images such as ultrasound, MRI, endoscopy, dermatoscopy, and pathological tissue sections, and to obtain pixel-level segmentation results. It is a key step in computer-aided diagnosis, quantitative analysis, treatment planning, and prognostic assessment. Compared with natural scene images, medical images typically have characteristics such as small target area size, complex background noise, low foreground-background contrast, blurred tissue boundaries, irregular lesion morphology, and significant cross-modal imaging differences.

[0003] The existing U-shaped encoder-decoder structure based on convolutional neural networks is widely used in medical image segmentation tasks. However, due to the limitation of the local receptive field of convolution, it is insufficient in modeling the global contextual relationships and long-distance dependencies between anatomical structures. In scenarios with significant multi-scale changes, blurred boundaries, or complex tissue structures, problems such as target region breakage, inaccurate boundary localization, or background missegmentation are prone to occur.

[0004] While Vision Transformer-based medical image segmentation methods can establish long-distance spatial dependencies through self-attention mechanisms, the image segmentation and feature embedding processes in these methods can easily weaken pixel-level spatial continuity, leading to insufficient representation of small lesions, local textures, and boundary details. For lesion regions with low contrast, irregular shapes, or blurred boundaries, relying solely on global semantic representation may still result in contour smoothing and boundary localization errors.

[0005] To balance local details and global context, existing methods have proposed hybrid architectures combining CNNs and Transformers. Sequential hybrid architectures typically extract low-level features first using convolutional networks, then introduce Transformers at deeper or bottleneck layers for global semantic enhancement. However, unidirectional information flow may disrupt the continuous transmission of multi-scale spatial features between network layers. While parallel dual-branch architectures can extract both local and global features simultaneously, common fusion methods such as pixel-by-pixel addition or channel concatenation ignore the distributional differences between the two heterogeneous representation spaces, easily leading to semantic conflicts and the propagation of redundant background noise.

[0006] Furthermore, during the downsampling process of the convolutional coding branch, high-frequency structural information may be aliased, resulting in boundary misalignment, blurred subtle structures, or morphological distortion in the spatial domain. In boundary-supervised training, if overly strong boundary supervision is used in the early stages of model training, it can easily lead to gradient oscillations or the model focusing excessively on noisy boundaries; conversely, if boundary supervision is too weak, it is difficult to drive the model to learn complex boundaries.

[0007] Therefore, given the obvious shortcomings of existing models in dealing with multi-scale changes in lesions, complex background interference, and fuzzy boundary localization, there is an urgent need in this field for a medical image segmentation method that can adaptively fuse heterogeneous representation spatial features, suppress downsampling aliasing, and stably strengthen boundary constraints. Summary of the Invention

[0008] The purpose of this invention is to provide a medical image segmentation method with dual-branch cross-layer collaboration and progressive boundary constraints, in order to solve the technical problems in existing medical image segmentation methods, such as difficulty in taking into account both local details and global semantics, easy introduction of noise by heterogeneous feature fusion, boundary degradation caused by downsampling aliasing, and unstable boundary supervision training.

[0009] The method includes a dual-branch encoding step, a hierarchical collaborative step, a boundary-guided decoding step, and a progressive boundary-supervised optimization step. First, the medical image to be segmented is acquired and preprocessed. The preprocessed medical image is input into a CNN-KAN encoding branch and a hierarchical Swing Transformer encoding branch to extract local and global encoding features, respectively. In the shallow encoding stage, the two encoding branches extract features independently. Before downsampling in the CNN-KAN encoding branch, a learnable anti-aliasing module performs frequency domain decomposition and dynamic weighting on the downsampled encoding features. In the deep encoding stage, the same-scale features output by the two encoding branches are gated and weighted and fused. The decoder upsamples the deep fused encoding features step-by-step. A boundary learner generates boundary prior information based on the shallow and deep encoding features. In the three decoding stages (excluding the initial decoding stage), the boundary prior information is injected into the decoding features using a boundary-guided module to obtain decoded features with fused boundary information. Segmentation prediction results are generated based on the decoded features with fused boundary information. In the model training stage, a joint loss function is constructed based on the region subject segmentation loss and the boundary localization loss.

[0010] In the dual-branch encoding step, the CNN-KAN encoding branch is used to extract local spatial features through convolutional encoding units and to perform nonlinear feature mapping on the token sequence transformed by convolutional features through KAN feature mapping units; the hierarchical Swing Transformer encoding branch is used to extract global contextual semantics through alternating stacking of window self-attention and shift window self-attention.

[0011] In the shallow encoding stage, the CNN-KAN encoding branch and the hierarchical Swing Transformer encoding branch extract features independently to preserve local spatial details and boundary structures. In the deep encoding stage, the local semantic features output by the CNN-KAN encoding branch and the global semantic features output by the hierarchical Swing Transformer encoding branch are aligned in terms of spatial size and number of channels. A lightweight gated attention module is then used to adaptively weight and fuse the aligned heterogeneous features at the same scale to obtain deep fused encoded features.

[0012] To suppress spectral aliasing caused by downsampling and preserve high-frequency structural information related to the target contour, this invention introduces a learnable anti-aliasing module. This module is set before the downsampling operation of each convolutional coding unit in the CNN-KAN coding branch: it transforms the coding features to be downsampled to the frequency domain, uses multiple binary masks to divide the frequency components, and dynamically weights the features of each frequency band through a learnable selection map.

[0013] The boundary learner integrates shallow spatial features and deep semantic features to generate single-channel boundary features. These single-channel boundary features serve two purposes: firstly, they are used as boundary prediction results in boundary localization loss calculations; secondly, they are used as explicit boundary prior inputs to the boundary guidance module. The boundary guidance module injects the boundary prior into the decoded features through element-wise multiplication, residual connections, multi-scale dilated convolutional branches, and local channel attention mechanisms.

[0014] During the training phase, the joint loss function includes the region subject segmentation loss and the boundary localization loss. In the progressive boundary supervision optimization step, the weight of the boundary localization loss is gradually increased from zero to a preset target weight with each training epoch. This allows the model to prioritize learning the distribution of the target region's subjects in the early stages of training, and gradually strengthen the complex boundary details in the later stages of training, thereby improving training stability and boundary localization accuracy.

[0015] Compared with existing technologies, this invention has at least the following beneficial effects: First, the CNN-KAN encoding branch and the hierarchical Swing Transformer encoding branch work in parallel, enabling the model to balance local detail fidelity and global context awareness; Second, the combination of shallow independent extraction and deep gating fusion can reduce shallow semantic conflicts and achieve complementary advantages in deep layers; Third, the lightweight gating attention module achieves adaptive fusion of heterogeneous features with low computational overhead and can filter background redundant noise in deep skip connections; Fourth, the learnable anti-aliasing module can suppress aliasing artifacts caused by downsampling; Fifth, the boundary learner and boundary guidance module can couple explicit boundary priors to the decoding and recovery process; Sixth, the dynamic progressive boundary supervision strategy can alleviate the optimization instability caused by excessive boundary supervision in the early stages of training. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall process of a medical image segmentation method with dual-branch cross-layer collaboration and progressive boundary constraints provided in an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the lightweight gating attention module provided in an embodiment of the present invention.

[0018] Figure 3 This is a schematic diagram of the boundary guidance module provided in an embodiment of the present invention.

[0019] Figure 4 This is a visual comparison of the segmentation effects of embodiments of the present invention with other medical image segmentation methods. Detailed Implementation

[0020] To make the technical solution, objectives, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] like Figure 1 As shown, this embodiment provides a medical image segmentation method with dual-branch cross-layer collaboration and progressive boundary constraints. After acquiring the medical image to be segmented, the medical image and its corresponding annotations are subjected to size unification processing; for histopathological slide images, the image and annotations are scaled to 512×512 pixels, and for breast ultrasound images, dermoscopy images, endoscopic polyp images, and two-dimensional slices converted from three-dimensional prostate MRI images, the image and annotations are scaled to 256×256 pixels. During the model training phase, the training image and its corresponding annotations are simultaneously subjected to random rotation and random flipping to obtain preprocessed medical images. Subsequently, the preprocessed medical images are input into the CNN-KAN encoding branch and the hierarchical Swin Transformer encoding branch, respectively; in the shallow encoding stage, the two branches extract features independently, and in the deep encoding stage, heterogeneous features of the same scale are fused through a lightweight gating attention module; a learnable anti-aliasing module is deployed before the convolutional encoding unit of the CNN-KAN encoding branch performs downsampling operation; in the decoding stage, an explicit boundary prior is coupled to the decoding process using a boundary learner and a boundary guidance module, and the segmentation prediction result is output.

[0022] In the shallow encoding stage, the CNN-KAN encoding branch preserves local spatial details and fine boundaries, while the hierarchical Swing Transformer encoding branch independently establishes intra-window and cross-window contextual relationships. No feature fusion occurs between the two branches. In the deep encoding stage, the convolutional encoded features output from the CNN-KAN encoding branch are unfolded spatially and projected as a token sequence. Figure 1The tokenized KAN module in the CNN-KAN encoding branch performs non-linear feature mapping to obtain local semantic features, which are then input into the lightweight gated attention module (LGAM). Figure 2 As shown, for the first Each deep coding stage (of which) Local semantic features after aligning spatial dimensions and channel dimensions With global semantic features By stitching along the channel dimension, the stitching feature is obtained. :

[0023] ,

[0024] in, This indicates a channel dimension splicing operation. Figure 2 In It is a lightweight gating network used to generate gating weight maps. It concatenates features. Input a lightweight gating network and generate a gating weight graph. Its expression is:

[0025] ,

[0026] in, This represents the Sigmoid activation function. Represents a non-linear activation function. Indicates batch normalization, and They represent Convolution and convolution; Indicates the first The gating weights of each deep coding stage are defined, and their values ​​range from [0,1].

[0027] According to the gating weight diagram The two features are then fused element-wise to obtain the deep fusion encoded features. :

[0028] ,

[0029] in, This indicates an element-wise multiplication operation. Indicates the first The final fusion feature of each deep coding stage; This indicates that the calculations have been performed by the CBR module, which sequentially includes... The feature processing units consist of convolutional layers, batch normalization layers, and ReLU activation functions. Furthermore, this lightweight gated attention module is deployed at deep skip connections between the dual-branch hybrid encoder and decoder to adaptively filter cross-layer features, intercepting background redundancy noise and enhancing salient features relevant to the target region.

[0030] In the CNN-KAN coding branch, a learnable anti-aliasing module is placed before the downsampling operation of each convolutional coding unit, constituting... Figure 1 The anti-aliasing encoder in the module. This module first samples the features to be downsampled and encoded. Through Fast Fourier Transform Transform to the frequency domain and utilize multiple binary masks. Extract the corresponding frequency components, and then perform an inverse fast Fourier transform. Obtain frequency band characteristics Its expression is:

[0031] ,

[0032] in, Indicates the first Encoded features to be downsampled in each coding stage Indicates the first The binary mask corresponding to each frequency band and , Indicates the first In the coding stage, the first Frequency band characteristics of each frequency band.

[0033] To divide different frequency components, a set of preset frequency thresholds is established. for:

[0034] ,

[0035] in, Represents a set of frequency thresholds; The value is 0. to This indicates the intermediate threshold set in increments of the frequency radius; Indicates the number of frequency bands; This represents the upper limit of the normalized frequency radius.

[0036] The horizontal and vertical coordinates of a point in the frequency domain are denoted as follows: and Then the binary mask Determined according to the following rules:

[0037]

[0038] in, and Representing frequency domain coordinates respectively; Represents a normalized frequency distance metric; and These represent adjacent frequency thresholds; when the frequency radius falls within the corresponding threshold interval... The value is 1, otherwise The value is 0.

[0039] The learnable anti-aliasing module inputs the features of each frequency band into grouped convolution and the sigmoid activation function respectively, to obtain the learnable selection map of the corresponding frequency band. The learnable selection map is multiplied element-wise with the corresponding frequency band features, and the features of each frequency band are then processed. We perform a weighted summation to obtain the dealiasing encoded features. Its expression is:

[0040] ,

[0041] in, Represents spatial domain coordinates, For the first Learnable selection map for each frequency band This is the dealiasing coding feature output after frequency domain dynamic weighting. Through the above frequency domain decomposition and learnable weighting, the model can suppress aliasing interference caused by high-frequency structural folding before performing spatial downsampling.

[0042] Boundary learner (BL) encodes features from shallow layers that contain spatial details. and deep encoded features containing semantic location information The learning target boundary prior is performed. Deep coding features are upsampled to a spatial size matching shallow coding features, then concatenated with shallow coding features along the channel dimension, and finally transformed to obtain single-channel boundary features. Its expression is:

[0043] ,

[0044] in, This represents a feature transformation function consisting of at least two CBR modules. Single-channel boundary features. On the one hand, it serves as the boundary prediction result for boundary localization loss calculation; on the other hand, it serves as the explicit boundary prior input for the boundary guidance module.

[0045] The deep fusion coding features output from the fourth coding stage of the dual-branch hybrid coding network are used as the decoding starting features. No boundary guidance module is set, and the third decoding stage, the second decoding stage, and the first decoding stage are executed in sequence according to the increasing spatial resolution.

[0046] like Figure 3 As shown, the Boundary Guidance Module (BGM) performs a boundary injection operation during the decoding phase. In the... In the decoding stage, the decoded features output from the previous stage are upsampled and fused with the encoded features at the corresponding scale using skip connections to obtain the features to be guided in the current decoding stage. The single-channel boundary features Downsampling to the feature to be guided Matching spatial dimensions yields downsampled boundary features. Preliminary fusion features are obtained through element-wise multiplication, residual connection, and CBR module. :

[0047] ,

[0048] in, This indicates a downsampling operation. This indicates element-wise addition, which is used to form residual connections. Indicates the first Decoding features to be guided in each decoding stage. This indicates that the calculations have been performed by the CBR module. Indicates the first The initial fusion features are injected after boundary priors in each decoding stage.

[0049] Subsequently, the preliminary fusion features The input is a multi-scale context extraction structure, which includes multiple parallel branches, each of which includes a CBR module and... The dilated convolutional layer has dilation rates of 1, 3, and 5 for different parallel branches. The outputs of each parallel branch are concatenated and then processed by a CBR module and a local channel attention mechanism to obtain the decoded features that fuse boundary information.

[0050] During the model training phase, the current training round is set to... ,and Starting from 0, the preset warm-up training rounds are: The objective weight of the boundary loss is Then the current round Dynamic boundary loss weights satisfy:

[0051] ,

[0052] in, This indicates that the function takes the smaller value. This setting allows boundary supervision to gradually increase from weak to strong in the early stages of training, thereby reducing the interference of noisy boundaries on the learning of the main region.

[0053] Joint loss function Loss due to regional subject segmentation and boundary positioning loss Composition, its expression is:

[0054] ,

[0055] Among them, regional subject segmentation loss and boundary positioning loss They respectively satisfy:

[0056] ,

[0057] ,

[0058] in, This indicates Dice's loss. Represents the binary cross-entropy loss. This represents the weighting coefficient in the regional subject segmentation loss. and These represent the segmentation prediction result and the segmentation ground truth, respectively. and These represent the boundary prediction result and the true boundary value, respectively. In a specific embodiment, Set the value to 0.5 and preset the number of warm-up training rounds. Set to 10, boundary loss target weight Set to 1. The boundary truth value is obtained from the artificial boundary annotations corresponding to the medical image.

[0059] In one training example, a medical image segmentation network was built using Python 3.8 and PyTorch 2.3 and trained using a graphics processor supporting CUDA 12.8. The training batch size was set to 8, and the initial learning rate was set to... The optimizer uses the Adam optimizer, and the learning rate scheduling uses a cosine annealing strategy. The minimum learning rate is set to... The training rounds are set to 200 rounds.

[0060] To verify the effectiveness of the proposed method in medical image segmentation, training and testing were conducted using breast ultrasound datasets, histopathological slide datasets, dermoscopy datasets, prostate MRI datasets, and multiple endoscopic polyp image datasets. The segmentation results are visualized as follows: Figure 4 As shown, the method proposed in this invention is compared with mainstream medical image segmentation methods. The results show that the medical image segmentation method with bi-branch cross-layer collaboration and progressive boundary constraints proposed in this invention is superior to other methods.

[0061] In this embodiment, for three-dimensional medical images, they can be converted into two-dimensional slices and then input into the medical image segmentation network. For different medical image modalities, the input size, training rounds and loss weights can be adjusted according to the image resolution, target region size and annotation method, but the overall technical approach of cross-layer collaboration of heterogeneous features of two branches, learnable anti-aliasing, boundary-guided decoding and dynamic progressive boundary supervision is not changed.

[0062] The method of this invention can be applied to medical imaging modalities such as breast ultrasound images, dermoscopy images, endoscopic polyp images, histopathological section images, or prostate magnetic resonance images.

Claims

1. A medical image segmentation method with dual-branch cross-layer collaboration and progressive boundary constraints, characterized in that, Includes the following steps: Dual-branch encoding steps: Acquire the medical image to be segmented and preprocess it. Input the preprocessed medical image into the CNN-KAN encoding branch and the hierarchical Swing Transformer encoding branch respectively for feature extraction to obtain local encoding features and global encoding features. Layered collaborative steps: In the shallow encoding stage, the CNN-KAN encoding branch and the hierarchical SwinTransformer encoding branch extract features independently; before the CNN-KAN encoding branch performs downsampling, an anti-aliasing encoder is constructed using a learnable anti-aliasing module to perform frequency domain decomposition and dynamic weighting on the features to be downsampled; in the deep encoding stage, the same-scale features output by the two encoding branches are gated and weighted and fused. Boundary-guided decoding steps: The decoder upsamples the deep fused coding features step by step. The boundary learner generates boundary prior information based on the shallow and deep coding features. In the three decoding stages other than the initial decoding stage, the boundary guidance module injects the boundary prior information into the decoding features to obtain the decoded features with fused boundary information. The segmentation prediction result is generated based on the decoded features with fused boundary information. Progressive boundary supervision optimization steps: During the model training phase, a joint loss function is constructed based on the region subject segmentation loss and the boundary localization loss. During the training process, the weight of the boundary localization loss is gradually increased and the model parameters are optimized according to the joint loss function. The learnable anti-aliasing module includes: performing a Fast Fourier Transform (FFT) on the features to be downsampled to obtain frequency domain features; extracting frequency components of different frequency bands from the frequency domain features according to a preset frequency threshold and multiple binary masks, and obtaining multiple frequency band features by performing an Inverse Fast Fourier Transform (IFT) on each; and then... Grouped convolution and sigmoid activation function are used to obtain learnable selection maps corresponding to each frequency band feature. The learnable selection maps are then used to dynamically weight and sum multiple frequency band features element by element to obtain dealiasing coding features. The dealiasing coding features are then input into the downsampling operation of the corresponding convolutional coding unit. The process by which the boundary learner generates the boundary prior information includes: selecting the shallow encoded features containing spatial details and the deep encoded features containing semantic positioning information; upsampling the deep encoded features to the same spatial size as the shallow encoded features and then concatenating them with the shallow encoded features along the channel dimension; and sequentially inputting the concatenated features into two CBR modules. A convolutional layer and a sigmoid function are used to obtain single-channel boundary features. These single-channel boundary features are then used as the boundary prediction result and input as the boundary prior information into the boundary guidance module. The CBR module sequentially includes... Feature processing units consisting of convolutional layers, batch normalization layers, and ReLU activation functions.

2. The medical image segmentation method according to claim 1, characterized in that, The dual-branch coding step specifically includes: The CNN-KAN coding branch is used to extract local spatial features through the convolutional coding unit and to perform nonlinear feature mapping on the token sequence transformed by the convolutional features through the KAN feature mapping unit. The hierarchical Swin Transformer encoding branch is used to extract global contextual features by alternating between window self-attention and shifted window self-attention.

3. The medical image segmentation method according to claim 1, characterized in that, The shallow coding stage includes a first coding stage and a second coding stage, and the deep coding stage includes a third coding stage and a fourth coding stage; In the first and second encoding stages, no feature fusion is performed between the CNN-KAN encoding branch and the hierarchical SwinTransformer encoding branch; In the third and fourth encoding stages, the convolutional encoding features output by the CNN-KAN encoding branch are flattened in the spatial dimension and projected into a token sequence. The KAN feature mapping unit performs nonlinear feature mapping to obtain local semantic features. The lightweight gated attention module is then used to perform gated weighted fusion of the local semantic features and the global semantic features output by the hierarchical SwinTransformer encoding branch in the same stage.

4. The medical image segmentation method according to claim 3, characterized in that, The lightweight gating attention module performs the following processing: The local semantic features and the global semantic features, which are aligned by spatial size and number of channels, are concatenated along the channel dimension to obtain the concatenated features; Passing in sequence Convolutional layer, batch normalization layer, ReLU activation function Convolutional layers and the Sigmoid activation function process the concatenated features to obtain values ​​ranging from 1 to 10. Gating weights graph; Using the complementary weight map and the gated weight map, the local semantic features and the global semantic features are summed element-wise to obtain the deep fusion coding features. The lightweight gating attention module is also located at the deep skip connection between the dual-branch hybrid coding network and the decoder, and is used to adaptively filter the encoded features transmitted through the deep skip connection.

5. The medical image segmentation method according to claim 1, characterized in that, The process by which the boundary guidance module performs the boundary prior information injection includes: By fusing encoded and decoded features of the same scale through skip connections, the feature to be guided is obtained; the boundary prior information adjusted to the corresponding spatial size is multiplied element-wise with the feature to be guided, and a preliminary fused feature is obtained through residual connections and a CBR module; The preliminary fused features are input into a multi-scale context extraction structure, which includes multiple parallel branches. Each parallel branch includes a CBR module and a dilated convolutional layer, and the dilated convolutional dilation rates of the multiple parallel branches are 1, 3 and 5, respectively. The outputs of multiple parallel branches are concatenated along the channel dimension, and then decoded features of the fused boundary information are obtained through the CBR module and the local channel attention mechanism.

6. The medical image segmentation method according to claim 1, characterized in that, The asymptotic boundary supervision optimization step specifically includes: The segmentation loss of the main body of the region is determined based on the segmentation prediction results and the segmentation ground truth, and the boundary localization loss is determined based on the boundary prediction results and the boundary ground truth output by the boundary learner. The main body segmentation loss of the region is added to the boundary localization loss adjusted by the dynamic boundary loss weight to obtain the joint loss function, and the model parameters of the medical image segmentation network are updated according to the joint loss function.

7. The medical image segmentation method according to claim 6, characterized in that, The main body segmentation loss of the region includes the Dice loss between the segmentation prediction result and the segmentation ground truth and the binary cross-entropy loss, and the boundary localization loss includes the Dice loss between the boundary prediction result and the boundary ground truth. The weight of the boundary localization loss gradually increases from zero to the target weight during the preset warm-up training rounds, and remains unchanged after the target weight is reached.

Images