A medical image segmentation method based on boundary perception and bidirectional fusion

Abstract

The present invention discloses a medical image segmentation method based on boundary perception and bidirectional fusion. A data set is constructed according to a target task. A segmentation network model is built and trained. A dual-branch encoding and decoding structure is adopted, including an encoding module and a decoding module. The encoding module is used to encode an input image through a first encoding branch and a second encoding branch connected by a bidirectional fusion module. After the encoded image output by the previous layer is bidirectionally fused by the corresponding layer in the bidirectional fusion module, the first fused image and the second fused image are correspondingly output as the input of the next level. The decoding module is used to decode the encoded image, including a first decoding module and a second decoding module. The first decoding module includes a first decoding branch and a second decoding branch corresponding to the first encoding branch and the second encoding branch, and is used to realize initial decoding. The second decoding module is used to realize final decoding to complete image segmentation. Segmentation is performed using a trained segmentation network model.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a medical image segmentation method based on boundary perception and bidirectional fusion. Background Art

[0002] Dermoscopy is a non-invasive imaging technique used to detect and diagnose skin diseases such as melanoma and is generally considered the gold standard for early localization of skin lesions. Traditionally, malignant skin lesions can be manually marked and examined by experienced dermatologists, but this process is extremely tedious and time-consuming. Therefore, automatic skin lesion segmentation in computer-aided diagnosis (CAD) has been widely designed to help dermatologists analyze malignant skin diseases such as melanoma. For example, key information can be extracted from digital dermoscopic images, which can help doctors perform tasks such as quantitative analysis of lesion shape, diagnosis, and localization of lesions.

[0003] In recent years, various convolutional neural network (CNN)-based algorithms have performed well in skin lesion segmentation tasks due to their inherent advantages in local perception. However, under complex conditions where the segmentation targets have irregular shapes and varying scales, the CNN encoding process inevitably leads to the loss of positional and global contextual information, and the successive decoding operations cannot fully recover the spatial information of low-level details, which impairs the pixel assignment in skin lesion segmentation. Therefore, learning the long-range correlations between pixels in medical images, especially the correlations between lesion pixels and surrounding healthy skin pixels, plays a crucial role in improving the accuracy of skin lesion segmentation. Summary of the Invention

[0004] The present invention provides a medical image segmentation method based on boundary perception and bidirectional fusion, which uses a dual-branch parallel encoder structure. The Vision Transformer model of the first encoding branch has good global representation ability due to the advantages of the self-attention mechanism, which makes up for the limited global perception ability of the second encoding branch CNN model. The two can complement each other and complement each other's advantages. At the same time, a bidirectional fusion module is used to interactively fuse the features of the two modalities, which can maximize the retention of local information and have good global representation ability. Finally, the boundary perception decoder and the two decoders corresponding to the dual-branch encoder are used to obtain the final image segmentation result, so that the segmentation network can further mine the discriminative boundary features, thereby further improving the segmentation accuracy.

[0005] Combining the complementary advantages of the convolutional neural network model CNN and the Vision Transformer model ViT in analyzing local features and global contextual information, the local and global features can be effectively integrated. In addition, since dermatologists usually assign a rough location to skin lesions at the beginning and use detailed spatial features to finely outline the contours of the mask, it can be inferred that position and boundaries are two important features for distinguishing healthy skin from skin lesions. Therefore, the present invention also analyzes boundary clues and scale information, providing boundary prior knowledge for the entire network, so that the method can obtain more accurate segmentation results. It solves the technical problems of low segmentation accuracy and poor segmentation effect of existing methods.

[0006] The present invention can be achieved through the following technical solutions:

[0007] A medical image segmentation method based on boundary perception and bidirectional fusion includes the following steps:

[0008] S1. Construct a dataset according to the target task, wherein the dataset includes multiple images with pixel-level annotations of specific targets;

[0009] S2. Build a segmentation network model and input the images in the dataset into the built segmentation network model for training.

[0010] The segmentation network model adopts a dual-branch encoding and decoding structure, including an encoding module and a decoding module, wherein the encoding module is used to encode the input image, including a first encoding branch and a second encoding branch connected by a bidirectional fusion module, both of which adopt a layered structure and are arranged in a one-to-one correspondence. The encoded image output by the previous layer of the first encoding branch and the second encoding branch is bidirectionally fused in the corresponding layer in the bidirectional fusion module, and the first fused image and the second fused image are correspondingly output, which serve as the input of the next level of the first encoding branch and the second encoding branch respectively;

[0011] The decoding module is used to decode the encoded image to obtain the segmented image, and includes a primary decoding module and a secondary decoding module. The primary decoding module includes a first decoding branch and a second decoding branch corresponding to the first encoding branch and the second encoding branch, and is used to perform initial decoding and obtain significant features of the image. The secondary decoding module is used to perform final decoding and obtain edge features of the image to complete image segmentation.

[0012] S3. Input the image data to be segmented into the trained segmentation network model to obtain a high-precision segmentation result of the target image.

[0013] Furthermore, the bidirectional fusion module includes a feature alignment module, a feature filtering module and a feature fusion module. The feature alignment module is used to perform convolution and upsampling operations on the encoded image output by the corresponding layer of the first encoding branch, and to perform convolution and downsampling operations on the encoded image output by the corresponding layer of the second encoding branch, so as to align the features of the two; the feature filtering module uses an attention gating mechanism to process the aligned features separately, obtain the corresponding attention coefficients, and then multiply and calibrate them with the aligned features to reduce feature redundancy; the feature fusion module cross-processes the calibrated features with the encoded image output by the corresponding layer of the first encoding branch and the encoded image output by the corresponding layer of the second encoding branch to achieve feature fusion.

[0014] Furthermore, the feature filtering module uses a one-dimensional attention gate to process the features of the aligned encoded image output by the layer corresponding to the first encoding branch and the encoded image output by the layer corresponding to the second encoding branch to obtain a second attention coefficient, and then multiplies and calibrates the features of the aligned encoded image output by the layer corresponding to the second encoding branch to output a second filtered image;

[0015] A two-dimensional attention gate is used to process the features of the aligned encoded image output by the layer corresponding to the second encoding branch and the encoded image output by the layer corresponding to the first encoding branch to obtain a first attention coefficient, which is then multiplied and calibrated with the features of the aligned encoded image output by the layer corresponding to the first encoding branch to output a first filtered image;

[0016] The feature fusion module performs residual processing on the first filtered image and the coded image output by the corresponding layer of the second coding branch, and outputs a second fused image; performs residual processing on the second filtered image and the coded image output by the corresponding layer of the first coding branch, and outputs the first fused image.

[0017] Furthermore, the second decoding branch performs decoding by taking the coded images output by the two last layers in the second coding branch and the second fused image corresponding to the second coding branch finally output by the coding module as input;

[0018] The secondary decoding module adopts a hierarchical structure, and each layer includes a reverse attention module and a residual module. The reverse attention module of the first layer performs reverse attention processing on the second fused image corresponding to the second encoding branch and the final output image of the second decoding branch, and then performs residual processing on the output image of the second decoding branch.

[0019] The reverse attention modules in the middle layer take the output image of the previous layer and the coded image output by the corresponding layer of the second coding branch as input, and then perform residual processing on the output image of the previous layer. The coded images output by the corresponding layer of the second coding branch are from the first to the third layer from the last layer forward;

[0020] The reverse attention module of the last layer performs reverse attention processing on the final output image of the first decoding branch and the encoded image output by the second positive layer of the second encoding branch. Its output image is then subjected to residual processing with the final output image of the first decoding branch and the output image of the previous layer to achieve final decoding.

[0021] Furthermore, a deep supervision strategy is adopted for each layer of the first decoding branch, the second decoding branch and the secondary decoding module to correct the error between the predicted image and the true label.

[0022] Furthermore, the first encoding branch adopts a Transformer encoder, the second encoding branch adopts a CNN encoder, the first decoding branch adopts a CNN decoder, the second decoding branch adopts a partial decoder, and the secondary decoding module adopts a boundary-aware decoder;

[0023] The first encoding branch and the second encoding branch both adopt a four-layer network structure, each layer of the first encoding branch is composed of one or more Transformer modules, each of the Transformer modules includes a multi-head self-attention module and an MLP module, wherein the first layer and the second layer each include four Transformer modules, the third layer includes three Transformer modules, and the fourth layer includes one Transformer module;

[0024] Each layer of the second encoding branch is composed of multiple groups of cascaded bottleneck layers, each bottleneck layer includes a downsampling convolution layer, a spatial convolution layer, an upsampling convolution layer and a residual connection between the input and output, wherein the first layer includes four cascaded layers and three cascaded groups of bottleneck layers, the second layer includes four cascaded layers and three cascaded groups of bottleneck layers, the third layer includes six cascaded layers and three cascaded groups of bottleneck layers, and the fourth layer includes three cascaded layers and three cascaded groups of bottleneck layers.

[0025] The beneficial technical effects of the present invention are as follows:

[0026] (1) The constructed segmentation network adopts a CNN and Transformer dual-branch encoder structure, in which the CNN branch adopts a ResNet50-based structure to extract local clues, and the Transformer branch adopts a VisionTransformer-based structure to capture global context information. The feature information of the two branches complement each other.

[0027] (2) With the help of the bidirectional fusion module, the features from the CNN encoder branch and the Transformer encoder branch are aligned in terms of feature resolution and channel size, and then connected to the dual attention gate feature fusion module including the one-dimensional attention gate and the two-dimensional attention gate. From the actual effect, it seamlessly integrates the CNN and Transformer features, maximizes the retention of discriminative local detail information and global representation, and reduces feature redundancy.

[0028] (3) A boundary-aware decoder is designed, which uses multiple reverse attention blocks to process the segmentation mask obtained by the dual-branch decoder and the intermediate feature maps of the CNN encoder branch, so that the segmentation network can further mine discriminative boundary features to further improve the segmentation accuracy.

[0029] (4) By using a deep supervision strategy, the loss is calculated on a series of segmentation maps and true labels generated by the three decoders, making the network easier to optimize.

[0030] The cascaded self-attention module and multi-layer perceptron of the Vision transformer applied in the field of vision can obtain global information, map complex spatial transformations and extract long-range feature correlations, which is particularly helpful for processing downstream tasks such as image segmentation tasks. However, they still cannot extract enough local details to segment fuzzy boundaries. Convolutional neural networks have innate local perception and have a natural advantage in extracting local detail features such as texture and shape. Therefore, the present invention integrates the two and utilizes their complementary advantages in feature extraction to effectively integrate global features and local detail information. In addition, edge information plays an indispensable role in achieving accurate medical image segmentation results. The boundary-aware decoder designed in the present invention can effectively extract and refine edge features and improve the segmentation performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 1 is a flow chart of an image segmentation method according to an embodiment of the present invention;

[0032] Figure 2 Schematic diagram of the overall structure of the segmentation network model according to an embodiment of the present invention;

[0033] Figure 3 is a schematic structural diagram of a bidirectional fusion module according to an embodiment of the present invention;

[0034] Figure 4 3 is a schematic diagram comparing the segmentation results of the embodiment of the present invention with those of other algorithms. DETAILED DESCRIPTION

[0035] The specific implementation of the present invention is described in detail below with reference to the accompanying drawings and preferred embodiments.

[0036] The segmentation process of the present invention is described below by taking the segmentation of skin lesion images as an example. The present invention provides a medical image segmentation method based on boundary perception and bidirectional fusion. In a preferred embodiment, the overall schematic diagram of the method is as follows: Figure 1 As shown, the method includes the following steps:

[0037] S1. Construct a data set according to the target task. The data set includes multiple image samples with pixel-level annotations for specific targets, which can be three-dimensional images.

[0038] The dataset includes multiple labeled dermoscopic images. Each labeled dermoscopic image contains a foreground and background. The dataset is divided into a training set and a validation set, and the dataset is preprocessed to adapt to the input of the network model.

[0039] In this example, multiple dermoscopic image datasets are obtained. These datasets contain the original dermoscopic images, the corresponding labeled segmentation maps, and the dermoscopic images to be segmented along with their corresponding labels. The segmented regions are divided into skin lesions and background. The images are cropped and normalized to the same size, and data augmentation is performed by flipping and rotating the images to enhance the generalization capability of the image segmentation model.

[0040] In this example, 1815 images were ultimately obtained. To create the training and validation sets, the images were cropped at a fixed scale, randomly rotated and flipped with a certain probability, and finally normalized to achieve data augmentation. In this example, the total dataset was randomly split into training and validation sets at a sample size ratio of 8:2. Only the training dataset was used during training iterations, while the validation dataset was used only for accuracy testing during the model screening phase.

[0041] S2. Build a segmentation network model and input the images in the dataset into the built segmentation network model for training.

[0042] like Figure 2 As shown, the segmentation network model adopts a dual-branch encoding and decoding structure, including an encoding module and a decoding module, wherein the encoding module is used to encode the input image, including a first encoding branch and a second encoding branch connected by a bidirectional fusion module, both of which adopt a hierarchical structure and are set one-to-one. The encoded image output by the previous layer in the first encoding branch and the second encoding branch is bidirectionally fused by the corresponding layer in the bidirectional fusion module, and the first fused image and the second fused image are correspondingly output as the input of the next level in the first encoding branch and the second encoding branch, respectively.

[0043] The segmentation model's encoding module, the CNN-Transformer dual-branch encoder, consists of a Transformer branch based on the Vision Transformer, and a CNN branch based on ResNet50. The bidirectional fusion module consists of three main components: a feature alignment module, a feature filtering module based on an attention gating mechanism, and a feature fusion module. The model's decoder primarily corresponds to the dual-branch encoder, employing two progressively upsampling CNN decoders supplemented by a boundary-aware decoder.

[0044] The CNN branch with ResNet50 as the main trunk can be divided into four stages. Each stage consists of multiple convolution blocks containing n bottleneck layers. Each bottleneck layer contains a downsampling convolution layer, a spatial convolution layer, an upsampling convolution layer, and a residual connection between the input and output. The first stage is Conv2x in ResNet50. In Conv2x, the feature map is processed by three sets of cascaded bottleneck layers in turn as the input of the next stage. The next stage is composed of multiple sets of cascaded bottleneck layers. In general, the resolution of the feature map becomes one-quarter of the original image size after the first stage, and the number of channels is expanded to four times the original size. The second stage consists of four sets of cascaded bottleneck layers. The third stage consists of six cascaded bottleneck layers. After the third sampling block, the resolution of the image becomes one-sixteenth of the original size, and the number of channels becomes sixteen times the original size. The fourth stage consists of three bottleneck layers, with the resolution becoming one-thirty-second of the original size, and the number of channels becoming thirty-two times the original size. At the same time, the three-dimensional image features output by each stage are input into the bidirectional fusion module and the two-dimensional sequence vector of the transformer branch for feature interactive fusion, which can effectively improve the local perception ability of the transformer branch.

[0045] The Transformer branch encoder, based on the Vision Transformer, consists of 12 repeated transformer modules. Each transformer module consists of a multi-head self-attention module and an MLP module. Layer norms are applied to each layer and before the residual connection in the multi-head self-attention module and the MLP module. The original feature map is projected into a sequence vector to adapt to the input of the transformer block. The Transformer branch encoder is also divided into four stages, consisting of 4, 4, 3, and 1 transformer blocks respectively. The sequence vector output by each stage is input into the bidirectional fusion module for feature interaction fusion with the feature map of the CNN branch. After calculation, it is restored to a sequence vector and input into the next stage. The input and output feature dimensions of the transformer branch before and after each stage remain unchanged. After the feature extraction in the last stage, it is converted into a feature map using upsampling and channel number adjustment and input into the subsequent decoder.

[0046] like Figure 3 As shown in the figure, the bidirectional fusion module includes a feature alignment module, a feature filtering module and a feature fusion module. At each stage of the CNN-Transformer dual-branch encoder, the features of the two modalities obtained need to be interactively fused.

[0047] Among them, the feature alignment module is used to perform convolution and upsampling operations on the coded image output by the corresponding layer of the first coding branch, and perform convolution and downsampling operations on the coded image output by the corresponding layer of the second coding branch, so as to align the features of the two, that is, to use the convolution layer and downsampling operations to align the coded image corresponding to the feature x of the CNN branch. f Corresponding features x of the encoded image of the Transformer branch p Alignment. Similarly, in order to align the features of the two, it is necessary to reshape the Transformer features, then perform upsampling and convolution.

[0048] The feature filtering module uses an attention gating mechanism to process the aligned features separately, obtain the corresponding attention coefficients, and then multiply and calibrate them with the aligned features to reduce feature redundancy. Specifically, a one-dimensional attention gate is used to process the features of the aligned coded image output by the corresponding layer of the first coding branch and the coded image output by the corresponding layer of the second coding branch to obtain a second attention coefficient β. The features are then multiplied and calibrated with the aligned coded image output by the corresponding layer of the second coding branch, and a second filtered image is output to obtain the calibrated features, thereby reducing feature redundancy.

[0049] A two-dimensional attention gate is used to process the features of the aligned coded image output by the corresponding layer of the second coding branch and the coded image output by the corresponding layer of the first coding branch to obtain the first attention coefficient α, which is then multiplied and calibrated with the features of the aligned coded image output by the corresponding layer of the first coding branch, and the first filtered image is output to obtain the calibrated features, thereby reducing feature redundancy.

[0050] The feature fusion module performs residual processing on the calibrated features with the coded images output by the corresponding layer of the first coding branch and the coded images output by the corresponding layer of the second coding branch to achieve feature fusion. Specifically, the first filtered image and the coded image output by the corresponding layer of the second coding branch are subjected to residual processing to output a second fused image; the second filtered image and the coded image output by the corresponding layer of the first coding branch are subjected to residual processing to output a first fused image. Finally, the obtained first fused image and second fused image are input into the next stage of the first coding branch and the second coding branch respectively, so that the features of the model at each stage can be interactively coupled, gradually filling the semantic gap.

[0051] The decoding module is used to decode the encoded image and obtain the segmented image, including a first-level decoding module and a second-level decoding module, wherein the first-level decoding module includes a first decoding branch and a second decoding branch corresponding to the first encoding branch and the second encoding branch, which are used to implement initial decoding and obtain the salient features of the image, so that the features of the two modalities can learn different things while interacting. For the Transformer encoding branch, a progressively up-sampled CNN decoder is used, that is, the one-dimensional feature vector of the transformer encoding branch is reshaped into a feature map and input into the CNN decoder. Four consecutive decoding blocks are used to restore its spatial detail information and obtain the final segmentation result S1; for the CNN encoding branch, an efficient partial decoder is used to aggregate the high-level feature maps of the last three stages and obtain the segmentation result S g .

[0052] The secondary decoding module is used to achieve final decoding, obtain edge features of the image, and complete image segmentation. A boundary-aware decoder can be used to mine discriminative boundary features. The boundary-aware decoder adopts a hierarchical structure, and each layer includes an inverse attention module and a residual module. That is, residual learning is applied together with the inverse attention RA module to extract dense features for pixel-by-pixel prediction, and residual refinement is achieved by deep supervision of the input and output of the proposed boundary-aware decoder. Among them, the inverse attention module of the first layer performs inverse attention processing on the second fused image of the second encoding branch corresponding to the final output of the encoding module and the final output image of the second decoding branch. Its output image is then subjected to residual processing with the final output image of the second decoding branch.

[0053] The reverse attention modules in the middle layer take the output image of the previous layer and the coded image output by the corresponding layer of the second coding branch as input, and then perform residual processing on the output image of the previous layer. The coded images output by the corresponding layer of the second coding branch are from the first to the third layer from the last layer forward;

[0054] The reverse attention module of the last layer performs reverse attention processing on the final output image of the first decoding branch and the encoded image output by the second positive layer of the second encoding branch. Its output image is then subjected to residual processing with the final output image of the first decoding branch and the output image of the previous layer to achieve final decoding.

[0055] Starting from the coarse saliency map generated by the penultimate layer of the CNN encoding branch, the predicted saliency maps S2, S3, S4, and S5 are generated in sequence, correcting the error between the predicted saliency map and the true label and guiding the learning process of the entire network well. Finally, the final segmentation map is obtained using the output of the second-level decoder.

[0056] S3. After configuring the loss function, optimizer, and training parameters for the model training phase, the training dataset is input into the segmentation network for model training. The model is screened based on the performance of the validation dataset to obtain the optimal image segmentation model.

[0057] In this embodiment, the size of the input image is H×W×3, where H, W, and 3 are the height, width, and number of channels of the two-dimensional image, respectively, and the data size format is 224*224*3; the initial learning rate of the network is set to 7e-5, and the learning rate gradually decreases with the increase in the number of iterations. The number of batches is 8, and the adam optimizer is used for training. The loss function uses the weighted sum of the Dice loss function and the cross entropy loss function, with the weights set to 0.6 and 0.4 respectively. The settings of the remaining network parameters can be understood conventionally and will not be repeated here. When the model performance of the segmentation network reaches the preset accuracy requirements, the segmentation network is used as the final image segmentation model. Model performance can be judged by commonly used evaluation parameters such as prediction accuracy, or the segmentation effect can be manually judged.

[0058] S4. Input the two-dimensional image data to be segmented into the trained segmentation network model to obtain a high-precision segmentation result of the target image.

[0059] Before the original target dermoscopic image is input into the image segmentation model, the target dermoscopic image needs to be preprocessed, including operations such as cropping and denoising. These operations will not be described in detail here, and relevant practitioners can perform them according to conventional understanding.

[0060] In order to intuitively demonstrate the feasibility of the medical image segmentation method of the present invention, we visualize the segmentation results of several typical challenging cases in the ISIC 2016 dataset, which are generated by U-Net, AttU-Net, CPFNet and FAT-Net. Figure 4 As shown, for lesion areas with complex and tortuous boundaries, as shown in the second row of the figure, the method of the present invention can obtain satisfactory results, but the other methods still need further improvement. However, in the case where the other four methods cannot handle unclear and small-protruding skin lesion areas well, as shown in the first and third rows of the figure, the segmentation network model of the present invention still maintains good performance. In addition, for extremely small targets, CNN-based methods often produce over-segmentation results, and other transformer-based methods may even ignore the objects to be segmented, while our method obtains more accurate results, and the method of the present invention can also be well applied to cases where hair is partially occluded. In summary, the method of the present invention can obtain the best results even in challenging low-contrast and complex edge situations, and is more suitable for the segmentation of skin lesions.

[0061] Although specific embodiments of the present invention are described above, those skilled in the art should understand that these are merely examples and that various changes or modifications may be made to these embodiments without departing from the principles and essence of the present invention. Therefore, the scope of protection of the present invention is limited by the appended claims.

Claims

1. A medical image segmentation method based on boundary perception and bidirectional fusion, characterized by The following steps are involved: S1. Construct a dataset according to the target task, wherein the dataset includes multiple images with pixel-level annotations of specific targets; S2. Build a segmentation network model and input the images in the dataset into the built segmentation network model for training. The segmentation network model adopts a dual-branch encoding and decoding structure, including an encoding module and a decoding module, wherein the encoding module is used to encode the input image, including a first encoding branch and a second encoding branch connected by a bidirectional fusion module, both of which adopt a layered structure and are arranged in a one-to-one correspondence. The encoded image output by the previous layer of the first encoding branch and the second encoding branch is bidirectionally fused in the corresponding layer in the bidirectional fusion module, and the first fused image and the second fused image are correspondingly output, which serve as the input of the next level of the first encoding branch and the second encoding branch respectively; The decoding module is used to decode the encoded image to obtain the segmented image, and includes a primary decoding module and a secondary decoding module. The primary decoding module includes a first decoding branch and a second decoding branch corresponding to the first encoding branch and the second encoding branch, and is used to perform initial decoding and obtain significant features of the image. The secondary decoding module is used to perform final decoding and obtain edge features of the image to complete image segmentation. S3. Input the image data to be segmented into the trained segmentation network model to obtain a high-precision segmentation result of the target image; The bidirectional fusion module includes a feature alignment module, a feature filtering module and a feature fusion module. The feature alignment module is used to perform convolution and upsampling operations on the coded image output by the corresponding layer of the first coding branch, and to perform convolution and downsampling operations on the coded image output by the corresponding layer of the second coding branch, so as to align the features of the two; the feature filtering module uses an attention gating mechanism to process the aligned features separately, obtain the corresponding attention coefficients, and then multiply and calibrate them with the aligned features to reduce feature redundancy; the feature fusion module cross-processes the calibrated features with the coded images output by the corresponding layers of the first coding branch and the coded images output by the corresponding layers of the second coding branch to achieve feature fusion.

2. The medical image segmentation method based on boundary perception and bidirectional fusion according to claim 1, characterized in that: The feature filtering module uses a one-dimensional attention gate to process the features of the encoded image output by the corresponding layer of the first encoding branch and the aligned encoded image output by the corresponding layer of the second encoding branch to obtain a second attention coefficient, and then multiplies and calibrates the features with the aligned features of the encoded image output by the corresponding layer of the second encoding branch to output a second filtered image; A two-dimensional attention gate is used to process the features of the aligned encoded image output by the layer corresponding to the second encoding branch and the encoded image output by the layer corresponding to the first encoding branch to obtain a first attention coefficient, which is then multiplied and calibrated with the features of the aligned encoded image output by the layer corresponding to the first encoding branch to output a first filtered image; The feature fusion module performs residual processing on the first filtered image and the coded image output by the corresponding layer of the second coding branch, and outputs a second fused image; performs residual processing on the second filtered image and the coded image output by the corresponding layer of the first coding branch, and outputs the first fused image.

3. The medical image segmentation method based on boundary perception and bidirectional fusion according to claim 1, characterized in that: The second decoding branch performs decoding by taking the coded images output by the two last layers in the second coding branch and the second fused image corresponding to the second coding branch finally output by the coding module as input; The secondary decoding module adopts a hierarchical structure, and each layer includes a reverse attention module and a residual module. The reverse attention module of the first layer performs reverse attention processing on the second fused image corresponding to the second encoding branch and the final output image of the second decoding branch, and then performs residual processing on the output image of the second decoding branch. The reverse attention modules in the middle layer take the output image of the previous layer and the coded image output by the corresponding layer of the second coding branch as input, and then perform residual processing on the output image of the previous layer. The coded images output by the corresponding layer of the second coding branch are from the first to the third layer from the last layer forward; The reverse attention module of the last layer performs reverse attention processing on the final output image of the first decoding branch and the encoded image output by the second positive layer of the second encoding branch. Its output image is then subjected to residual processing with the final output image of the first decoding branch and the output image of the previous layer to achieve final decoding.

4. The medical image segmentation method based on boundary perception and bidirectional fusion according to claim 3, characterized in that: A deep supervision strategy is used for the prediction results of the first decoding branch, the second decoding branch, and each layer of the secondary decoding module to correct the error between the predicted image and the true label.

5. The medical image segmentation method based on boundary perception and bidirectional fusion according to claim 1, characterized in that: The first encoding branch adopts a Transformer encoder, the second encoding branch adopts a CNN encoder, the first decoding branch adopts a CNN decoder, the second decoding branch adopts a partial decoder, and the secondary decoding module adopts a boundary-aware decoder; The first encoding branch and the second encoding branch both adopt a four-layer network structure, each layer of the first encoding branch is composed of one or more Transformer modules, each of the Transformer modules includes a multi-head self-attention module and an MLP module, wherein the first layer and the second layer each include four Transformer modules, the third layer includes three Transformer modules, and the fourth layer includes one Transformer module; Each layer of the second encoding branch is composed of multiple groups of cascaded bottleneck layers, each bottleneck layer includes a downsampling convolution layer, a spatial convolution layer, an upsampling convolution layer and a residual connection between the input and output, wherein the first layer includes four cascaded layers and three cascaded groups of bottleneck layers, the second layer includes four cascaded layers and three cascaded groups of bottleneck layers, the third layer includes six cascaded layers and three cascaded groups of bottleneck layers, and the fourth layer includes three cascaded layers and three cascaded groups of bottleneck layers.

Images