Medical image segmentation method based on multi-branch multi-scale attention mechanism
Through the multi-level encoder and adaptive multi-branch patch module combined with the multi-scale attention mechanism, the problem of CNN local features ignoring the global context and high computational complexity of transformer is solved, and efficient medical image segmentation is achieved.
Patent Information
- Application Number
- CN202510499976.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-21
- Publication Date
- 2025-07-18
AI Technical Summary
CNN network only focuses on local regional features and ignores global context information, while transformer computing is highly complex and requires high computer hardware.
A multi-level encoder is used for feature extraction, combining the adaptive multi-branch patch module and the multi-scale attention mechanism, and adjusting the patch weight and the multi-head attention mechanism fusion characteristics through adaptive parameters, solving the problem of local feature extraction of CNN and reducing the computational complexity of transformer.
High-quality medical image segmentation is realized, the model's sensitivity to global context information is improved, the calculation complexity is reduced, the category imbalance problem is alleviated, and high-quality segmentation results are generated.
Smart Images
Figure CN120339625A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical image segmentation, and specifically relates to a medical image segmentation method based on a multi-branch multi-scale attention mechanism. Background Art
[0002] Medical image segmentation technology is a technology in the field of computer vision and is widely used in medical image analysis, intelligent diagnosis, treatment, etc. Its purpose is to separate lesions, tumors or organs from the background through pixel-level classification of medical images; traditional medical image segmentation mainly relies on experienced doctors for manual segmentation, which requires a large amount of manpower and material resources and has extremely low efficiency; deep learning methods, especially convolutional neural networks (CNNs), can automatically learn features in images and significantly improve the efficiency and accuracy of medical image segmentation; however, CNNs move on convolutional kernels to extract features, only focusing on pixels in local regions each time and capturing local features. This method performs well in extracting detailed information but has certain limitations in grasping the global context relationship between features.
[0003] Transformer encodes an image into a long sequence and effectively captures long-range dependence and global context information in the image by means of position encoding to retain the position information between pixels; this encoding method breaks through the limitation of CNNs that only focus on local features, enabling the model to understand the content of the image globally and thus achieving high-quality segmentation of the image. However, the computational complexity of Transformer is relatively high and requires high computer hardware. And when generating the next token of the sequence, Transformer must recalculate the attention of the entire sequence even if these sequence tokens have already been generated.
[0004] The above problems need to be solved urgently. For this reason, the present invention proposes a medical image segmentation method based on a multi-branch multi-scale attention mechanism. Summary of the Invention
[0005] The technical problem to be solved by the present invention is: how to solve the problem that the CNN network only focuses on local region features and ignores global context information, while the computational complexity of Transformer is high and requires high computer hardware, and provides a medical image segmentation method based on a multi-branch multi-scale attention mechanism.
[0006] The present invention solves the above technical problems through the following technical solutions. The present invention includes the following steps:
[0007] S1: Data preprocessing
[0008] Obtain a medical image dataset and perform image enhancement processing on the medical images in the dataset;
[0009] S2: Shallow Semantic Feature Extraction
[0010] Input the medical image after image enhancement processing into the encoder of the medical segmentation model for multi-level feature extraction to obtain a feature map that can represent shallow semantic features;
[0011] S3: Feature Decoding
[0012] Input the feature map obtained in step S2 into the decoder of the medical segmentation model, and perform decoding processing on the feature map through a multi-branch multi-scale attention mechanism to obtain local details and global context information of the medical image, and perform multi-scale fusion on the information to obtain a fused feature map;
[0013] S4: Image Segmentation
[0014] Input the fused feature map in step S3 into the segmentation head of the medical segmentation model for segmentation operation to obtain a segmentation result.
[0015] Furthermore, in step S1, the image enhancement processing includes normalization, random flipping, and random scaling processing. When performing image enhancement processing, the medical images in the dataset and the corresponding segmentation label masks are processed for image enhancement simultaneously.
[0016] Furthermore, in step S2, the encoder encodes the shallow semantic features of the input image layer by layer to obtain a feature map that can represent shallow semantic features. The input image is the medical image. The encoder includes a five-layer encoding structure, and each layer of the encoding structure performs convolution and downsampling operations. Moreover, each layer of the encoding structure not only inputs the result into the corresponding decoding structure but also passes it down to the next layer of the encoding structure.
[0017] Furthermore, in step S3, the decoder includes an adaptive multi-branch patch module, a convolutional block attention module integrating residual blocks, and a multi-scale feature fusion module; the adaptive multi-branch patch module is used for deep feature extraction of the input feature map, the convolutional block attention module integrating residual blocks is used to retain important features in the feature map obtained by the adaptive multi-branch patch module, and the multi-scale feature fusion module extracts multi-scale information through pooling operations of different scales and fuses the multi-scale information to obtain a fused feature map.
[0018] Furthermore, the adaptive multi-branch patch module includes three branches, and two of the branches are used for adaptive patch operations. The specific processing process is as follows:
[0019] S301: For the input x of the i-th layer i, different branches divide the input into ρ×ρ blocks along the two dimensions of the input width W and height H through a chunking operation, and use average pooling operation and two MLPs to process the feature map to obtain an intermediate result l i；ρ , ρ = 2, 4;
[0020] S302: Each branch calculates the weight map of each branch according to the corresponding intermediate result l i；ρ to adjust the weight of each branch;
[0021] S303: Introduce a learnable parameter to further adjust the weight of each branch, and multiply the intermediate result l i；ρ by the weight map to obtain an intermediate result APM ρ ;
[0022] Another branch replaces the adaptive patch operation of the above two branches with two convolution operations to obtain the corresponding output;
[0023] Finally, by adding the outputs of the three branches and performing dropout and normalization operations, the output m of the adaptive multi-branch patch module is obtained i , the output m i is a feature map with only the lesion location and contour.
[0024] Furthermore, the convolutional block attention module of the integrated residual block includes the channel attention block of the integrated residual block and the spatial attention block of the integrated residual block. The channel attention block of the integrated residual block obtains two channel-weighted feature maps through global average pooling and global maximum pooling. After adding the two feature maps, perform a Sigmoid operation and multiply and add with the residual block pixel by pixel to obtain the channel attention map; the spatial attention block of the integrated residual block processes the channel attention map through average pooling and maximum pooling, adds the two spatially weighted feature maps and then performs convolution, performs a Sigmoid operation on the convolved feature map, and multiplies and adds with the residual block element by element to obtain the spatial attention map. For the input m i , m i first passes through the channel attention block of the integrated residual block, and then passes through the spatial attention block of the integrated residual block for processing to obtain the output z of the convolutional block attention module of the integrated residual block i , the output z i is a feature map about the lesion details.
[0025] Furthermore, in the multi-scale feature fusion module, for the input z i, three sizes of pooling operations are used to pool the original input feature map into feature blocks of different scales. The feature set containing detailed information is obtained through splicing and used as the key vector value and value vector value of the multi-head attention mechanism. By rearranging the original input, it is used as the query vector value of the multi-head attention mechanism. And position encoding is added to the query vector, key vector, and value vector. The features are fused through the attention operation to obtain the attention map. The attention map is processed by two MLPs and a normalization operation is performed to obtain the output s of the multi-scale feature module i , the output s i is the fused feature map
[0026] Furthermore, the medical image segmentation method based on the multi-branch multi-scale attention mechanism further includes the following steps
[0027] S5: Model parameter optimization
[0028] Set the parameters of the model, calculate the overall loss value between the segmentation result and the ground truth through the overall loss function, and use the backpropagation algorithm and stochastic gradient descent to update the parameters of the model until the model reaches the iteration threshold
[0029] Furthermore, in the step S5, the overall loss function L Total includes the weighted Dice loss L Dice and the cross-entropy loss L BCE , and the specific calculation formula is as follows
[0030] L Total =αL BCE +βL Dice
[0031] where α and β are the weight parameters of the corresponding type of loss
[0032] Furthermore, the calculation formula of the weighted Dice loss L Dice is as follows
[0033]
[0034] where N represents the number of pixels, M represents the number of batches, and are the weighted ground truth and prediction respectively
[0035] The calculation formula of the cross-entropy loss L BCE is as follows
[0036]
[0037] where t i and p irepresent the true value and the predicted value, pos i and neg i represent that for the true value, pixel i is a positive class or a negative class, N pos and N neg represent the total numbers of the positive class and the negative class in the true value, w pos and w neg represent the weight values for the positive class and the negative class.
[0038] The present invention has the following advantages compared with the prior art:
[0039] 1. The present invention uses a multi-level encoder for shallow feature extraction and an adaptive multi-branch patch module in the decoder stage for deep feature extraction, improving the problem that existing CNN methods can only extract local features and are insensitive to global context information of images, while the computational complexity of methods based on transformers is too high. Each layer of the multi-level encoder uses convolution and downsampling to gradually reduce the size of the features and increase the number of channels of the features; the adaptive multi-branch patch module uses three-branch patch operations to supplement the detailed features into the main branch and uses adaptive parameters to dynamically adjust the weights of the patches; information such as the position, contour, and edge of the lesion or organ is obtained in the feature extraction stage.
[0040] 2. The present invention obtains a high-quality segmentation result through the refined processing of the feature map by the decoder; the feature map obtained in the first stage of the decoder is input into the convolutional block attention module of the integrated residual block to extract the spatial information and channel information most beneficial to image segmentation, and the detailed information of the lesion or organ is restored in the feature map obtained at this stage; the spatial information and channel information are input into the multi-scale feature fusion module for feature fusion; different-sized pooling operations are used in the feature fusion stage to obtain multi-scale information, and in combination with positional encoding, the multi-head attention mechanism is used to fuse the features, and the detailed information of the lesion and its surrounding tissues is restored in the fused feature map; finally, the segmentation head further processes the fused information to obtain the segmentation result and generate the predicted mask.
[0041] 3. The present invention uses weighted Dice and BCE losses to calculate the loss between the predicted value and the true value. In medical image segmentation, due to the large differences in the frequencies and areas of different tissues and lesions appearing in the image, there is a problem of class imbalance, resulting in the model being biased towards the classes with a larger number during the training process and ignoring the classes with a smaller number, leading to missegmentation; the loss function combining the weighted Dice loss and the BCE loss can alleviate the problem of class imbalance; by calculating the numbers of different classes, a weight is added to each class to improve the model's attention to the classes and improve the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1It is a schematic diagram of the overall structure of the medical image segmentation model in Embodiment 1 of the present invention;
[0043] Figure 2 It is a schematic diagram of the structure of the adaptive multi-branch patch module of the medical image segmentation model in Embodiment 1 of the present invention;
[0044] Figure 3 It is a schematic diagram of the convolutional block attention module of the integrated residual block of the medical image segmentation model in Embodiment 1 of the present invention;
[0045] Figure 4 It is a schematic diagram of the structure of the multi-scale feature fusion module of the medical image segmentation model in Embodiment 1 of the present invention;
[0046] Figure 5 It is the overall flowchart of the medical image segmentation method based on the multi-branch multi-scale attention mechanism in Embodiment 1 of the present invention;
[0047] Figure 6 It is the implementation process schematic diagram of the medical image segmentation method based on the multi-branch multi-scale attention mechanism in Embodiment 1 of the present invention;
[0048] Figure 7 It is the qualitative result comparison diagram of the comparative experiment on the BUSI dataset in Embodiment 2 of the present invention. Among them, (a) is the original input medical image, (b)-(j) are the experimental results of other comparative models, (k) is the experimental result of the method of the present invention, and (l) is the segmentation label of the input medical image;
[0049] Figure 8 It is the qualitative result comparison diagram of the experiment on the ISIC2018 dataset in Embodiment 2 of the present invention. Among them, (a) is the original input medical image, (b)-(j) are the experimental results of other comparative models, (k) is the experimental result of the method of the present invention, and (l) is the segmentation label of the input medical image;
[0050] Figure 9 It is the qualitative result comparison diagram of the experiment on the GlaS dataset in Embodiment 2 of the present invention. Among them, (a) is the original input medical image, (b)-(j) are the experimental results of other comparative models, (k) is the experimental result of the method of the present invention, and (l) is the segmentation label of the input medical image;
[0051] Figure 10 It is the qualitative result comparison diagram of the experiment on the PolS dataset in Embodiment 2 of the present invention. Among them, (a) is the original input medical image, (b)-(j) are the experimental results of other comparative models, (k) is the experimental result of the method of the present invention, and (l) is the segmentation label of the input medical image. Detailed implementation manners
[0052] The embodiments of the present invention will be described in detail below. These embodiments are implemented on the premise of the technical solution of the present invention, and detailed implementation manners and specific operation processes are given. However, the protection scope of the present invention is not limited to the following embodiments.
[0053] Embodiment 1
[0054] As Figure 5 、 6 shown, this embodiment provides a technical solution: a medical image segmentation method based on a multi-branch multi-scale attention mechanism, including the following steps:
[0055] Step 1: Obtain a medical image dataset, and perform image enhancement processing using operations such as normalization, random flipping, and scaling.
[0056] As an example, extract the BUSI, ISIC2018, GlaS, and PolS datasets from a standard medical image library. For each image, set a random number. If the random number is greater than 0.5, perform image enhancement; if the random number is less than or equal to 0.5, keep the original image. The image enhancement processing includes operations such as normalization, random flipping, and random scaling.
[0057] It should be noted that the input image contains the source image and the segmentation label mask. When performing image enhancement operations, the source image and the segmentation label are processed simultaneously.
[0058] Step 2: Input the enhanced image into the multi-scale encoder of the medical segmentation model of the present invention for multi-level feature extraction to extract shallow semantic features such as the edges and contours of the image.
[0059] As Figure 1 shown, it is a schematic diagram of the overall structure of the medical segmentation model of the present invention.
[0060] As an example, input the enhanced dataset into the encoder (Encoder). The encoder will use convolution and downsampling operations to encode the shallow semantic features of the input image layer by layer to obtain shallow feature maps. The encoder of the present invention includes a five-layer coding structure. Each layer of the coding structure not only inputs the result into the corresponding decoding structure but also passes it down to the next layer of the coding structure; for the input x ∈ R H×W×C , the feature map obtained by the first layer of the encoder is x1 ∈ R H×W×C′ , the one obtained by the second layer is the one obtained by the third layer is And so on.
[0061] Step 3: Input the feature map obtained by feature extraction into the decoder (Decoder) of the medical segmentation model of the present invention, and perform refined processing on the features through a multi-branch multi-scale attention mechanism to obtain local details and global context information of the image, and fuse the information to enhance the feature representation ability.
[0062] As an example, as Figure 2 shown, for the input feature map, the Adaptive Multi-Branch Patch Module (AMBPM) will further perform in-depth feature extraction on the feature map; the Adaptive Multi-Branch Patch Module has three branches, and each branch will supplement the feature map into the backbone in the form of patches, and dynamically adjust the weights of the patches through adaptive parameters to adapt to the segmentation task. Specifically, for the input x of the i-th layer i , different branches divide the input into ρ×ρ blocks along the two dimensions of the input width W and height H through block operations, and use mean pooling operation Mean and two MLP operations to process the feature map to enrich the feature representation ability and obtain the intermediate result l i；ρ ; each branch of the Adaptive Multi-Branch Patch Module of the present invention calculates the weight map mask of each branch according to l i；ρ to adjust the weights of each branch. Further, by introducing a learnable parameter to further adjust the weights of each branch, and obtain the intermediate result APM through weight multiplication ρ ; the Adaptive Multi-Branch Patch Module of the present invention adds each branch and adopts operations such as random dropout Dropout and normalization Batchnorm to improve the generalization performance of the module, and obtains the module output result m i ; in a specific implementation, the present invention takes ρ = 1, 2, 4 as the parameters of the block operation. When ρ = 1, the entire feature map is a pseudo-block mapping. In order to maintain the same output format as the original block, the present invention adopts two convolution operations in the second branch to replace the original adaptive patch (APM) operation. The processing formula of the Adaptive Multi-Branch Patch Module is as follows:
[0063] l i；ρ = MLP2(MLP1(Mean(P ρ (x i ))))
[0064]
[0065] m i = Relu(BN(Drop(APM ρ＝2 (x i ) + Conv(Conv(x i )) + APM ρ＝4 (x i))))
[0066] Among them, P ρ represents the chunking operation, Drop represents random dropout, BN represents batch normalization, and Relu represents the Relu activation function; the output of the module is a feature map with only the lesion location and contour, so as to obtain fine lesion localization and contour information; this feature map m i is input into the convolutional block attention module with residual blocks (CBAMRB) (see Figure 3 ) for spatial and channel weighting. The convolutional block attention module of the integrated residual block includes two parts: the channel attention block (CAMRB) of the integrated residual block and the spatial attention block (SAMRB) of the integrated residual block. The channel attention block of the integrated residual block obtains two channel-weighted feature maps through global average pooling Avg and global max pooling Max. After adding the two feature maps, they are processed through the Sigmoid operation and multiplied and added pixel by pixel with the residual block to obtain the channel attention map CAMRB i ; the spatial attention block of the integrated residual block processes the channel attention map through mean pooling Mean and max pooling Max, adds the two spatially weighted feature maps, performs convolution Conv, and performs the Sigmoid operation on the convolved feature map, and multiplies and adds it element by element with the residual block to obtain the spatial attention map SAMRB i , the input m i first passes through CAMRB, and the output result then passes through SAMRB for processing to obtain the output z of the convolutional block attention module of the integrated residual block i , and its formula is as follows:
[0067] CAMRB i = Sig(Avg(m i )) + Max(m i )) × m i + m i
[0068] SAMRB i = Sig(Conv(Meab(CAMRB i )) + Max(CAMRB i ))) × CAMRB i + CAMRB i
[0069] z i = SAMRB i (CAMRB i (m i ))
[0070] where Sig represents the Sigmoid operation; in this module, only the task-beneficial and non-redundant spaces and channels are retained; the output of the module is a feature map of the details of the lesion, only retaining the detailed information of the lesion; finally, the feature map is input into the multi-scale feature fusion module (MSFFM) (see Figure 4 ) for feature fusion. For the input z i , three sizes of pooling operations are used to pool the original input feature map into feature blocks of different scales. By concatenation, a feature set containing detailed information is obtained, which serves as the K (key vector) and V (value vector) values of the multi-head attention mechanism. By rearranging the original input, it serves as the Q (query vector) value of the multi-head attention mechanism, and positional encoding is added to Q, K, and V. Through the attention operation, the features are fused to obtain the attention map att i . An MLP operation and a normalization Norm operation are performed on the attention map to obtain the output s i of the multi-scale feature module, and its formula is as follows:
[0071]
[0072] s i = Norm2(MLP2(Norm1(MLP1(att i ))))
[0073] where MHA represents the multi-head attention mechanism, pos represents the positional encoding, Pool represents the pooling operation, j represents different sizes of pooling operations, and Norm represents the normalization operation. Feature fusion uses pooling operations of different sizes to pool the input, adds positional encoding, and feeds it into the multi-head attention mechanism for feature fusion; the fused image restores the lesion and surrounding detailed information of the image.
[0074] Step 4: Input the above fused result into the segmentation head of the medical segmentation model of the present invention for further processing to obtain a high-quality segmentation result;
[0075] As an example, for the fused image, when input into the segmentation head, the model will, based on the previous understanding of the image, judge pixel by pixel whether it belongs to the lesion area or the non-lesion area; after being processed by the segmentation head, the model generates a black-and-white prediction result map, where the lesion area is white and other areas are black.
[0076] Step 5: Set the parameters of the model, calculate the loss value between the segmentation result and the true result using weighted Dice and cross-entropy loss, and use the backpropagation algorithm and stochastic gradient descent to update the parameters of the model until the model reaches the iteration threshold.
[0077] As an example, the output result of the model and the true result are used to calculate the weighted Dice loss pixel by pixel according to the formula and the cross-entropy loss Finally, the total loss obtained is weighted and summed to obtain the final loss L Total = αL BCE + βL Dice ; The gradient of the model is calculated by the backpropagation algorithm and the stochastic gradient descent optimizer, and the model parameters are updated through the formula θ ′ is the updated model parameter; After the model parameters are updated, the iteration count is incremented by one. The iteration count is set to 100 times, and the iteration is repeated until the model reaches the iteration threshold
[0078] The working principle of the present invention is as follows
[0079] Obtain a medical image dataset, and perform image enhancement using operations such as normalization, random flipping, and scaling; Input the enhanced images into a multi-scale encoder for multi-level feature extraction to extract shallow semantic features such as the edges and contours of the images; Input the feature maps obtained from feature extraction into the decoder, and use a multi-branch multi-scale attention mechanism to refine the features to obtain the local details and global context information of the images, and fuse the information to enhance the representation ability of the features; Input the above fused results into the segmentation head for further processing to obtain high-quality segmentation results; Set the parameters of the model, calculate the loss value between the segmentation result and the true result using weighted Dice and cross-entropy losses, and use the backpropagation algorithm and stochastic gradient descent to update the parameters of the model until the model reaches the iteration threshold
[0080] Embodiment 2
[0081] To further illustrate the technical solution of the present invention, the present invention selects four publicly available datasets, namely BUSI, ISIC2018, GlaS, and PolS, to conduct experiments, and conducts comparative experiments with some existing methods. Two of the most commonly used medical segmentation metrics, Dice and IoU, are selected as the evaluation metrics for this embodiment. Specifically, for the BUSI dataset, the BUSI dataset contains 891 benign breast cancer images and 421 malignant breast cancer images. In the experiments of the present invention, the BUSI dataset is stratified and sampled into three sets in a ratio of 6:2:2: a training set, a validation set, and a test set; for the ISIC2018 dataset, the ISIC2018 dataset contains 2,594 high-resolution dermoscopic images. In the experiments of the present invention, the first 600 images of the ISIC2018 dataset are sampled into three sets in a ratio of 6:2:2: a training set, a validation set, and a test set; for the GlaS dataset, the training set of the GlaS dataset contains 37 benign and 48 malignant gland images, and the test set contains 37 benign and 43 malignant images. In the experiments of the present invention, the original test set is retained, and the original training set is divided into a training set and a validation set in a ratio of 8:2; for the PolS dataset, the PolS dataset contains 1,000 medical images. In the experiments of the present invention, the first 600 images of the PolS dataset are divided into a training set, a validation set, and a test set in a ratio of 6:2:2. For the BUSI, ISIC2018, and PolS datasets, the experiments of the present invention and the comparative experiments are both trained for 100 rounds on the three datasets; for the GlaS dataset, in order to avoid the phenomenon of underfitting due to fewer images, we trained the comparative experiment for 1,000 rounds and trained the experiment of the present invention (abbreviated as M2ANet) for 100 rounds; for all experiments, under the same experimental settings, we repeated the experiments three times and took the average value and variance of the experiments. The comparative diagram of the experimental qualitative results on each dataset is shown in Figures 7 - 10 .
[0082] Table 1 Results of the experiments of the present invention and the comparative experiments on the Dice score evaluation metric (the larger the Dice value, the better the experimental result)
[0083] Comparison method BUSI ISIC2018 GlaS PolS U-Net 71.66±0.008 85.99±0.003 87.57±0.02 76.99±0.047 MultiResUNet 71.64±0.003 88.9±0.004 87.1±0.011 78.57±0.015 SwinB 77.44±0.011 88.38±0.002 87.63±0.051 88.95±0.005 UCTransNet 75.69±0.007 85.63±0.004 86.54±0.019 79.24±0.035 Swin-Unet 73.61±0.001 86.48±0.002 86.75±0.004 80.59±0.006 SMESwin Unet 70.81±0.01 86.07±0.002 84.76±0.004 80.03±0.007 UNeXt 74.34±0.005 87.5±0.005 85.86±0.002 73.98±0.014 ACC-UNet 73.63±0.024 87.33±0.005 87.08±0.014 81.4±0.006 MADGNet 76.75±0.008 88.79±0.006 88.3±0.008 82.92±0.009 M2ANet (Ours) 79.28±0.004 89.07±0.01 89.97±0.008 89.1±0.003
[0084] Table 2 Results of the experiments of the present invention and the comparative experiments on the IoU score evaluation metric (the larger the IoU value, the better the experimental result)
[0085] Comparison method BUSI ISIC2018 GlaS PolS U-Net 61.44±0.012 77.53±0.001 79.16±0.032 67.57±0.053 MultiResUNet 60.94±0.006 81.2±0.008 78.61±0.012 69.93±0.011 SwinB 68.08±0.007 80.61±0.003 79.02±0.076 82.3±0.005 UCTransNet 66.09±0.01 77.21±0.006 77.66±0.028 69.58±0.041 Swin-Unet 62.65±0.001 78.11±0.004 77.29±0.007 70.17±0.008 SMESwin Unet 58.72±0.011 77.06±0.002 74.35±0.005 69.09±0.009 UNeXt 64.67±0.005 79.4±0.005 76.64±0.001 64.33±0.018 ACC-UNet 64.19±0.023 79.47±0.008 78.43±0.024 72.5±0.007 MADGNet 67.25±0.008 81.04±0.009 80.26±0.011 74.65±0.011 M2ANet (Ours) 70.06±0.003 81.58±0.014 82.53±0.012 82.48±0.007
[0086] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.
Claims
1. A medical image segmentation method based on a multi-branch multi-scale attention mechanism, characterized in that It includes the following steps: S1: Data preprocessing Obtain a medical image dataset and perform image enhancement processing on the medical images in the dataset; S2: Shallow semantic feature extraction Input the medical images after image enhancement processing into the encoder of the medical segmentation model for multi-level feature extraction to obtain a feature map that can represent shallow semantic features; S3: Feature decoding Input the feature map obtained in step S2 into the decoder of the medical segmentation model, and perform decoding processing on the feature map through a multi-branch multi-scale attention mechanism to obtain the local details and global context information of the medical image, and perform multi-scale fusion on the information to obtain a fused feature map; S4: Image segmentation Input the fused feature map obtained in step S3 into the segmentation head of the medical segmentation model for segmentation operation to obtain a segmentation result.
2. The medical image segmentation method based on a multi-branch multi-scale attention mechanism according to claim 1, wherein In step S1, the image enhancement processing includes normalization, random flipping, and random scaling processing. When performing image enhancement processing, the medical images in the dataset and the corresponding segmentation label masks are subjected to image enhancement processing simultaneously.
3. The medical image segmentation method based on the multi-branch multi-scale attention mechanism according to claim 1, characterized in that, In step S2, the encoder encodes the shallow semantic features of the input image layer by layer to obtain a feature map that can represent shallow semantic features. The input image is the medical image. The encoder includes a five-layer encoding structure, and each layer of the encoding structure performs convolution and downsampling operations. Moreover, each layer of the encoding structure not only inputs the result into the corresponding decoding structure but also passes it down to the next layer of the encoding structure.
4. The medical image segmentation method based on a multi-branch multi-scale attention mechanism according to claim 3, wherein In step S3, the decoder includes an adaptive multi-branch patch module, a convolutional block attention module integrating residual blocks, and a multi-scale feature fusion module; the adaptive multi-branch patch module is used for deep feature extraction of the input feature map, the convolutional block attention module integrating residual blocks is used to retain the important features in the feature map obtained by the adaptive multi-branch patch module, and the multi-scale feature fusion module extracts multi-scale information through pooling operations of different scales and fuses the multi-scale information to obtain a fused feature map.
5. The medical image segmentation method based on the multi-branch multi-scale attention mechanism according to claim 4, wherein The adaptive multi-branch patch module includes three branches, and two of the branches are used for adaptive patch operations. The specific processing process is as follows: S301: For the input x of the i-th layer i , different branches divide the input into ρ×ρ blocks along the two dimensions of the width W and height H of the input through a chunking operation, and use average pooling operation and two MLPs to process the feature map to obtain an intermediate result l i；ρ , ρ = 2, 4; S302: Each branch calculates the weight map of each branch according to the corresponding intermediate result l i；ρ to adjust the weights of each branch; S303: Introduce a learnable parameter to further adjust the weight of each branch, and obtain the intermediate result APM by multiplying the intermediate result l i；ρ by the weight map ρ ; The other branch replaces the adaptive patch operations of the above two branches through two convolution operations to obtain the corresponding output; Finally, the outputs of the three branches are added together, and random dropout and normalization operations are performed to obtain the output m of the adaptive multi-branch patch module i , the output m i is a feature map with only the lesion location and contour 6. The medical image segmentation method based on the multi-branch multi-scale attention mechanism according to claim 5, wherein, The convolutional block attention module of the integrated residual block includes the channel attention block of the integrated residual block and the spatial attention block of the integrated residual block. The channel attention block of the integrated residual block obtains two channel-weighted feature maps through global average pooling and global max pooling. After adding the two feature maps and performing the Sigmoid operation, the result is multiplied pixel by pixel with the residual block and then added to obtain the channel attention map. The spatial attention block of the integrated residual block processes the channel attention map through average pooling and max pooling to obtain two spatially weighted feature maps, which are added and then convolved. After performing the Sigmoid operation on the convolved feature map, the result is multiplied element by element with the residual block and then added to obtain the spatial attention map. For the input m i , m i first passes through the channel attention block of the integrated residual block and then through the spatial attention block of the integrated residual block to obtain the output z of the convolutional block attention module of the integrated residual block i , and the output z i is a feature map regarding the details of the lesion.
7. The medical image segmentation method based on the multi-branch multi-scale attention mechanism according to claim 6, characterized in that, In the multi-scale feature fusion module, for the input z i , three sizes of pooling operations are used to pool the original input feature map into feature blocks of different scales. The feature set containing detailed information is obtained through splicing and used as the key vector value and value vector value of the multi-head attention mechanism. By rearranging the original input, it is used as the query vector value of the multi-head attention mechanism. And position encoding is added to the query vector, key vector, and value vector. The features are fused through the attention operation to obtain the attention map. The attention map is processed by two MLPs and a normalization operation is performed to obtain the output s i of the multi-scale feature module. The output s i is the fused feature map.
8. The medical image segmentation method based on a multi-branch multi-scale attention mechanism according to claim 1 or 7, characterized in that, The medical image segmentation method based on the multi-branch multi-scale attention mechanism further includes the following steps: S5: Model parameter optimization Set the parameters of the model, calculate the overall loss value between the segmentation result and the true result through the overall loss function, and use the backpropagation algorithm and stochastic gradient descent to update the parameters of the model until the model reaches the iteration threshold.
9. The medical image segmentation method based on a multi-branch multi-scale attention mechanism according to claim 8, wherein In the step S5, the overall loss function L Total includes the weighted Dice loss L Dice and the cross-entropy loss L BCE , and the specific calculation formula is as follows: L Total = αL BCE + βL Dice Among them, α and β are the weight parameters of the corresponding type of loss.
10. The medical image segmentation method based on a multi-branch multi-scale attention mechanism according to claim 9, wherein, Weighted Dice Loss L Dice is calculated as follows: where N represents the number of pixels and M represents the number of batches, and are the weighted true value and predicted value, respectively; Cross-entropy loss L BCE The calculation formula is as follows: Among them, t i and p i represent the true value and the predicted value respectively, pos i and neg i represent that in the true value, pixel i is a positive class and a negative class respectively. N pos and N neg represent the total numbers of the positive class and the negative class in the true value respectively. w pos and w neg represent the weight values for the positive class and the negative class.