Multi-scale decoupled em attention module and medical image segmentation method

Abstract

The application discloses a multiscale decoupled EM attention module and a medical image segmentation method.The module comprises a plurality of decoupled EM attention branches, which are used for respectively extracting regional features and saliency boundary features of a target to be segmented in the decoupled EM attention branches; and a multiscale attention feature map is constructed, different sparsity attention feature maps generated by the plurality of decoupled EM attention branches are fused, and the multiscale attention feature map is used for segmenting a medical image target; and the medical image segmentation method segments medical image targets with different shapes and sizes by fusing different sparsity attention feature maps generated by the plurality of decoupled EM attention branches.The application can be widely applied to medical image segmentation tasks, can effectively improve medical image segmentation precision, and has wide application prospects.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing and computer vision, and particularly relates to a multiscale decoupled EM attention module and a medical image segmentation method. BACKGROUND

[0002] Medical image segmentation is a process of dividing the region of interest object from a medical image according to certain characteristics. Medical image segmentation plays an important role in clinical processes such as radiotherapy, image-guided surgery and pathological diagnosis. Therefore, it has important academic and application value to study accurate medical image segmentation methods.

[0003] In recent years, with the rapid development of hardware computing resources and artificial intelligence, convolutional neural networks have been widely used in medical image segmentation tasks.

[0004] The UNet proposed in [Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation [C] / / Munich, Germany, International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015: 234-241.] has played an important role in promoting the development of medical image segmentation field, and its segmentation accuracy has surpassed traditional methods. In order to further improve the accuracy of segmentation results,

[0005] The dense atrous convolution module and residual multi-kernel pooling module are proposed in [Gu Z W, Cheng J, Fu H Z, et al. CE-Net: context encoder network for 2D medical images segmentation [J]. IEEE Transactions on Medical Imaging, 2019, 38(10): 2281-2292.] and applied to medical image segmentation tasks. Compared with traditional convolution, the use of atrous convolution can expand the receptive field of the convolution kernel to extract rich context semantic features without increasing the number of convolution layer parameters, thereby enhancing the segmentation performance of the network.

[0006] However, the contrast of medical images is low and the signal-to-noise ratio is low. Firstly, due to the adhesion of damaged tissues and surrounding normal tissues, it is difficult for the above method to completely extract the boundary feature information of the target to be segmented, resulting in inaccurate segmentation of the damaged tissue region. Therefore,

[0007] Nguyen T C, Nguyen T P, Diep G H, et al. CCBANet: cascading context and balancing att-ention for polyp segmentation [C] / / Strasbourg, France, International Conference on Medical Image Computing and Computer-Assisted Intervention, 2021: 633-643. proposed CCBANet to better fuse neighborhood features and global features in the network, thereby enhancing the network's attention at the boundary. However, this method mainly considers feature selection and does not consider the structure design of the network, thus limiting their generalization ability. Secondly, due to the large difference in shape and size of the target to be segmented in medical images, it is difficult for convolutional neural networks to capture multi-scale feature information to accurately segment the target. To solve this problem, Zhang R, Lai P, Wan X, et al. Lesion-Aware Dynamic Kernel for Polyp Segmentation [C] / / Singapore, Singapore, International Conference on Medical Image Computing and Computer Assisted Intervention, 2022: 99-109. proposed a lesion-aware cross-attention module to pass local context features from the encoder to the decoder, increasing attention to key areas in the previous layer prediction map. However, this method only focuses on easily segmented areas, ignoring those difficult to segment. SUMMARY

[0008] In order to overcome the defects existing in the prior art, the purpose of the present application is to provide a multiscale decoupled EM attention module and a medical image segmentation method, which comprises a plurality of decoupled EM attention branches. Under the premise of reducing computational complexity, information containing target region to be segmented and significant boundary features is extracted at the same time. The plurality of decoupled EM attention branches generate attention feature maps of different sparseness, i.e. construct multiscale attention feature maps to segment medical image targets. Especially for medical image targets with adhesion to surrounding tissues and different shapes and sizes, this method can achieve more accurate medical image segmentation.

[0009] The purpose of the present application is achieved by the following technical solutions:

[0010] The multiscale decoupled EM attention module comprises a plurality of decoupled EM attention branches, which are respectively used to extract regional features and saliency boundary features of a target to be segmented; and different sparsity attention feature maps generated by the plurality of decoupled EM attention branches are fused to construct a multiscale attention feature map for segmenting medical image targets of different shapes and sizes.

[0011] The plurality of decoupled EM attention branches are in parallel lines 2-3.

[0012] A medical image segmentation method based on the multiscale decoupled EM attention module, and the specific segmentation steps include:

[0013] Step 1, pre-process the medical image X to be segmented, and initialize the settings before training the segmentation model;

[0014] Step 2, use the encoder of the network to extract features from the medical image X pre-processed in step 1, and output a feature map X0;

[0015] Step 3, input the feature map X0 extracted in step 2 into a plurality of decoupled EM attention branches respectively, and perform linear mapping on the feature map X0 in each decoupled EM attention branch to calculate a feature matrix Query;

[0016] Step 4, randomly initialize a group of subsets μ i with different feature quantities in the spatial dimension in each decoupled EM attention branch, and perform linear mapping on the subsets μ i respectively to calculate feature matrices Key i and Value i , wherein i represents the i-th decoupled EM attention branch, and {i∈N|0≤i≤2};

[0017] Step 5, in each decoupled EM attention branch, the feature matrix Query in step 3 and the feature matrix Key i in step 4 are calculated to obtain a pairwise item pairwise_item i ;

[0018] Step 6, in each decoupled EM attention branch, the subsets μ i in step 4 are calculated to obtain a unary item unary_item i ;

[0019] Step 7, in each decoupled EM attention branch, the pairwise item pairwise_item i generated in step 5 and the unary item unary_item iPixel by pixel addition for feature fusion, get attention weight attention_weights i ;

[0020] Step 8, in each decoupled EM attention branch, the attention weight attention_weights i generated in step 7 is transposed and matrix multiplied with the feature X0 in step 2 to calculate a new set of subsets μ i ;

[0021] Step 9, loop iteration of steps 5-8 until the attention weight attention_weights i and the subset μ i each reach convergence;

[0022] Step 10, in each decoupled EM attention branch, the converged attention weight attention_weights i in step 9 is transposed and matrix multiplied with the converged subset μ i in step 9 to calculate and output the attention feature map Y i ;

[0023] Step 11, the attention feature maps Y i generated by all decoupled EM branches in step 10 are feature fused and feature fused with the feature map X0 output by step 2 to construct a multi-scale attention feature map, outputting a multi-scale attention feature map Y;

[0024] Step 12, the multi-scale attention feature map Y generated by step 11 is decoded by decoder upsampling and outputs a segmentation result.

[0025] Further, the specific steps of step 1 mainly include:

[0026] a) normalizing the value range of the input image X in the RGB color space from [0, 255] to the range [0, 1];

[0027] b) calculating the mean μ and variance σ of the three channels of the input image X in the RGB color space in the training data set, and obtaining a new distribution image after standardizing the distribution of the input image x;

[0028] c) using the combination of binary cross entropy loss and Dice loss as the loss function, using the Adam optimizer, training the maximum iteration number in the range of [100, 150], the batch size in the range of [24, 48], the initial learning rate in the range of [0.001, 0.008], the momentum in the range of [0.85, 0.95] and the weight decay value in the range of [0.0001, 0.0005].

[0029] Further, the step 3 uses 1x1 convolution to perform linear mapping on the feature X0 extracted in step 2, and calculates the feature matrix Query.

[0030] Further, the specific steps of the step 4 include:

[0031] a) Randomly initialize subsets μ i of sizes (B, C, N / 8), (B, C, N / 4) and (B, C, N / 2) in each decoupled EM attention branch, where B represents batch size, C represents channel number, N represents the number of pixels in the spatial dimension of the feature X0 output by step 2, i represents the i-th decoupled EM attention branch, and {i∈N|0≤i≤2};

[0032] b) In each decoupled EM attention branch, use two groups of 1x1 convolution to perform linear mapping on the subsets μ i , respectively, to calculate the feature matrix Key i and the feature matrix Value i .

[0033] Further, the pairwise_item i is calculated in each decoupled EM attention branch in the step 5, and the specific steps include:

[0034] a) Calculate the mean value of the feature space dimension of the feature matrix Key i and Value i , and subtract the mean value of the feature space dimension from the feature matrix Key i and Value i , respectively, which represents the whitening operation;

[0035] b) After transposing the feature matrix Key i that has undergone the whitening operation, perform matrix multiplication with the feature matrix Value i , and perform Softmax normalization operation on the multiplication result to obtain and output the pairwise_item i ;

[0036] Further, the unary_item i is calculated in each decoupled EM attention branch in the step 6, and the specific steps include:

[0037] a) Perform linear mapping on the subset μ i in step 4 by 1x1 convolution, so as to reduce the channel dimension of the subset μ i from C to 1;

[0038] b) to the subset μ i performing channel expansion to raise the channel to C;

[0039] c) performing a Softmax normalization operation to calculate a unary item unary_item i .

[0040] Further, the step 11 of constructing a multi-scale attention feature map includes the following steps:

[0041] a) performing channel concatenation on the attention feature maps Y i performing channel concatenation, where i represents the ith decoupled EM attention branch, and {i∈N|0≤i≤2};

[0042] b) performing feature fusion on the features processed in step 11a) through a composite operation of convolution, BN, and ReLU activation function;

[0043] c) performing residual connection on the features fused in step 11b) and the output features X0 in step 2 to generate and output a multi-scale attention feature map Y.

[0044] Further, the step 12 of outputting a segmentation result map includes the following steps:

[0045] a) performing up-sampling decoding on the multi-scale attention feature map Y processed in step 11 through a decoder;

[0046] b) generating a segmentation probability map through a Sigmoid operation on the image processed in step 12a);

[0047] c) obtaining and outputting a final segmentation result map through threshold processing on the segmentation probability map generated in step 12b).

[0048] Compared with the prior art, the present application has the following beneficial effects:

[0049] 1. The multi-scale decoupled EM attention module of the present application includes multiple decoupled EM attention branches, which can decouple the attention weight into a pair term representing the features between regions and a unary term representing the saliency boundary features, thereby extracting the boundary features of the target to be segmented and the surrounding tissues, greatly reducing the computational complexity.

[0050] 2. The multiple decoupled EM attention branches included in the multi-scale decoupled EM attention module of the present application can simultaneously extract sparse and dense attention features; the sparse attention features filter noise interference and are suitable for segmenting small-size targets; the dense attention features are suitable for capturing large-size targets containing more position information and are more universal.

[0051] 3. The application is based on a multi-scale decoupled EM attention module, which fuses attention feature maps of different sparseness generated by multiple decoupled EM attention branches to construct multi-scale attention feature maps for segmenting medical image targets of different shapes and sizes. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is a flowchart of the application;

[0053] Figure 2 is a structural diagram of the decoupled EM attention branch of the application;

[0054] Figure 3 is a structural diagram of the application for decoupling attention features into the sum of paired items and unary items;

[0055] Figure 4 is a visualization result graph of the ablation experiment on the polyp dataset;

[0056] Figure 5 is a visualization result graph of the comparison experiment on the polyp dataset; DETAILED DESCRIPTION

[0057] The application will be described in further detail below with reference to the accompanying drawings and specific embodiments.

[0058] A multi-scale decoupled EM attention module includes 2-3 parallel decoupled EM attention branches, which are respectively used to extract regional features and saliency boundary features of the target to be segmented; and fuse attention feature maps of different sparseness generated by multiple decoupled EM attention branches to construct multi-scale attention feature maps for segmenting medical image targets of different shapes and sizes.

[0059] Referring to Figures 1 to 5 A medical image segmentation method based on a multi-scale decoupled EM attention module, the specific segmentation steps include:

[0060] Step 1, pre-process the medical image X to be segmented, and initialize the settings before training the segmentation model; the specific steps include:

[0061] a) normalize the value range of the input image X in the RGB color space from [0, 255] to the range [0, 1];

[0062] b) calculate the mean μ and variance σ of the three channels of the input image X in the RGB color space in the training data set, and obtain a new distribution image after standardizing the distribution of the input image x;

[0063] c) A combination of binary cross-entropy loss and Dice loss is used as the loss function. The Adam optimizer is used, with the maximum number of training iterations ranging from [100, 150], the batch size ranging from [24, 48], the initial learning rate ranging from [0.001, 0.008], the momentum ranging from [0.85, 0.95], and the weight decay values ​​ranging from [0.0001, 0.0005].

[0064] Step 2: Use the encoder of the network to extract features from the medical image X after preprocessing in Step 1, and output the feature map X0.

[0065] Step 3: Input the features X0 extracted in Step 2 into multiple decoupled EM attention branches, and perform linear mapping on features X0 in each decoupled EM attention branch to calculate the feature matrix Query; specifically: use 1×1 convolution to perform linear mapping on features X0 extracted in Step 2 to calculate the feature matrix Query.

[0066] Step 4: In each decoupled EM attention branch, randomly initialize a subset μ with different numbers of features in the spatial dimension. i and for subset μ i Perform linear mappings separately to calculate the feature matrix Key. i and Value i Where i represents the i-th decoupled EM attention branch, and {i∈N|0≤i≤2}; its specific steps include:

[0067] a) In each decoupled EM attention branch, randomly initialize subsets μ of size (B,C,N / 8), (B,C,N / 4), and (B,C,N / 2). i Where B represents the batch size, C represents the number of channels, and N represents the number of pixels of the output feature X0 in step 2 in the spatial dimension;

[0068] b) Use two sets of 1×1 convolutions in each decoupled EM attention branch, respectively, on the subset μ i Perform a linear mapping to calculate the feature matrix Key. i and the feature matrix Value i .

[0069] Step 5: In each decoupled EM attention branch, the feature matrix Query from Step 3 and the feature matrix Key from Step 4 are used. i Calculations are performed to obtain the pairwise item. i The specific steps include:

[0070] a) For the feature matrix Key i and Valuei Subtracting the mean of the feature space dimensions from each dimension represents the whitening operation;

[0071] b) The feature matrix Key after whitening operation i After transposing, it is compared with the feature matrix Value. i Perform matrix multiplication, apply Softmax normalization to the results, and output the paired items (pairwise_item). i .

[0072] Step 6: In each decoupled EM attention branch, the subset μ from step 4 is used. i The calculation yields the unary item unary_item. i The specific steps include:

[0073] a) Take the subset μ from step 4 i A linear mapping is performed using 1×1 convolution, thereby transforming the subset μ i The channel dimension is reduced from C to 1;

[0074] b) For subset μ i Expand the channel to C;

[0075] c) After Softmax normalization, the unary item is calculated. i .

[0076] Step 7: In each decoupled EM attention branch, the pairwise item generated in Step 5 is used. i and the unary_item generated in step 6 i Feature fusion is performed by adding features pixel by pixel to obtain attention weights. i Specifically, in each decoupled EM attention branch, the paired item `pairwise_item` is processed. i and unary_item i Perform pixel-wise feature fusion to calculate the attention weights. i .

[0077] Step 8: The attention weights generated in Step 7 for each decoupled EM attention branch. i The converged subset μ is obtained by calculating the feature map X0 output in step 2. i Specifically, in each decoupled EM attention branch, the attention weights generated in step 7 are applied to the attention weights. iAfter transposition, matrix multiplication is performed with the feature map X0 in step 2 to calculate the converged subset μ i .

[0078] Step 9, loop iteration of steps 5-8 until the attention weight attention_weights i and the subset μ i each reach convergence;

[0079] Step 10, in each decoupled EM attention branch, the converged attention weight attention_weights i in step 9 is transposed and matrix multiplied with the converged subset μ i in step 9 to calculate and output the attention feature map Y i ;

[0080] Step 11, the attention feature maps Y i generated by all decoupled EM branches in step 10 are fused and fused with the feature map X0 output by step 2 to output a multi-scale attention feature map Y through a multi-scale attention feature map construction strategy; the specific steps include:

[0081] a) the attention feature maps Y i generated by each decoupled EM attention branch in step 10 are channel connected;

[0082] b) the features processed in step 11 a) are fused through convolution, BN and ReLU activation function operations;

[0083] c) after residual connection between the fused features in step 11 b) and the output feature X0 in step 2, a multi-scale attention feature map Y is generated and output.

[0084] Step 12, the multi-scale attention feature map Y generated by step 11 is decoded by decoder upsampling and outputs a segmentation result; the specific steps include:

[0085] a) the multi-scale attention feature map Y processed in step 11 is decoded by decoder upsampling;

[0086] b) the image processed in step 12 a) is generated into a segmentation probability map through Sigmoid operation;

[0087] c) the segmentation probability map generated in step 12 b) is obtained and output through threshold processing to obtain the final segmentation result map.

[0088] The present application is verified as follows: for ease of description, the following refers to the present application as MSD-EMA: a multi-scale decoupled EM attention module and a medical image segmentation method thereof.

[0089] In order to effectively evaluate the segmentation performance of the MSD-EMA proposed in the present application on medical images, ablation experiments and comparative experiments are carried out on the five published polyp data sets of Kvasir, ETIS, CVC-ColonDB, CVC-ClinicDB and CVC-300. In the Kvasir data set, 900 images are randomly selected as the training data set, and 550 images are randomly selected from the CVC-ClinicDB data set as the training data set, a total of 1450 images; the remaining images of the above five data sets are selected as the test data set, a total of 798 images.

[0090] The verification environment is: the CPU is Intel(R) Xeon(R) Gold 6226R; the memory is 32GB; the GPU is Nvidia Geforce RTX 3090, and the video memory is 24GB. The experiment is carried out on the Ubuntu 16.04.10 operating system. The deep learning framework used to realize the model proposed in the present application is PyTorch.

[0091] In order to further compare the segmentation performance of different network models, IoU and Dice are selected as evaluation indexes to measure the segmentation performance. IoU and Dice are shown in formulas (1) and (2) respectively:

[0092]

[0093]

[0094] Wherein, TP, FP and FN represent true positive, false positive and false negative respectively.

[0095] Firstly, the effectiveness of the decoupled EM attention branch (Decoupled EM Attention, D-EMA) and the multi-scale attention feature map construction strategy designed in the MSD-EMA proposed in the present application is verified through the ablation experiment, and the following ablation experiment is carried out on the five polyp data sets. In the experiment, the maximum number of iterations is 100, the learning rate is 0.001, the momentum parameter is 0.9, and the weight decay parameter is 0.0001.

[0096] For the ablation experiment of the decoupled EM attention branch: the decoupled EM attention branch proposed in the present application is compared with the EMAttention and DNL modules, and the ablation experiment visualization is as shown in Figure 4 Figure 4 ​In the middle, UNet, UNet+EM Attention, UNet+DNL and UNet+D-EMA in Kvasir polyp dataset are sequentially displayed and compared. First, after introducing the EM Attention module alone, the generated attention feature map can focus on more polyp target positions and establish global connections in the feature map. However, due to the interference of similar periphery, a part of the attention feature map focuses on the noise area. Therefore, after introducing the EM Attention module, the segmentation result is improved compared with the original UNet, but there is still noise interference; secondly, after introducing the DNL module alone, the attention feature map extracts and retains the boundary feature information in the image; however, the attention area only focuses on a small part of the polyp to be segmented; therefore, the segmentation result is also improved compared with the original UNet, but the segmented target has the phenomenon of over-segmentation. Finally, after introducing the decoupled EM attention branch, it can be seen that the generated attention feature map focuses more on the segmented area, and the segmentation result is also closer to the label.

[0097] In order to further analyze the performance of introducing the decoupled EM attention branch, the experimental evaluation index is given for quantitative comparison, as shown in Table 1. From the 4th and 7th rows in Table 1, it can be seen that after introducing the decoupled attention branch, the segmentation accuracy of the original UNet and ResNet on the five polyp datasets is significantly improved. The results prove that the decoupled EM attention branch after decoupling the solving process of the attention feature map of the EMAttention module can not only focus on the polyp area through the established global connection, but also extract the boundary information of the polyp, which can effectively improve the segmentation performance.

[0098] Table 1 Ablation experiment results on five polyp datasets

[0099]

[0100] For the ablation experiment of the strategy of constructing multi-scale attention feature maps: in order to verify the effectiveness of the constructed multi-scale attention feature maps, the MSD-EMA is introduced in the last column of the ablation experiment results in Table 1. Figure 4 As shown in Table 1, the last column shows the multi-scale attention feature map and segmentation result after introducing the MSD-EMA. As shown in Table 1, compared with the introduction of the decoupled EM attention branch, the constructed multi-scale attention feature map can obviously improve the attention degree in the target polyp area and reduce the attention degree in the irrelevant area, thereby avoiding the interference of noise; and it can be found that the segmentation result is also closer to the label image. Figure 4

[0101] ​To further quantify the improvement effect of the multi-scale attention feature map construction strategy proposed in the present application on the network performance, specific experimental data is also given in Table 1 for evaluation. As can be seen from the 5th and 8th rows of Table 1, the indicators after constructing the multi-scale attention feature map are improved compared with the single-branch decoupled EM attention branch on the five data sets. Among them, the segmentation accuracy is improved the most on the CVC-300 polyp data set, and the IoU and Dice coefficient evaluation indicators are improved by 3% and 2.4%, respectively. In summary, the ablation experiment results of the decoupled EM attention branch and the multi-scale attention feature map construction strategy verify the effectiveness of the MSD-EMA structure design in improving the segmentation accuracy.

[0102] Table 2 is the experimental results of the MSD-EMA proposed in the present application and the current mainstream networks UNet, UNet++, EMA-Net, ResUNet++, PraNet, MS-Net, SA-Net and TGA-Net on five polyp data sets. The IoU scores of the MSD-EMA proposed in the present application are 0.684, 0.691, 0.846 and 0.849, respectively; the Dice scores are 0.763, 0.775, 0.903 and 0.912, respectively, and the best segmentation performance is achieved on the above four data sets. In the CVC-ClinicDB polyp data set, the IoU and Dice scores of the MSD-EMA proposed in the present application are 0.863 and 0.914, respectively, which are only inferior to MS-Net, but still better than other networks. From Figure 5 It can be seen from Table 2 that the MSD-EMA module proposed in the present application enhances the ability of the network to extract features in the polyp region and the boundary, and further improves the accuracy of the polyp segmentation result. In summary, the MSD-EMA proposed in the present application can improve the polyp segmentation accuracy, and the segmentation result on multiple test sets exceeds the current excellent segmentation network.

[0103] Table 2 Performance evaluation of all methods on five polyp data sets (the best value is shown in bold)

[0104]

Claims

1. A multi-scale decoupled EM attention module comprising a plurality of decoupled EM attention branches, characterized in that: The decoupled EM attention branches are respectively used for extracting regional features and saliency boundary features of the target to be segmented. The different sparsity attention feature maps generated by the multiple decoupled EM attention branches are fused to construct a multi-scale attention feature map for segmenting medical image targets of different shapes and sizes. The decoupled EM attention branches are respectively used for extracting regional features and saliency boundary features of the target to be segmented, which are decoupled into a pair of terms representing inter-regional features and a unary term representing saliency boundary features, so that the boundary features of the target to be segmented and the surrounding tissues are extracted. The different sparsity attention feature maps generated by the multiple decoupled EM attention branches are fused to construct a multi-scale attention feature map for segmenting medical image targets of different shapes and sizes, which are: a) each decoupled EM attention branch generates an attention feature map Y i each decoupled EM attention branch generates an attention feature map Y i a channel connection operation is performed; b) performing feature fusion on the features processed in step a) through a composite operation of convolution, BN and ReLU activation function; c) performing residual connection on the features fused in step b) and the medical image X to extract features using the encoder of the network and output a feature map X0, and then generating and outputting a multi-scale attention feature map Y; d) performing up-sampling decoding on the multi-scale attention feature map Y generated in step c) through the decoder; e) generating a segmentation probability map through Sigmoid operation on the image processed in step d); f) obtaining and outputting the final segmentation result map through threshold processing on the segmentation probability map generated in step e); The 2-3 parallel lines of the multiple decoupled EM attention branches can simultaneously extract sparse and dense attention features; wherein the sparse attention features filter noise interference and are suitable for segmenting small-size targets; and the dense attention features are suitable for capturing large-size targets containing more position information.

2. A medical image segmentation method based on a multi-scale decoupled EM attention module, characterized in that, The specific segmentation steps include: Step 1, pre-processing the medical image X to be segmented and initializing the settings before training the segmentation model; Step 2, using the encoder of the network to extract features from the medical image X pre-processed in step 1 and output a feature map X0; Step 3, inputting the feature map X0 extracted in step 2 into the multiple decoupled EM attention branches respectively, and performing linear mapping on the feature map X0 in each decoupled EM attention branch to calculate a feature matrix Query; Step 4, randomly initialize a set of subsets μ in each decoupled EM attention branch i , which have different numbers of features in the spatial dimension i , respectively, and perform linear mapping to obtain feature matrices Key i and Value i , where i represents the ith decoupled EM attention branch, and {i ∈ N | 0 ≤ i ≤ 2} Step 5. In each decoupled EM attention branch, compute the feature matrix Query from the feature matrix Query in Step 3 and the feature matrix Key in Step 4 i Compute the pairwise_item i ; Step 6. In each decoupled EM attention branch, by subset μ i Compute unary_item i ; Step 7, in each decoupled EM attention branch, pairwise_item generated in step 5 is added to unary_item generated in step 6 pixel by pixel to obtain attention_weights i i i ;​​ Step 8, in each decoupled EM attention branch, the attention weights attention_weights generated by step 7 i After transposition, matrix multiplication is performed with the feature X0 in step 2 to obtain a new set of subsets μ i ; Step 9, loop iterates steps 5 - step 8 until attention weights attention_weights i and subset μ i each reach convergence; Step 10, in each decoupled EM attention branch, the converged attention weights attention_weights in step 9 are multiplied by the input feature map X to obtain the attention feature map Y i After transposition, and the converged subset μ in step 9 i Matrix multiplication is performed to calculate and output the attention feature map Y i ; Step 11, all decoupled EM branch generated attention feature map Y in step 10 i Feature fusion is performed, and feature fusion is performed with the feature map X0 output by step 2 to construct a multi-scale attention feature map, and a multi-scale attention feature map Y is output. Step 12, up-sampling decoding the multi-scale attention feature map Y generated in step 11 through the decoder and outputting the segmentation result.

3. The medical image segmentation method based on the multi-scale decoupled EM attention module according to claim 2, characterized in that, The specific steps of step 1 include: a) normalizing the value range of the input image X in the RGB color space from [0, 255] to the range [0, 1]; b) calculating the mean μ and variance σ of the three channels of the input image X in the RGB color space in the training data set, and obtaining a new distribution image after standardizing the distribution of the input image X. c) Training with the combination of binary cross-entropy loss and Dice loss as the loss function, Adam optimizer, the maximum number of iterations ranging from [100, 150], the batch size ranging from [24, 48], the initial learning rate ranging from [0.001, 0.008], the momentum ranging from [0.85, 0.95] and the weight decay value ranging from [0.0001, 0.0005].

4. The medical image segmentation method based on the multi-scale decoupled EM attention module according to claim 2, characterized in that, The step 3 uses a 1x1 convolution to linearly map the features X0 extracted in step 2 to obtain a feature matrix Query.

5. The medical image segmentation method based on the multi-scale decoupled EM attention module according to claim 2, characterized in that, The specific steps of the step 4 include: a) randomly initializing subsets μ of size (B, C, N / 8), (B, C, N / 4) and (B, C, N / 2) in each decoupled EM attention branch i where B represents batch size, C represents channel number, N represents the number of pixels in spatial dimension by step 2 output feature X0, and i represents the i-th decoupled EM attention branch, and {i ∈ N | 0 ≤ i ≤ 2}. b) Use two groups of 1x1 convolution in each decoupled EM attention branch to linearly map the subset μ i , respectively, to compute the feature matrix Key i and the feature matrix Value i .

6. The medical image segmentation method based on the multi-scale decoupled EM attention module according to claim 2, characterized in that, The step 5 calculates a pairwise item in each decoupled EM attention branch i The specific steps include: a) compute the mean of the feature space dimension of the feature matrix Key i and Value i and subtract the mean of the feature space dimension from the feature matrix Key i and Value i respectively, i.e. representing a whitening operation; b) performing a whitening operation on the feature matrix Key i After transposition, the feature matrix Value i After matrix multiplication and Softmax normalization operation on the multiplication result, the pairwise_item is obtained and output i .

7. The medical image segmentation method based on the multi-scale decoupled EM attention module according to claim 2, characterized in that, The unary item unary_item is computed in each decoupled EM attention branch in step 6 i The specific steps include: a) subset μ i is linearly mapped by a 1 x 1 convolution, thereby reducing the channel dimension of subset μ i from C to 1 ; b) for the subset μ i Perform channel dilation, raising the channel to be C; c) After Softmax normalization operation, the unary term unary_item is calculated i .

8. The medical image segmentation method based on the multi-scale decoupled EM attention module according to claim 2, characterized in that, The specific steps of constructing the multi-scale attention feature map in the step 11 include: a) the attention feature maps Y generated by each decoupled EM attention branch in step 10 i performing a channel connection operation, where i represents the ith decoupled EM attention branch, and {i∈N|0≤i≤2}; b) Feature fusion is performed on the features processed in step 11a) through a composite operation of convolution, BN and ReLU activation function; c) After residual connection of the features fused in step 11b) and the output features X0 in step 2, a multi-scale attention feature map Y is generated and output.

9. The medical image segmentation method based on the multi-scale decoupled EM attention module according to claim 2, characterized in that, The specific steps of outputting the segmentation result map in the step 12 include: a) The multi-scale attention feature map Y processed in step 11 is up-sampled and decoded by the decoder; b) The image processed in step 12a) is processed by Sigmoid operation to generate a segmentation probability map; c) The segmentation probability map generated in step 12b) is processed by thresholding to obtain and output the final segmentation result map.

Images