Double-attention mechanism medical image segmentation method based on improved TransUnet network

Abstract

The invention belongs to the technical field of medical image segmentation, and particularly provides a double-attention mechanism medical image segmentation method based on an improved TransUnet network, which is used for improving the segmentation precision of a medical image. According to the double-attention-mechanism image segmentation network architecture based on the TransUnet network, firstly, an SE attention mechanism module is added and fused to an information extraction module part of an encoder to enhance utilization of detail information of a local area; then, a two-channel attention mechanism module (CBAM module) is added to an encoder part and a jump connection layer part, so that the training precision of the model is improved, the problems of insufficient utilization of medical image data and insufficient information fusion are solved, the image information is utilized more comprehensively and more carefully, and the accuracy of the model is improved. And finally, the segmentation result can be more convincing and accurate.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image segmentation, and specifically provides a medical image segmentation method based on a dual attention mechanism of an improved TransUnet network. Background Art

[0002] With the rapid development of artificial intelligence, cross-applications derived from artificial intelligence and medical treatment are gradually gaining a larger share in the development process; image segmentation, as a relatively mature classic application in the field of computer vision, can provide concise and reliable image feature information, effectively improve the processing efficiency of subsequent visual tasks, and is of great significance, especially in the current auxiliary role in medical imaging, which has achieved an effect that cannot be ignored. The use of limited image segmentation technology can realize the regional segmentation, quantitative analysis, pathological detection, volume calculation and other functions of skeletal muscles in medical images, provide auxiliary decision-making basis for disease diagnosis, and assist medical staff in pre-diagnosis of clinical diseases, analysis of prognosis results, and establishment of corresponding diagnostic measures. Provide scientific basis, which can better improve the recovery rate of patients and hospitals.

[0003] At present, the field of image segmentation is represented by network architectures such as Transformer and Unet, which have achieved good results. However, these network architectures are not suitable for the segmentation of all image areas, and there are still many limitations for medical images. The main reasons are: 1) The amount of data is small, the available data sources are relatively few or the data has a lot of noise and its usability is limited; 2) Image information is not fully utilized and features are severely localized. In the process of information transmission, some information cannot be used, and there are also problems such as local area detail information not being used and the model being overfitted, resulting in weak model generalization ability, which not only greatly reduces the segmentation effect, but also wastes existing precious data; 3) When the training task is too small or the amount of data is small, the model features cannot be integrated and the feature information is redundant, making it impossible for the model to correctly, effectively and comprehensively represent the model information, and it is very susceptible to environmental influences.

[0004] In summary, although the TransUnet network structure performs well in image segmentation applications, there is still room for improvement in data utilization and network structure optimization; therefore, according to the actual application scenarios, it is necessary to flexibly use other frameworks of the deep learning model and propose further optimized image segmentation methods to meet the needs of different segmentation tasks, so as to achieve more efficient segmentation and prediction of medical images, and make the segmentation results more accurate and convincing. Summary of the invention

[0005] The purpose of the present invention is to provide a medical image segmentation method with a dual attention mechanism based on an improved TransUnet network to improve the segmentation accuracy of medical images; the present invention is based on the dual attention mechanism image segmentation network architecture of the TransUnet network. First, an SE attention mechanism module is added and integrated into the information extraction module part of the encoder to enhance the utilization of local area detail information; then, a dual-channel attention mechanism module (CBAM module) is added to the encoder part and the jump connection layer part to improve the training accuracy of the model, and solve the problems of insufficient utilization of medical image data and insufficient information fusion, so as to make fuller and more detailed use of image information, and finally make the segmentation result more convincing and accurate.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A dual attention mechanism medical image segmentation method based on an improved TransUnet network, characterized in that it includes the following steps:

[0008] An image segmentation model based on an improved TransUnet network is constructed and trained; the image segmentation model includes: an encoder, a decoder and an output layer; the encoder includes: a SE-ResNet50 module, a CBAM attention mechanism module, a Linear Projection module and a ViT-Transformer module, wherein the SE-ResNet50 module includes: a stem module and three Block modules. After the input image X is sent to the SE-ResNet50 module, the stem module and the three Block modules sequentially output feature maps H0, feature maps X1, feature maps X2 and feature maps X3. Feature maps H0, feature maps X1 and feature maps X2 are correspondingly jumped connected to the encoder. Feature map X3 is reshaped after passing through the CBAM attention mechanism module, the Linear Projection module and the ViT-Transformer module in turn to obtain feature map Y0 and send it to the decoder; the decoder includes: three upsampling modules, which sequentially output feature maps Y1 to Y3, and feature map Y3 passes through the output layer to obtain a segmentation mask Y;

[0009] The medical image to be processed is input into the trained image segmentation model, and the model outputs the segmentation mask to complete the medical image segmentation.

[0010] Furthermore, in the SE-ResNet50 module, the specific structure of the stem module is: Conv(7×7,2,3)+GroupNorm layer+ReLu activation function+pooling layer. The input of the stem module is image X and the output is feature map X0'. At the same time, the feature map H0 is derived from the convolution layer Conv(7×7,2,3);

[0011] The three Block modules are represented as Block1, Block2, and Block3 respectively. The specific structure of Block1 is unit1×3, the specific structure of Block2 is unit2+unit1×3, and the specific structure of Block3 is unit2+unit1×8; the input of Block1 is feature map X0' and the output is feature map X1, the input of Block2 is feature map X1 and the output is feature map X2, and the input of Block3 is feature map X2 and the output is feature map X3;

[0012] Both unit2 and unit1 use unit units, which include residual blocks and SE modules. The input of unit units is represented by feature maps A. in , feature map A in After the residual block, feature map A1 is obtained. After feature map A1 passes through the SE module, feature map A2 is obtained. Feature map A2 and feature map A1 are scaled to obtain feature map A3. Feature map A3 and feature map A in Perform residual connection and finally obtain the output feature map A of the unit unit out .

[0013] Furthermore, the specific structure of the SE module is: Global Pooling layer + fully connected layer + ReLu activation function + fully connected layer + Sigmoid activation function.

[0014] Furthermore, the specific structure of the residual block in unit1 is: Conv(1×1,1,0)+Conv(3×3,1,1)+Conv(1×1,1,0), and the specific structure of the residual block in unit2 is: Conv(1×1,2,0)+Conv(3×3,2,1)+Conv(1×1,2,0).

[0015] Furthermore, the CBAM attention mechanism module includes: channel attention mechanism module and spatial attention mechanism module. The channel attention mechanism module includes: average pooling layer, maximum pooling layer, MLP perceptron and Sigmoid activation function, which is expressed as:

[0016]

[0017] Among them, F represents the input of the channel attention mechanism module, F′ represents the output of the channel attention mechanism module, AvgPool represents the average pooling layer, MaxPool represents the maximum pooling layer, MLP represents the MLP perceptron, and σ represents the Sigmoid activation function;

[0018] The spatial attention mechanism module includes: average pooling layer, maximum pooling layer, convolution layer and Sigmoid activation function, expressed as:

[0019]

[0020] Among them, F″ represents the output of the spatial attention mechanism module, and Conv represents the convolutional layer Conv(7×7,1,3).

[0021] Furthermore, the specific structure of the Linear Projection module is: Conv(1×1,1,0)+flattening layer+position encoding layer+dropout layer.

[0022] Furthermore, the ViT-Transformer module consists of 12 layers of ViT-Transformer Layer, which includes: two Norm normalization layers, a dropout layer, an MSA multi-head attention mechanism layer and an MLP layer. The input of the ViT-Transformer Layer is represented as a feature vector B in , the eigenvector B in After passing through the first Norm normalization layer + dropout layer + MSA multi-head attention mechanism layer, the feature vector B1 is obtained. The feature vector B1 and the feature vector B in Perform residual connection to obtain feature vector B2. Feature vector B2 passes through the second Norm normalization layer + MLP layer to obtain feature vector B3. Feature vector B3 is residually connected with feature vector B2 to obtain feature vector B out .

[0023] Furthermore, the specific structure of the MLP layer is: Linear layer + GELU layer + dropout layer + Linear layer + GELU layer + dropout layer.

[0024] Furthermore, the specific process of reshaping is as follows: first, the feature map X6 output by the ViT-Transformer module is reshaped, and the two-dimensional feature vector is restored to a three-dimensional feature map, and then the feature map Y0 is output after convolution operation through the convolution layer Conv(1×1,1,0) and sent to the decoding module.

[0025] Furthermore, in the decoder, the specific structure of each upsampling module is: upsampling layer + Conv(3×3,1,1) + ReLu activation function;

[0026] The input of the first upsampling module is feature map Y0, and the skip connection feature map X2; feature map Y0 is passed through the upsampling layer to obtain feature map Y0', feature map X2 is passed through the CBAM attention mechanism module to obtain feature map X2', feature map X2' and feature map Y0' are concatenated to obtain feature map Y0", and feature map Y0" is passed through Conv(3×3,1,1)+ReLu activation function to obtain feature map Y1;

[0027] The input of the second upsampling module is feature map Y1, and the skip connection feature map X1; feature map Y1 passes through the upsampling layer to obtain feature map Y1', feature map X1 passes through the CBAM attention mechanism module to obtain feature map X1', feature map X1' and feature map Y1' are concatenated to obtain feature map Y1", and feature map Y1" passes through Conv(3×3,1,1)+ReLu activation function to obtain feature map Y2;

[0028] The input of the third upsampling module is feature map Y2, and the feature map H0 is skip-connected; feature map Y2 is passed through the upsampling layer to obtain feature map Y2', feature map H0 is passed through the convolution layer Conv(1×1,1,0) to obtain feature map H1, feature map H1 is passed through the CBAM attention mechanism module to obtain feature map H1', ​​feature map H1' and feature map Y2' are concatenated to obtain feature map Y2", and feature map Y2" is passed through Conv(3×3,1,1)+ReLu activation function to obtain feature map Y3.

[0029] Furthermore, the output layer adopts a convolutional layer Conv(1×1,1,0), and the feature map Y3 is upsampled again to obtain the feature map Y3', and the feature map Y3' passes through the output layer to obtain the segmentation mask Y.

[0030] Based on the above technical solution, the beneficial effects of the present invention are:

[0031] The present invention provides a dual attention mechanism medical image segmentation method based on an improved TransUnet network, which can effectively improve the accuracy of medical image segmentation. Specifically:

[0032] The present invention can improve segmentation performance and reduce the problem of model overfitting. By adding SE modules and CBAM modules to the key network layers of the encoder, the recognition accuracy of the model for subtle structures in medical images can be improved. At the same time, the introduction of the dual attention mechanism module as a regularization method helps to reduce the overfitting of the model and improve the generalization ability of the model on unknown data.

[0033] The present invention can improve the model's localization ability of information utilization and enhance the model's representation ability; the Transformer part in TransUNet is good at capturing global context, while the SE attention mechanism module helps to dynamically adjust the importance of feature channels, enhance the expression ability of important features, and suppress irrelevant features; the CBAM module can optimize the feature fusion process, so that features at different levels can be more effectively combined, improving the overall feature representation ability, which is particularly important for accurate medical image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a model framework diagram of the dual attention mechanism medical image segmentation method based on the improved TransUnet network in the present invention.

[0035] Figure 2 This is a network structure diagram of a single unit of the Block module of the SE-ResNet50 module in the present invention.

[0036] Figure 3 This is the network structure diagram of the CBAM attention mechanism module (CBAM-Block) in the present invention.

[0037] Figure 4 This is a network structure diagram of the channel attention module (CAM) of the CBAM attention mechanism module in the present invention.

[0038] Figure 5 This is a network structure diagram of the spatial attention module (SAM) of the CBAM attention mechanism module in the present invention.

[0039] Figure 6 This is a network structure diagram of the Transformer Layer of the ViT-Transformer module in the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0041] This embodiment provides a dual attention mechanism medical image segmentation method based on an improved TransUnet network to complete the image segmentation task of CT medical images. The process is as follows: Figure 1 As shown, the specific steps include:

[0042] Step 1: Perform random inversion, rotation and shear preprocessing operations on the input original image. After preprocessing, the image is represented as X0;

[0043] Step 2: Use the SE-ResNet50 module to extract the features of the input image data. SE-ResNet50 includes a stem module and three Block modules. The Block module introduces the SE module into the residual block of the ResNet50 network.

[0044] Specifically, the SE-ResNet50 module consists of a stem module and three Block modules;

[0045] The specific structure of the stem module is: Conv(7×7,2,3)+GroupNorm layer+ReLu activation function+pooling layer. The input of the stem module is image X0 (size is 224×224×3), and the output is feature map X0'. The size of X0' is 56×56×64, that is, the number of output channels C is 64. It should be noted that: Conv(k×k,s,p) in the present invention represents a convolutional layer with a convolution kernel size of k×k, a stride of s, and a padding of p; the size of the feature map or feature matrix is ​​expressed as H×W×C, where H, W, and C represent the length, width, and number of channels of the feature map respectively.

[0046] The three Block modules are represented as Block1, Block2, and Block3 respectively. The specific structure of Block1 is unit1×3, the specific structure of Block2 is unit2+unit1×3, and the specific structure of Block3 is unit2+unit1×8; the input of Block1 is feature map X0', and the output is feature map X1 (size is 56×56×256), the input of Block2 is feature map X1, and the output is feature map X2 (size is 28×28×512), and the input of Block3 is feature map X2, and the output is feature map X3 (size is 14×14×1024);

[0047] Furthermore, both unit2 and unit1 adopt unit units, and the unit units are as follows: Figure 2 As shown, it includes: residual block and SE module, and the input of unit unit is represented as feature map A in , feature map A in After the residual block, feature map A1 is obtained. After feature map A1 passes through the SE module, feature map A2 is obtained. Feature map A2 and feature map A1 are scaled (point multiplication between feature matrices) to obtain feature map A3. Feature map A3 and feature map A in Perform residual connection and finally obtain the output feature map A of the unit unit out ;

[0048] The specific structure of the SE module is: Global Pooling layer + fully connected layer + ReLu activation function + fully connected layer + Sigmoid activation function, where the Global Pooling layer is used as the compression part (F sq ), fully connected layer + ReLu activation function + fully connected layer + Sigmoid activation function as the excitation part (F ex ); The compression part uses a global average pooling operation on the feature map along the direction of the number of channels C to capture the feature map information of each channel; the excitation part uses the feature information obtained by compression to fully capture the channel dependency and learn the nonlinear interaction and non-mutually exclusive relationship between the information of the channels;

[0049] The difference between unit2 and unit1 is that the network parameters in the residual block are different. Specifically, the specific structure of the residual block in unit1 is: Conv(1×1,1,0)+Conv(3×3,1,1)+Conv(1×1,1,0), and the specific structure of the residual block in unit2 is: Conv(1×1,2,0)+Conv(3×3,2,1)+Conv(1×1,2,0);

[0050] Step 3: Input the image data feature X3 obtained in step 2 into the CBAM attention mechanism module (CBAM-Block), combine channel attention and spatial attention to achieve dual refinement of image data features, and obtain a more comprehensive and effective feature representation;

[0051] Specifically, the CBAM attention mechanism module is as follows Figure 3 As shown, it includes: a channel attention mechanism module (ChannelAttention Module, referred to as CAM module) and a spatial attention mechanism module (SpatialAttention Module, referred to as SAM module). The input of the CBAM attention mechanism module is the feature map X3 (size is 14×14×1024), and the output is the feature map X4 (size is 14×14×1024).

[0052] The channel attention mechanism module is as follows Figure 4 As shown, it includes: average pooling layer (AvgPool), maximum pooling layer (MaxPool), MLP perceptron and Sigmoid activation function, which are specifically expressed as:

[0053]

[0054] Among them, F represents the input of the channel attention mechanism module, that is, the feature map X3, F′ represents the output of the channel attention mechanism module, AvgPool represents the average pooling layer, MaxPool represents the maximum pooling layer, MLP represents the MLP perceptron, σ represents the Sigmoid activation function, Represents the dot product operation;

[0055] In more detail, the feature map F is passed through the parallel AvgPool and MaxPool layers to obtain the output feature map F avg 、F max , where AvgPool pooling obtains aggregated spatial information, while MaxPooling collects important clues of features to infer more refined channel attention. In other words, AvgPool obtains macroscopic information, while MaxPool obtains local information. At the same time, using MaxPool can use the network to obtain robustness; feature map F avg 、F max Input them into the MLP perceptron respectively, add the two results output by the MLP and activate them using the activation function to obtain the output feature map M c (F), M c The size of (F) is 1×1×1024;

[0056] The spatial attention mechanism module is as follows Figure 5 As shown, it includes: average pooling layer (AvgPool), maximum pooling layer (MaxPool), convolution layer and Sigmoid activation function, which are specifically expressed as:

[0057]

[0058] Among them, F″ represents the output of the spatial attention mechanism module, that is, the feature map X4, and Conv represents the convolutional layer Conv(7×7,1,3);

[0059] Step 4: Send the image data feature map X4 obtained in step 3 to the Linear Projection module and convert it into patch form;

[0060] The specific structure of the Linear Projection module is: Conv(1×1,1,0)+flattening layer+position encoding layer+dropout layer. The Linear Projection module is mainly used to convert the feature map X4 into a patch form suitable for processing by the Transformer Layer in the ViT-Transformer module (i.e., the vector form of word embedding). The feature map X4 first passes through a convolution operation with a convolution kernel size of 1×1, a step size of 1, and a number of convolution kernels of 768, and then outputs a feature vector of size 14×14×768. The vector is further passed through a flattening layer, which directly flattens the feature vector after the convolution layer in the spatial dimension to retain each data feature. The flattened output size is a token vector (i.e., patch) of 196×768. The position encoding layer then provides a position vector for the token vector, that is, a position vector is added to each token vector, and the size of the position vector is consistent with the token. After the flattened feature vector is added with the position vector, it passes through the dropout layer, and outputs a feature vector X5 of size 196×768 in the Hidden Feature layer.

[0061] Step 5: The image data feature vector X5 obtained in step 4 is downsampled and input into the ViT-Transformer module to further obtain the feature vector X6;

[0062] Specifically, the feature vector X5 is input into the ViT-Transformer module after downsampling, and after passing through the ViT-Transformer module, the feature vector X6 with a size of 196×768 is output in the Hidden Feature layer;

[0063] The ViT-Transformer module is composed of 12 layers of ViT-Transformer Layer. Figure 6 As shown in Figure 1, it includes: two Norm normalization layers, a dropout layer, an MSA multi-head attention mechanism layer, and an MLP layer; the input of the ViT-Transformer Layer is represented as a feature vector B in , the eigenvector B in After passing through the first Norm normalization layer + dropout layer + MSA multi-head attention mechanism layer, the feature vector B1 is obtained. The feature vector B1 and the feature vector B in Perform residual connection to obtain feature vector B2. Feature vector B2 passes through the second Norm normalization layer + MLP layer to obtain feature vector B3. Feature vector B3 is residually connected with feature vector B2 to obtain feature vector B out, the eigenvector B out With the eigenvector B in Keep the same size;

[0064] The specific structure of the MLP layer is: Linear layer + GELU layer + dropout layer + Linear layer + GELU layer + dropout layer, where the first linear layer can be expressed as W1 represents the linear matrix (size is 768×3072), b1 represents the bias (size is 1×3072), and the second linear layer can be expressed as W2 represents the linear matrix (size is 3072×768), b2 represents the bias (size is 1×3072);

[0065] Step 6: Reshape the feature vector obtained by the encoder and decode it. Finally, decode it through Segmentations and output it.

[0066] Specifically, the feature vector X6 is first reshaped from 196×768 to 14×14×768, and then convolution is performed through the convolution layer Conv(1×1,1,0) to output the feature map Y0 (size is 14×14×512), which is then sent to the decoding module.

[0067] The decoding module includes three upsampling modules, which conform to the structure of the Unet decoder. The upsampling recovery is used to decode the features to output the final segmentation mask. The specific structure of each upsampling module is: upsampling layer + Conv (3 × 3, 1, 1) + ReLu activation function;

[0068] The input of the first upsampling module is feature map Y0, and the skip connection feature map X2; feature map Y0 passes through the upsampling layer to obtain feature map Y0' (size is 28×28×512), feature map X2 passes through the CBAM attention mechanism module to obtain feature map X2', feature map X2' and feature map Y0' are concatenated to obtain feature map Y0", and feature map Y0" passes through Conv(3×3,1,1)+ReLu activation function to obtain feature map Y1 (size is 28×28×256);

[0069] The input of the second upsampling module is feature map Y1, and the skip connection feature map X1; feature map Y1 passes through the upsampling layer to obtain feature map Y1' (size is 56×56×256), feature map X1 passes through the CBAM attention mechanism module to obtain feature map X1', feature map X1' and feature map Y1' are concatenated to obtain feature map Y1", and feature map Y1" passes through Conv(3×3,1,1)+ReLu activation function to obtain feature map Y2 (size is 56×56×128);

[0070] The input of the third upsampling module is feature map Y2, and the skip connection feature map H0; feature map Y2 is passed through the upsampling layer to obtain feature map Y2' (size is 112×112×256), feature map X0 is passed through the stem module Conv(7×7,2,3) to obtain feature map H0, feature map H0 is passed through the convolution layer Conv(1×1,1,0) to obtain feature map H1, feature map H1 is passed through the CBAM attention mechanism module to obtain feature map H1', ​​feature map H1' and feature map Y2' are concatenated to obtain feature map Y2", and feature map Y2" is passed through Conv(3×3,1,1)+ReLu activation function to obtain feature map Y3 (size is 112×112×64);

[0071] The output layer uses a convolutional layer Conv(1×1,1,0). The feature map Y3 is upsampled again to obtain the feature map Y3'. The feature map Y3' passes through the output layer to complete the output data dimension setting and output the segmentation mask Y (size is 224×224×16);

[0072] It should be noted that the above step 1 is sample data preprocessing, which increases sample diversity through random inversion, rotation and shearing preprocessing operations; all modules in the above steps 2 to 6 together constitute a complete neural network model according to the data flow, such as Figure 1 As shown, the neural network model has completed offline training in advance. In the image segmentation task of CT medical images, the CT medical image to be segmented is input into the trained neural network model, and the model outputs a segmentation mask.

[0073] Based on the above embodiment, in order to illustrate the advantages of the present invention, "TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation" is used as a comparative example, and the present embodiment and the comparative example are compared and tested, and DSC, HD95 and Loss are used as indicators. The comparison results are shown in the following table:

[0074] DSC HD95 Loss Comparative Example 77.4842％ 31.6935 0.0581 This embodiment 78.5522％ 18.2502 0.0374

[0075] It can be seen from the table that the DSC of this embodiment has better effect, which indicates that the more accurate segmentation effect can better capture the characteristics of the target area and thus improve the overall performance; the HD95 index of this embodiment is lower than that of the comparative example, which shows that this embodiment is more outstanding in the accuracy of the target boundary, effectively reducing the maximum distance between the prediction and the actual target and reducing the risk of false detection and missed detection; in terms of Loss, it shows that this embodiment has faster convergence speed and more stable performance in the training process, which can effectively reduce the time and resource consumption required for training.

[0076] The above description is only a specific implementation mode of the present invention. Any feature disclosed in this specification, unless otherwise stated, can be replaced by other alternative features that are equivalent or have similar purposes; all the disclosed features, or all the steps in the methods or processes, except for mutually exclusive features and / or steps, can be combined in any way.

Claims

1. A dual attention mechanism medical image segmentation method based on an improved TransUnet network, characterized in that: The following steps are involved: Build an image segmentation model based on the improved TransUnet network and complete the training; The image segmentation model includes: an encoder, a decoder and an output layer; the encoder includes: an SE-ResNet50 module, a CBAM attention mechanism module, a LinearProjection module and a ViT-Transformer module, wherein the SE-ResNet50 module includes: a stem module and three Block modules. After the input image X is sent to the SE-ResNet50 module, the stem module and the three Block modules sequentially output feature maps H0, feature maps X1, feature maps X2 and feature maps X3. Feature maps H0, feature maps X1 and feature maps X2 are correspondingly jump-connected to the encoder. Feature map X3 is reshaped after passing through the CBAM attention mechanism module, the Linear Projection module and the ViT-Transformer module in sequence to obtain feature map Y0 and send it to the decoder; the decoder includes: three upsampling modules, which sequentially output feature maps Y1 to Y3. Feature map Y3 passes through the output layer to obtain a segmentation mask Y; The medical image to be processed is input into the trained image segmentation model, and the model outputs the segmentation mask to complete the medical image segmentation.

2. According to claim 1, the dual attention mechanism medical image segmentation method based on the improved TransUnet network is characterized in that: In the SE-ResNet50 module, the specific structure of the stem module is: Conv(7×7,2,3)+GroupNorm layer+ReLu activation function+pooling layer. The input of the stem module is image X and the output is feature map X0'. At the same time, the feature map H0 is derived from the convolution layer Conv(7×7,2,3); The three Block modules are represented as Block1, Block2, and Block3 respectively. The specific structure of Block1 is unit1×3, the specific structure of Block2 is unit2+unit1×3, and the specific structure of Block3 is unit2+unit1×8; the input of Block1 is feature map X0' and the output is feature map X1, the input of Block2 is feature map X1 and the output is feature map X2, and the input of Block3 is feature map X2 and the output is feature map X3; Both unit2 and unit1 use unit units, which include residual blocks and SE modules. The input of unit units is represented by feature maps A. in , feature map A in After the residual block, feature map A1 is obtained. After feature map A1 passes through the SE module, feature map A2 is obtained. Feature map A2 and feature map A1 are scaled to obtain feature map A3. Feature map A3 and feature map A in Perform residual connection and finally obtain the output feature map A of the unit unit out .

3. According to claim 2, the dual attention mechanism medical image segmentation method based on the improved TransUnet network is characterized in that: The specific structure of the SE module is: Global Pooling layer + fully connected layer + ReLu activation function + fully connected layer + Sigmoid activation function.

4. According to claim 2, the dual attention mechanism medical image segmentation method based on the improved TransUnet network is characterized in that: The specific structure of the residual block in unit1 is: Conv(1×1,1,0)+Conv(3×3,1,1)+Conv(1×1,1,0), and the specific structure of the residual block in unit2 is: Conv(1×1,2,0)+Conv(3×3,2,1)+Conv(1×1,2,0).

5. According to claim 1, the dual attention mechanism medical image segmentation method based on the improved TransUnet network is characterized in that: The CBAM attention mechanism module includes: channel attention mechanism module and spatial attention mechanism module. The channel attention mechanism module includes: average pooling layer, maximum pooling layer, MLP perceptron and Sigmoid activation function, which is expressed as: Among them, F represents the input of the channel attention mechanism module, F′ represents the output of the channel attention mechanism module, AvgPool represents the average pooling layer, MaxPool represents the maximum pooling layer, MLP represents the MLP perceptron, and σ represents the Sigmoid activation function; The spatial attention mechanism module includes: average pooling layer, maximum pooling layer, convolution layer and Sigmoid activation function, expressed as: Among them, F″ represents the output of the spatial attention mechanism module, and Conv represents the convolutional layer Conv(7×7,1,3).

6. According to claim 1, the dual attention mechanism medical image segmentation method based on the improved TransUnet network is characterized in that: The specific structure of the Linear Projection module is: Conv(1×1,1,0)+flattening layer+position encoding layer+dropout layer.

7. The medical image segmentation method based on the dual attention mechanism of the improved TransUnet network according to claim 1, characterized in that: The ViT-Transformer module consists of 12 layers of ViT-Transformer Layer. The ViT-Transformer Layer includes: two Norm normalization layers, a dropout layer, an MSA multi-head attention mechanism layer, and an MLP layer. The input of the ViT-Transformer Layer is represented as a feature vector B in , the eigenvector B in After passing through the first Norm normalization layer + dropout layer + MSA multi-head attention mechanism layer, the feature vector B1 is obtained. The feature vector B1 and the feature vector B in Perform residual connection to obtain feature vector B2. Feature vector B2 passes through the second Norm normalization layer + MLP layer to obtain feature vector B3. Feature vector B3 is residually connected with feature vector B2 to obtain feature vector B out ; The specific structure of the MLP layer is: Linear layer + GELU layer + dropout layer + Linear layer + GELU layer + dropout layer.

8. According to claim 1, the dual attention mechanism medical image segmentation method based on the improved TransUnet network is characterized in that: The specific process of reshaping is as follows: first, the feature map X6 output by the ViT-Transformer module is reshaped, and it is restored from a two-dimensional feature vector to a three-dimensional feature map. After that, the feature map Y0 is output after convolution operation through the convolution layer Conv(1×1,1,0) and sent to the decoding module.

9. The medical image segmentation method based on the dual attention mechanism of the improved TransUnet network according to claim 1, characterized in that: In the decoder, the specific structure of each upsampling module is: upsampling layer + Conv (3 × 3, 1, 1) + ReLu activation function; The input of the first upsampling module is feature map Y0, and the skip connection feature map X2; feature map Y0 is passed through the upsampling layer to obtain feature map Y0', feature map X2 is passed through the CBAM attention mechanism module to obtain feature map X2', feature map X2' and feature map Y0' are concatenated to obtain feature map Y0", and feature map Y0" is passed through Conv(3×3,1,1)+ReLu activation function to obtain feature map Y1; The input of the second upsampling module is feature map Y1, and the skip connection feature map X1; feature map Y1 passes through the upsampling layer to obtain feature map Y1', feature map X1 passes through the CBAM attention mechanism module to obtain feature map X1', feature map X1' and feature map Y1' are concatenated to obtain feature map Y1", and feature map Y1" passes through Conv(3×3,1,1)+ReLu activation function to obtain feature map Y2; The input of the third upsampling module is feature map Y2, and the feature map H0 is skip-connected; feature map Y2 is passed through the upsampling layer to obtain feature map Y2', feature map H0 is passed through the convolution layer Conv(1×1,1,0) to obtain feature map H1, feature map H1 is passed through the CBAM attention mechanism module to obtain feature map H1', ​​feature map H1' and feature map Y2' are concatenated to obtain feature map Y2", and feature map Y2" is passed through Conv(3×3,1,1)+ReLu activation function to obtain feature map Y3.

10. The medical image segmentation method based on the dual attention mechanism of the improved TransUnet network according to claim 1, characterized in that: The output layer uses a convolutional layer Conv(1×1,1,0). The feature map Y3 is upsampled again to obtain the feature map Y3', and the feature map Y3' passes through the output layer to obtain the segmentation mask Y.

Images