A dual attention mechanism medical image segmentation method based on an improved TransUnet network

Abstract

This invention belongs to the field of medical image segmentation technology, specifically providing a medical image segmentation method based on an improved TransUnet network with a dual attention mechanism to improve the segmentation accuracy of medical images. The dual attention mechanism image segmentation network architecture of this invention, based on the TransUnet network, firstly adds and fuses an SE attention mechanism module to the information extraction module of the encoder to enhance the utilization of local region detail information; then, a dual-channel attention mechanism module (CBAM module) is added to the encoder and skip connection layers to improve the training accuracy of the model and solve the problems of insufficient utilization of medical image data and inadequate information fusion, making the use of image information more comprehensive and detailed, ultimately resulting in more convincing and accurate segmentation results.

Description

Technical Field

[0001] This invention belongs to the field of medical image segmentation technology, specifically providing a medical image segmentation method based on a dual attention mechanism of an improved TransUnet network. Background Technology

[0002] With the rapid development of artificial intelligence, the cross-applications of AI and medicine are gradually gaining a significant share. Image segmentation, as a relatively mature and classic application in computer vision, can provide concise and reliable image feature information, effectively improving the processing efficiency of subsequent visual tasks and playing a vital role, especially in the current field of medical imaging, where its auxiliary role has become undeniable. Finite image segmentation technology can be used to segment, quantitatively analyze, pathologically detect, and calculate volume in skeletal and muscular regions of medical images, providing auxiliary decision-making basis for disease diagnosis. It can assist medical personnel in providing a scientific basis for pre-diagnosis, prognostic analysis, and the establishment of corresponding diagnostic measures, thereby improving patient recovery rates and hospital cure rates.

[0003] Currently, image segmentation, represented by network architectures such as Transformer and UNet, has achieved good results. However, these network architectures are not suitable for segmenting all image regions and still have many limitations for medical images. The main reasons are: 1) The amount of data is relatively small, and the available data sources are limited or contain a lot of noise, which limits their usability; 2) Image information is not fully utilized, and feature localization is severe. During the information transmission process, some information cannot be used, and there are also problems such as the failure to utilize local regional details and overfitting of the model, resulting in poor model generalization ability. This not only greatly reduces the segmentation effect but also wastes the existing valuable data; 3) When the training task is too small or the amount of data is small, the model features cannot be fused, and the feature information is redundant, making the model unable to correctly, effectively, and comprehensively represent the model information, and it is very susceptible to environmental influences.

[0004] In summary, although the TransUnet network structure performs well in image segmentation applications, there is still room for improvement in data utilization and network structure optimization. Therefore, based on the actual application scenarios, it is necessary to flexibly utilize other frameworks of deep learning models and propose further optimized image segmentation methods to meet the needs of different segmentation tasks, so as to achieve more efficient segmentation and prediction of medical images and make the segmentation results more accurate and convincing. Summary of the Invention

[0005] The purpose of this invention is to provide a medical image segmentation method based on an improved TransUnet network with a dual attention mechanism, in order to improve the segmentation accuracy of medical images. The dual attention mechanism image segmentation network architecture of this invention, based on the TransUnet network, firstly adds and fuses an SE attention mechanism module to the information extraction module of the encoder to enhance the utilization of local region detail information; then, a dual-channel attention mechanism module (CBAM module) is added to the encoder and skip connection layers to improve the training accuracy of the model and solve the problems of insufficient utilization of medical image data and inadequate information fusion, making the use of image information more comprehensive and detailed, ultimately resulting in more convincing and accurate segmentation results.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A medical image segmentation method based on a dual attention mechanism of an improved TransUnet network, characterized by the following steps:

[0008] An image segmentation model based on an improved TransUnet network was constructed and trained. The image segmentation model includes an encoder, a decoder, and an output layer. The encoder includes an SE-ResNet50 module, a CBAM attention mechanism module, a Linear Projection module, and a ViT-Transformer module. The SE-ResNet50 module includes a stem module and three block modules. After the input image X is fed into the SE-ResNet50 module, the stem module and the three block modules sequentially output feature maps H0, X1, X2, and X3. Feature maps H0, X1, and X2 are connected to the encoder in a skip connection. Feature map X3 is reconstructed after passing through the CBAM attention mechanism module, the Linear Projection module, and the ViT-Transformer module to obtain feature map Y0, which is then fed into the decoder. The decoder includes three upsampling modules that sequentially output feature maps Y1 to Y3. Feature map Y3 is processed by the output layer to obtain the segmentation mask Y.

[0009] The medical image to be processed is input into the trained image segmentation model, and the model outputs a segmentation mask to complete the medical image segmentation.

[0010] Furthermore, in the SE-ResNet50 module, the specific structure of the stem module is: Conv(7×7,2,3) + GroupNorm layer + ReLU activation function + pooling layer. The input of the stem module is the image X, and the output is the feature map X0'. At the same time, the feature map H0 is derived from the convolutional layer Conv(7×7,2,3).

[0011] The three Block modules are represented as Block1, Block2, and Block3, respectively. The specific structure of Block1 is unit1×3, the specific structure of Block2 is unit2+unit1×3, and the specific structure of Block3 is unit2+unit1×8. The input of Block1 is feature map X0' and the output is feature map X1, the input of Block2 is feature map X1 and the output is feature map X2, and the input of Block3 is feature map X2 and the output is feature map X3.

[0012] Both unit2 and unit1 use unit units, which include residual blocks and SE modules. The input of a unit unit is represented by feature map A. in Feature map A in After passing through the residual block, feature map A1 is obtained. Feature map A1 is then processed by the SE module to obtain feature map A2. Feature map A2 and feature map A1 are then scaled to obtain feature map A3. Feature map A3 and feature map A1 are then compared. in By performing residual connections, the output feature map A of the unit cell is finally obtained. out .

[0013] Furthermore, the specific structure of the SE module is: Global Pooling layer + fully connected layer + ReLU activation function + fully connected layer + Sigmoid activation function.

[0014] Furthermore, the specific structure of the residual block in unit1 is: Conv(1×1,1,0)+Conv(3×3,1,1)+Conv(1×1,1,0), and the specific structure of the residual block in unit2 is: Conv(1×1,2,0)+Conv(3×3,2,1)+Conv(1×1,2,0).

[0015] Furthermore, the CBAM attention mechanism module includes: a channel attention mechanism module and a spatial attention mechanism module. The channel attention mechanism module includes: an average pooling layer, a max pooling layer, an MLP perceptron, and a sigmoid activation function, represented as:

[0016]

[0017] Where F represents the input of the channel attention mechanism module, F′ represents the output of the channel attention mechanism module, AvgPool represents the average pooling layer, MaxPool represents the max pooling layer, MLP represents the MLP perceptron, and σ represents the Sigmoid activation function.

[0018] The spatial attention mechanism module includes: average pooling layer, max pooling layer, convolutional layer, and sigmoid activation function, represented as:

[0019]

[0020] Where F″ represents the output of the spatial attention mechanism module, and Conv represents the convolutional layer Conv(7×7,1,3).

[0021] Furthermore, the specific structure of the Linear Projection module is: Conv(1×1,1,0) + flattening layer + position encoding layer + dropout layer.

[0022] Furthermore, the ViT-Transformer module consists of 12 ViT-Transformer Layers, including: two Norm normalization layers, a dropout layer, an MSA multi-head attention mechanism layer, and an MLP layer. The input to the ViT-Transformer Layer is represented as the feature vector B. in eigenvector B in After passing through the first Norm normalization layer + dropout layer + MSA multi-head attention mechanism layer, feature vector B1 is obtained. Feature vector B1 and feature vector B in A residual connection is performed to obtain feature vector B2. Feature vector B2 is then passed through a second Norm normalization layer + MLP layer to obtain feature vector B3. Feature vector B3 is then residually connected with feature vector B2 to obtain feature vector B. out .

[0023] Furthermore, the specific structure of the MLP layer is: Linear layer + GELU layer + dropout layer + Linear layer + GELU layer + dropout layer.

[0024] Furthermore, the specific process of reshaping is as follows: First, the feature map X6 output by the ViT-Transformer module is reshaped in dimension, restoring it from a two-dimensional feature vector to a three-dimensional feature map. Then, after convolution operation by the convolutional layer Conv(1×1,1,0), the feature map Y0 is output and sent to the decoding module.

[0025] Furthermore, in the decoder, the specific structure of each upsampling module is: upsampling layer + Conv(3×3,1,1) + ReLU activation function;

[0026] The first upsampling module takes feature map Y0 as input and skips the connection to feature map X2. Feature map Y0 is upsampled to obtain feature map Y0'. Feature map X2 is passed through the CBAM attention mechanism module to obtain feature map X2'. Feature map X2' and feature map Y0' are concatenated to obtain feature map Y0'. Feature map Y0' is then passed through the Conv(3×3,1,1)+ReLU activation function to obtain feature map Y1.

[0027] The input to the second upsampling module is feature map Y1, and it skips the connection to feature map X1. Feature map Y1 is upsampled to obtain feature map Y1'. Feature map X1 is passed through the CBAM attention mechanism module to obtain feature map X1'. Feature map X1' and feature map Y1' are concatenated to obtain feature map Y1'". Feature map Y1' is passed through the Conv(3×3,1,1)+ReLU activation function to obtain feature map Y2.

[0028] The input to the third upsampling module is feature map Y2, and it skips the connection to feature map H0. Feature map Y2 is upsampled to obtain feature map Y2'. Feature map H0 is convolved by Conv(1×1,1,0) to obtain feature map H1. Feature map H1 is passed through the CBAM attention mechanism module to obtain feature map H1'. Feature map H1' and feature map Y2' are concatenated to obtain feature map Y2'". Feature map Y2' is passed through Conv(3×3,1,1)+ReLU activation function to obtain feature map Y3.

[0029] Furthermore, the output layer uses a convolutional layer Conv(1×1,1,0), and the feature map Y3 is upsampled again to obtain the feature map Y3'. The feature map Y3' is then passed through the output layer to obtain the segmentation mask Y.

[0030] Based on the above technical solution, the beneficial effects of the present invention are as follows:

[0031] This invention provides a medical image segmentation method based on a dual-attention mechanism using an improved TransUnet network, which can effectively improve the accuracy of medical image segmentation. Specifically:

[0032] This invention can improve segmentation performance and reduce model overfitting. By adding SE and CBAM modules to the key network layers of the encoder, the model can improve the recognition accuracy of fine structures in medical images. At the same time, the introduction of the dual attention mechanism module as a regularization method helps to reduce model overfitting and improve the model's generalization ability on unknown data.

[0033] This invention can improve the model's ability to utilize information locally and enhance its representation ability. The Transformer part in TransUNet is good at capturing global context, while the SE attention mechanism module helps to dynamically adjust the importance of feature channels, enhance the expressive power of important features, and suppress irrelevant features. The CBAM module can optimize the feature fusion process, so that features at different levels can be combined more effectively, improving the overall feature representation ability, which is especially important for accurate medical image segmentation. Attached Figure Description

[0034] Figure 1 This is a model framework diagram of the medical image segmentation method based on the improved TransUnet network with dual attention mechanism in this invention.

[0035] Figure 2 This is a network structure diagram of a single unit cell of the Block module of the SE-ResNet50 module in this invention.

[0036] Figure 3 This is a network structure diagram of the CBAM attention mechanism module (CBAM-Block) in this invention.

[0037] Figure 4 This is a network structure diagram of the Channel Attention Module (CAM) of the CBAM attention mechanism module in this invention.

[0038] Figure 5 This is a network structure diagram of the spatial attention module (SAM) of the CBAM attention mechanism module in this invention.

[0039] Figure 6 This is a network structure diagram of the Transformer Layer of the ViT-Transformer module in this invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0041] This embodiment provides a medical image segmentation method based on a dual-attention mechanism of an improved TransUnet network to complete the image segmentation task of CT medical images. The process is as follows: Figure 1 As shown, the specific steps include:

[0042] Step 1: Perform random reversal, rotation and cropping preprocessing operations on the input original image. The image after preprocessing is represented as X0.

[0043] Step 2: Use the SE-ResNet50 module to extract features from the input image data. SE-ResNet50 includes a stem module and three block modules. The block module introduces the SE module into the residual block of the ResNet50 network.

[0044] Specifically, the SE-ResNet50 module consists of a stem module and three block modules;

[0045] The specific structure of the stem module is: Conv(7×7,2,3) + GroupNorm layer + ReLU activation function + pooling layer. The input of the stem module is image X0 (size 224×224×3), and the output is feature map X0'. The size of X0' is 56×56×64, that is, the number of output channels C is 64. It should be noted that: in this invention, Conv(k×k,s,p) represents a convolutional layer with a kernel size of k×k, a stride of s, and padding of p; the size of the feature map or feature matrix is ​​represented as H×W×C, where H, W, and C represent the length, width, and number of channels of the feature map, respectively.

[0046] The three Block modules are represented as Block1, Block2, and Block3, respectively. The specific structure of Block1 is unit1×3, the specific structure of Block2 is unit2+unit1×3, and the specific structure of Block3 is unit2+unit1×8. The input of Block1 is feature map X0' and the output is feature map X1 (size 56×56×256). The input of Block2 is feature map X1 and the output is feature map X2 (size 28×28×512). The input of Block3 is feature map X2 and the output is feature map X3 (size 14×14×1024).

[0047] Furthermore, both unit2 and unit1 use unit units, which are as follows: Figure 2 As shown, it includes: residual blocks and SE modules, with the input of the unit cell represented as feature map A. in Feature map A in After passing through the residual block, feature map A1 is obtained. Feature map A1 is then processed by the SE module to obtain feature map A2. Feature map A2 and feature map A1 are then scaled (dot multiplication between feature matrices) to obtain feature map A3. Feature map A3 and feature map A1 are then compared. in By performing residual connections, the output feature map A of the unit cell is finally obtained. out ;

[0048] The specific structure of the SE module is: Global Pooling layer + fully connected layer + ReLU activation function + fully connected layer + Sigmoid activation function, where the Global Pooling layer serves as the compression part (F sq The activation part is a fully connected layer + ReLU activation function + fully connected layer + Sigmoid activation function (F). ex The compression part uses global average pooling along the channel number C direction to capture the feature map information of each channel; the excitation part uses the feature information obtained by compression to fully capture the channel dependencies and learn the nonlinear interaction and non-exclusive relationship between the information of the channels.

[0049] The difference between unit2 and unit1 lies in the network parameters in the residual block. Specifically, the structure of the residual block in unit1 is: Conv(1×1,1,0)+Conv(3×3,1,1)+Conv(1×1,1,0), while the structure of the residual block in unit2 is: Conv(1×1,2,0)+Conv(3×3,2,1)+Conv(1×1,2,0).

[0050] Step 3: Input the image data feature X3 obtained in Step 2 into the CBAM attention mechanism module (CBAM-Block), and combine channel attention and spatial attention to achieve dual refinement of image data features, so as to obtain a more comprehensive and effective feature representation;

[0051] Specifically, the CBAM attention mechanism module, such as Figure 3 As shown, it includes: Channel Attention Module (CAM module) and Spatial Attention Module (SAM module). The input of the CAM module is feature map X3 (size 14×14×1024) and the output is feature map X4 (size 14×14×1024).

[0052] The channel attention mechanism module is as follows: Figure 4 As shown, it includes: average pooling layer (AvgPool), max pooling layer (MaxPool), MLP perceptron and sigmoid activation function, specifically represented as follows:

[0053]

[0054] Where F represents the input of the channel attention mechanism module, i.e., the feature map X3, F′ represents the output of the channel attention mechanism module, AvgPool represents the average pooling layer, MaxPool represents the max pooling layer, MLP represents the MLP perceptron, and σ represents the sigmoid activation function. This represents the dot product operation;

[0055] More specifically, the feature map F is passed through parallel AvgPool and MaxPool layers to obtain the output feature map F. avg F max In this model, AvgPool pooling obtains aggregated spatial information, while MaxPooling collects important clues from the features to infer more refined channel attention. Alternatively, AvgPool obtains macroscopic information, while MaxPool obtains local information. Furthermore, using MaxPool can leverage the network to achieve robustness. Feature map F avg F max The two outputs are fed into an MLP perceptron, and the two outputs of the MLP are added together and activated using an activation function to obtain the output feature map M. c (F), M c The size of (F) is 1×1×1024;

[0056] The spatial attention mechanism module is as follows: Figure 5 As shown, it includes: average pooling layer (AvgPool), max pooling layer (MaxPool), convolutional layer, and sigmoid activation function, specifically represented as follows:

[0057]

[0058] Where F″ represents the output of the spatial attention mechanism module, i.e. the feature map X4, and Conv represents the convolutional layer Conv(7×7,1,3);

[0059] Step 4: Input the image data feature map X4 obtained in Step 3 into the Linear Projection module and convert it into patch format;

[0060] The specific structure of the Linear Projection module is: Conv(1×1,1,0) + flattening layer + positional encoding layer + dropout layer. The Linear Projection module is mainly used to convert the feature map X4 into a patch form (i.e., the vector form of word embedding) suitable for processing by the Transformer Layer in the ViT-Transformer module. The feature map X4 first passes through a convolution operation with a kernel size of 1×1, a stride of 1, and a number of kernels of 768, and outputs a feature vector of size 14×14×768. This vector is further passed through a flattening layer, which directly flattens the feature vector after the convolution layer in spatial dimension to retain each data feature. The output of the flattened feature vector is a token vector of size 196×768 (i.e., patch). Then, the positional encoding layer provides position vectors for the token vector, that is, a position vector is added to each token vector. The size of the position vector is the same as that of the token. After the flattened feature vector is added with position vectors, it passes through a dropout layer and outputs a feature vector X5 of size 196×768 in the Hidden Feature layer.

[0061] Step 5: The image data feature vector X5 obtained in Step 4 is downsampled and then input into the ViT-Transformer module to further obtain the feature vector X6;

[0062] Specifically, the feature vector X5 is downsampled and then input into the ViT-Transformer module. After passing through the ViT-Transformer module, the Hidden Feature layer outputs a feature vector X6 of size 196×768.

[0063] The ViT-Transformer module consists of 12 ViT-Transformer Layers, as shown in the figure. Figure 6 As shown, it includes: two Norm normalization layers, a dropout layer, an MSA multi-head attention mechanism layer, and an MLP layer; the input of the ViT-Transformer Layer is represented as the feature vector B. in eigenvector B in After passing through the first Norm normalization layer + dropout layer + MSA multi-head attention mechanism layer, feature vector B1 is obtained. Feature vector B1 and feature vector B in A residual connection is performed to obtain feature vector B2. Feature vector B2 is then passed through a second Norm normalization layer + MLP layer to obtain feature vector B3. Feature vector B3 is then residually connected with feature vector B2 to obtain feature vector B. outeigenvector B out With eigenvector B in Keep the same size;

[0064] The specific structure of the MLP layer is: Linear layer + GELU layer + dropout layer + Linear layer + GELU layer + dropout layer, where the first linear layer can be represented as... W1 represents a linear matrix (768×3072), b1 represents a bias (1×3072), and the second linear layer can be represented as follows: W2 represents a linear matrix (size 3072×768), and b2 represents the bias (size 1×3072);

[0065] Step 6: Reshape the feature vectors obtained by the encoder and decode them. Finally, complete the decoding and output through Segmentations.

[0066] Specifically, the feature vector X6 is first reshaped in dimension, from 196×768 to 14×14×768, and then the feature map Y0 (size 14×14×512) is output after convolution operation by the convolutional layer Conv(1×1,1,0) and sent to the decoding module.

[0067] The decoding module includes three upsampling modules, which conform to the construction of the Unet decoder. Upsampling recovery is used to decode features to output the final segmentation mask. The specific structure of each upsampling module is: upsampling layer + Conv(3×3,1,1) + ReLU activation function.

[0068] The first upsampling module takes feature map Y0 as input and skips to feature map X2. Feature map Y0 is upsampled to obtain feature map Y0' (size 28×28×512). Feature map X2 is passed through the CBAM attention mechanism module to obtain feature map X2'. Feature map X2' and feature map Y0' are concatenated to obtain feature map Y0'. Feature map Y0' is then activated by Conv(3×3,1,1)+ReLU to obtain feature map Y1 (size 28×28×256).

[0069] The input to the second upsampling module is feature map Y1, and it skips the connection to feature map X1. Feature map Y1 passes through the upsampling layer to obtain feature map Y1' (size 56×56×256). Feature map X1 passes through the CBAM attention mechanism module to obtain feature map X1'. Feature map X1' and feature map Y1' are concatenated to obtain feature map Y1'". Feature map Y1' passes through the Conv(3×3,1,1)+ReLU activation function to obtain feature map Y2 (size 56×56×128).

[0070] The input to the third upsampling module is feature map Y2, and it skips the connection to feature map H0. Feature map Y2 passes through the upsampling layer to obtain feature map Y2' (size 112×112×256). Feature map X0 passes through Conv(7×7,2,3) in the stem module to obtain feature map H0. Feature map H0 then passes through the convolutional layer Conv(1×1,1,0) to obtain feature map H1. Feature map H1 passes through the CBAM attention mechanism module to obtain feature map H1'. Feature map H1' and feature map Y2' are concatenated to obtain feature map Y2'. Feature map Y2' passes through Conv(3×3,1,1)+ReLU activation function to obtain feature map Y3 (size 112×112×64).

[0071] The output layer uses a convolutional layer Conv(1×1,1,0). The feature map Y3 is upsampled again to obtain the feature map Y3'. The feature map Y3' passes through the output layer to complete the output data dimension setting and output the segmentation mask Y (size is 224×224×16).

[0072] It should be noted that step 1 above is sample data preprocessing, which increases sample diversity through random reversal, rotation, and shearing operations; all modules in steps 2 to 6 above together constitute a complete neural network model according to the data flow, such as... Figure 1 As shown, the neural network model is pre-trained offline. In the image segmentation task of CT medical images, the CT medical image to be segmented is input into the trained neural network model, and the model outputs a segmentation mask.

[0073] Based on the above embodiments, in order to illustrate the advantages of the present invention, "TransUNet: Transformers MakeStrong Encoders for Medical Image Segmentation" is used as a comparative example. This embodiment is compared with the comparative example, and DSC, HD95 and Loss are used as indicators. The comparison results are shown in the table below:

[0074] DSC HD95 Loss Comparative Example 77.4842％ 31.6935 0.0581 This embodiment 78.5522％ 18.2502 0.0374

[0075] As can be seen from the table, the DSC performance of this embodiment is superior, indicating that it is more accurate in segmentation and can better capture the features of the target region, thereby improving the overall performance. The HD95 index of this embodiment is lower than that of the comparative embodiment, which shows that this embodiment is more outstanding in terms of the accuracy of the target boundary, effectively reducing the maximum distance between the prediction and the actual target and reducing the risk of false detection and false negative detection. In terms of loss, this embodiment shows that the convergence speed during the training process is faster and the performance is more stable, which can effectively reduce the time and resource consumption required for training.

[0076] The above description is merely a specific embodiment of the present invention. Any feature disclosed in this specification may be replaced by other equivalent or similar features unless otherwise specified. All disclosed features, or steps in all methods or processes, may be combined in any way except for mutually exclusive features and / or steps.

Claims

1. A medical image segmentation method based on a dual-attention mechanism using an improved TransUnet network, characterized in that, Includes the following steps: Construct and train an image segmentation model based on an improved TransUnet network; The image segmentation model includes an encoder, a decoder, and an output layer. The encoder includes an SE-ResNet50 module, a CBAM attention mechanism module, a Linear Projection module, and a ViT-Transformer module. The SE-ResNet50 module includes a stem module and three block modules. After the input image X is fed into the SE-ResNet50 module, the stem module and the three block modules sequentially output feature maps H0, X1, X2, and X3. Feature maps H0, X1, and X2 are connected to the encoder in a skip connection. Feature map X3 is reconstructed after passing through the CBAM attention mechanism module, the Linear Projection module, and the ViT-Transformer module to obtain feature map Y0, which is then fed into the decoder. The decoder includes three upsampling modules that sequentially output feature maps Y1 to Y3. Feature map Y3 passes through the output layer to obtain the segmentation mask Y. The medical image to be processed is input into the trained image segmentation model, and the model outputs a segmentation mask to complete the medical image segmentation.

2. The medical image segmentation method based on the improved TransUnet network with dual attention mechanism according to claim 1, characterized in that, In the SE-ResNet50 module, the specific structure of the stem module is: Conv(7×7,2,3) + GroupNorm layer + ReLU activation function + pooling layer. The input of the stem module is the image X and the output is the feature map X0'. At the same time, the feature map H0 is drawn out after the convolutional layer Conv(7×7,2,3). The three Block modules are represented as Block1, Block2, and Block3, respectively. The specific structure of Block1 is unit1×3, the specific structure of Block2 is unit2+unit1×3, and the specific structure of Block3 is unit2+unit1×8. The input of Block1 is feature map X0' and the output is feature map X1, the input of Block2 is feature map X1 and the output is feature map X2, and the input of Block3 is feature map X2 and the output is feature map X3. Both unit2 and unit1 use unit units, which include residual blocks and SE modules. The input of a unit unit is represented by feature map A. in Feature map A in After passing through the residual block, feature map A1 is obtained. Feature map A1 is then processed by the SE module to obtain feature map A2. Feature map A2 and feature map A1 are then scaled to obtain feature map A3. Feature map A3 and feature map A1 are then compared. in By performing residual connections, the output feature map A of the unit cell is finally obtained. out .

3. The medical image segmentation method based on the improved TransUnet network with dual attention mechanism according to claim 2, characterized in that, The specific structure of the SE module is: Global Pooling layer + fully connected layer + ReLU activation function + fully connected layer + Sigmoid activation function.

4. The medical image segmentation method based on the improved TransUnet network with dual attention mechanism according to claim 2, characterized in that, The specific structure of the residual block in unit1 is: Conv(1×1,1,0)+Conv(3×3,1,1)+Conv(1×1,1,0), and the specific structure of the residual block in unit2 is: Conv(1×1,2,0)+Conv(3×3,2,1)+Conv(1×1,2,0).

5. The medical image segmentation method based on the improved TransUnet network with dual attention mechanism according to claim 1, characterized in that, The CBAM attention mechanism module includes two parts: a channel attention mechanism module and a spatial attention mechanism module. The channel attention mechanism module includes: an average pooling layer, a max pooling layer, an MLP perceptron, and a sigmoid activation function, represented as follows: Where F represents the input of the channel attention mechanism module, F′ represents the output of the channel attention mechanism module, AvgPool represents the average pooling layer, MaxPool represents the max pooling layer, MLP represents the MLP perceptron, and σ represents the Sigmoid activation function. The spatial attention mechanism module includes: average pooling layer, max pooling layer, convolutional layer, and sigmoid activation function, represented as: Where F″ represents the output of the spatial attention mechanism module, and Conv represents the convolutional layer Conv(7×7,1,3).

6. The medical image segmentation method based on the dual attention mechanism of the improved TransUnet network according to claim 1, characterized in that, The specific structure of the Linear Projection module is: Conv(1×1,1,0) + flattening layer + position encoding layer + dropout layer.

7. The medical image segmentation method based on the dual attention mechanism of the improved TransUnet network according to claim 1, characterized in that, The ViT-Transformer module consists of 12 ViT-Transformer layers, including: two Norm normalization layers, a dropout layer, a MSA multi-head attention mechanism layer, and an MLP layer. The input of each ViT-Transformer layer is represented as a feature vector B. in eigenvector B in After passing through the first Norm normalization layer + dropout layer + MSA multi-head attention mechanism layer, feature vector B1 is obtained. Feature vector B1 and feature vector B in A residual connection is performed to obtain feature vector B2. Feature vector B2 is then passed through a second Norm normalization layer + MLP layer to obtain feature vector B3. Feature vector B3 is then residually connected with feature vector B2 to obtain feature vector B. out The specific structure of the MLP layer is: Linear layer + GELU layer + dropout layer + Linear layer + GELU layer + dropout layer.

8. The medical image segmentation method based on the improved TransUnet network with dual attention mechanism according to claim 1, characterized in that, The specific process of reshaping is as follows: First, the feature map X6 output by the ViT-Transformer module is reshaped in dimension, restoring it from a two-dimensional feature vector to a three-dimensional feature map. Then, after convolution operation by the convolutional layer Conv(1×1,1,0), the feature map Y0 is output and sent to the decoding module.

9. The medical image segmentation method based on the improved TransUnet network with dual attention mechanism according to claim 1, characterized in that, In the decoder, the specific structure of each upsampling module is: upsampling layer + Conv(3×3,1,1) + ReLU activation function; The first upsampling module takes feature map Y0 as input and skips the connection to feature map X2. Feature map Y0 is upsampled to obtain feature map Y0'. Feature map X2 is passed through the CBAM attention mechanism module to obtain feature map X2'. Feature map X2' and feature map Y0' are concatenated to obtain feature map Y0'. Feature map Y0' is then passed through the Conv(3×3,1,1)+ReLU activation function to obtain feature map Y1. The input to the second upsampling module is feature map Y1, and it skips the connection to feature map X1. Feature map Y1 is upsampled to obtain feature map Y1'. Feature map X1 is passed through the CBAM attention mechanism module to obtain feature map X1'. Feature map X1' and feature map Y1' are concatenated to obtain feature map Y1'". Feature map Y1' is passed through the Conv(3×3,1,1)+ReLU activation function to obtain feature map Y2. The input to the third upsampling module is feature map Y2, and it skips the connection to feature map H0. Feature map Y2 is upsampled to obtain feature map Y2'. Feature map H0 is convolved by Conv(1×1,1,0) to obtain feature map H1. Feature map H1 is passed through the CBAM attention mechanism module to obtain feature map H1'. Feature map H1' and feature map Y2' are concatenated to obtain feature map Y2'". Feature map Y2' is passed through Conv(3×3,1,1)+ReLU activation function to obtain feature map Y3.

10. The medical image segmentation method based on the improved TransUnet network with dual attention mechanism according to claim 1, characterized in that, The output layer uses a convolutional layer Conv(1×1,1,0). The feature map Y3 is upsampled again to obtain the feature map Y3'. The feature map Y3' is then passed through the output layer to obtain the segmentation mask Y.

Images