Medical Image Segmentation Method and System Based on Efficient Dual Attention

Abstract

The present invention relates to a medical image segmentation method and system based on efficient dual attention; belonging to the technical field of medical image segmentation, it includes: 1) obtaining a medical image data set, which is preprocessed and divided into a training set and a test set; 2) constructing a medical image segmentation model based on efficient dual attention; 3) training the medical image segmentation model based on efficient dual attention; 4) inputting the test set into the optimized medical image segmentation model based on efficient dual attention, and outputting the final medical image segmentation result. The present invention effectively fuses the semantic features between the high layer and the low layer, thereby extracting more significant features and retaining the semantic space information.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and particularly to a medical image segmentation method and system based on efficient dual attention, aiming to improve the segmentation accuracy and processing efficiency of medical images. Background Art

[0002] With the development of deep learning, especially convolutional neural networks (CNNs), traditional CNN-based image segmentation methods, such as U-Net, are widely used in medical image segmentation. However, their local receptive fields and inductive biases limit the modeling of long-range dependencies. At the same time, methods based on Transformer perform excellently in capturing global semantics but are weak in processing local details. Although hybrid models that combine Transformer and CNN, such as TransUNet, etc., alleviate these problems to a certain extent, they still face challenges such as high computational costs and large training data requirements. Therefore, although medical image segmentation has made remarkable progress driven by deep learning, especially convolutional neural networks and Transformer architectures, its deficiencies in local details, computational complexity, and dependence on labeled data still remain problems that need to be urgently solved in current research. Especially in dealing with local details and multi-scale information fusion. Summary of the Invention

[0003] To overcome the deficiencies of the prior art, the present invention provides a medical image segmentation method based on efficient dual attention. This method aims to solve problems such as poor local detail processing and insufficient segmentation accuracy in the prior art.

[0004] The present invention also provides a medical image segmentation system based on efficient dual attention.

[0005] The technical solution of the present invention is as follows:

[0006] The medical image segmentation method based on efficient dual attention includes:

[0007] 1) Obtain a medical image dataset, perform data preprocessing, and divide the obtained dataset into a training set and a test set;

[0008] 2) Construct a medical image segmentation model based on efficient dual attention;

[0009] 3) Input the training set into the medical image segmentation model based on efficient dual attention, and output a feature map;

[0010] 4) Train the medical image segmentation model based on efficient dual attention to obtain an optimized medical image segmentation model based on efficient dual attention;

[0011] 5) Input the test set into the optimized medical image segmentation model based on efficient dual attention, and the final medical image segmentation result is obtained as the output.

[0012] Furthermore, step 1) includes the following steps:

[0013] 1-1) Data collection and preprocessing: Use the Synapse dataset; Apply different data augmentation methods to the Synapse dataset, including horizontal flipping, vertical flipping, Gaussian noise, blurring, and random brightness contrast, to increase the number of samples, and obtain a dataset named Data.

[0014] 1-2) Data division: Divide the dataset Data into a training set D1 and a test set D2.

[0015] Furthermore, the medical image segmentation model based on efficient dual attention includes an efficient dual attention module, namely the DualTransformer module, an encoder module, a decoder module, a multi-scale normalization channel attention module, namely the MNCA bottleneck module, and a feature fusion residual module, namely the FFRM module.

[0016] The Dual Transformer module includes spatial attention and channel attention. [[ID=,17]]

[0017] In the Dual Transformer module, spatial attention is first performed, and then channel attention; By combining spatial attention and channel attention, local features and global features are captured simultaneously. At the same time, through residual connection and normalization operations.

[0018] Furthermore, in spatial attention, the dimension of the input feature matrix X is n×d, where n is the sequence length and d is the feature dimension; Generate a query matrix Q (Query), a key matrix K (Key), and a value matrix V (Value) from the input feature matrix X, and the dimension of each matrix is n×di, where di is the dimension; Perform a transpose operation on the key matrix K to obtain K T , and the dimension becomes dk*n; Calculate the dot product of V and K T to obtain the attention weight matrix, and then multiply the attention weight matrix by Q to obtain the spatial attention output; Perform a residual connection on the spatial attention output and the input X, and then process it through normalization (Norm) and a feed-forward network (FFN) to obtain the final spatial attention output.

[0019] In channel attention, the input feature matrix X has a dimension of n×d; Generate a query matrix Q1, a key matrix K1, and a value matrix V1 from the input X, and the dimension of each matrix is n×di, where di is the dimension; Perform a transpose operation on the key matrix K1 to obtain K1 T / β, where β is the scaling factor, and calculate Q1 and K1T The dot product of / β is calculated to obtain the attention weight matrix, and then the attention weight matrix is multiplied by V1 to obtain the channel attention output. The channel attention output is connected with the input X through a residual connection, and then processed through normalization (Norm) and a feed-forward network (FFN) to obtain the final channel attention output.

[0020] Furthermore, the encoder module includes a Patch Partition layer (i.e., a block partitioning layer), a Linear Embedding layer (i.e., a linear embedding layer), a first downsampling module, a second downsampling module, and a third downsampling module.

[0021] The first downsampling module, the second downsampling module, and the third downsampling module each include two consecutive DualTransformer blocks and a Patch Merging layer (i.e., a patch merging layer).

[0022] First, through the Patch Partition layer and the Linear Embedding layer, the Patch Partition layer divides the image into multiple non-overlapping image patches (patches); the Linear Embedding layer maps the feature vector of each image patch to an embedding vector of a fixed dimension through a linear transformation (fully connected layer), divides the input image into non-overlapping blocks of size 4×4, and maps the non-overlapping blocks to a new feature dimension C (W / 4 × H / 4 × C).

[0023] Next, representation learning is performed through two consecutive Dual Transformer blocks.

[0024] Subsequently, through the Patch Merging layer that performs 2-fold downsampling, while reducing the resolution, the feature dimension is increased. This process is repeated three times in the encoder to gradually refine the feature representation of the image.

[0025] According to the preference of the present invention, the execution process in the MNCA bottleneck module is as follows:

[0026] First, through a depthwise separable convolution module (i.e., DASConv) with expansion ratios of 1, 3, 5, and 7 respectively, the receptive field is expanded to extract rich multi-scale feature information.

[0027] DASConv is a depthwise separable convolution, including depth convolution and pointwise convolution; depth convolution performs convolution operations on each input channel separately, and pointwise convolution performs linear combinations between channels on the results of depth convolution.

[0028] In addition, DASConv is fused with the residual channel Fin to extract features of various sizes of objects.

[0029] Secondly, further adopt channel attention based on normalization to enhance the correlation between feature channels; use the BN function to perform batch normalization on DASout, suppress unimportant weights, and then multiply the channel weight Wr by NAMout to enforce the dependency relationship between feature channels;

[0030] Finally, obtain the final output of the MNCA bottleneck module through the Sigmoid activation function.

[0031] Preferably according to the present invention, the decoder module from top to bottom includes a first upsampling module, a second upsampling module, and a third upsampling module, all of which include two consecutive Dual Transformer blocks and a Patch Expanding layer, i.e., a block expansion layer;

[0032] The Patch Expanding layer performs 2-fold upsampling on the extracted depth features and reshapes the adjacent dimensional feature maps into higher-resolution feature maps; correspondingly, the feature dimension is reduced to half of the original dimension, gradually restoring the feature maps to the original image size;

[0033] The encoder module and the decoder module are connected through the FFRM module.

[0034] Preferably according to the present invention, the FFRM module includes a deep feature map branch m and a shallow feature map branch n, which perform the same feature processing operations in parallel. Before fusion, the FFRM module performs an inverse residual operation on the low- and high-level feature maps respectively, and then performs summation to finally obtain the output feature map m + n.

[0035] Furthermore, in step 3), the training set is input into the medical image segmentation model based on efficient dual attention, and a feature map is output; the following steps are included:

[0036] 3-1) The i-th preprocessed image data in the training set D1, abbreviated as is input into the medical image segmentation model based on efficient dual attention, and after downsampling through the Patch Partition layer and the Liear Enbedding layer, the image is reduced to a size of H / 4 * W / 4 * C, where H and W are the height and width of the original image, and C is the number of channels, and a feature map is output ;

[0037] 3-2) is input into the first downsampling module; First, perform representation learning through two DualTransformer blocks, and output a feature map ; Subsequently, through the Patch Merging layer that performs 2x downsampling, while reducing the resolution, the feature dimension is increased, and the image size is reduced from W / 4*H / 4*C to W / 8*H / 8*2C, and the output is the feature map ;

[0038] 3-3) Input into the second downsampling module; First, it goes through two DualTransformer blocks for representation learning, and the output is the feature map ; Subsequently, through the Patch Merging layer that performs 2x downsampling, while reducing the resolution, the feature dimension is increased, and the image size is reduced from W / 8*H / 8*2C to W / 16*H / 16*4C, and the output is the feature map ;

[0039] 3-4) Input into the third downsampling module; First, it goes through two DualTransformer blocks for representation learning, and the output is the feature map ; Subsequently, through the Patch Merging layer that performs 2x downsampling, while reducing the resolution, the feature dimension is increased, and the image size is reduced from W / 16*H / 16*4C to W / 32*H / 32*8C, and the output is the feature map ;

[0040] 3-5) Input into the MNCA bottleneck module, the feature dimension and resolution remain unchanged, and the output is the feature map ;

[0041] 3-6) Input into the third upsampling module; First, it goes through the Patch Expanding layer to perform 2x upsampling on the extracted depth features, and reshapes the adjacent dimension feature maps into higher-resolution feature maps. The image size is expanded from H / 32*W / 32*8C to H / 16*W / 16*4C, and the output is the feature map ; Secondly, it goes through the FFRM module, and adds it to obtained from 3-4) to reduce the spatial information loss caused by downsampling, and the output is the feature map ; Finally, it goes through two Dual Transformer blocks for representation learning. During this process, the feature dimension and resolution remain unchanged, and the output is the feature map ;

[0042] 3-7) Input Input to the second upsampling module; First, through the Patch Expanding layer, the extracted depth features are upsampled by a factor of 2, and the adjacent-dimensional feature maps are reshaped into higher-resolution feature maps. The image size is expanded from H / 16*W / 16*4C to H / 8*W / 8*2C, and the output is the feature map ; Second, Through the FFRM module, add it to the one obtained from 3-3) to reduce the loss of spatial information caused by downsampling, and the output is the feature map ; Finally, Through two Dual Transformer blocks for representation learning. During this process, the feature dimension and resolution remain unchanged, and the output is the feature map ;

[0043] 3-8) Input to the first upsampling module; First, through the Patch Expanding layer, the extracted depth features are upsampled by a factor of 2, and the adjacent-dimensional feature maps are reshaped into higher-resolution feature maps. The image size is expanded from H / 8*W / 8*2C to H / 4*W / 4*C, and the output is the feature map ; Second, Through the FFRM module, add it to the one obtained from 3-2) (italic) to reduce the loss of spatial information caused by downsampling, and the output is the feature map ; Finally, Through two Dual Transformer blocks for representation learning. During this process, the feature dimension and resolution remain unchanged, and the output is the feature map ;

[0044] 3-9) Input to the Patch Expanding layer to upsample the extracted depth features by a factor of 2, and reshape the adjacent-dimensional feature maps into higher-resolution feature maps. The image size is expanded from H / 4*W / 4*C to H / *W / *C, and the output is the feature map ;

[0045] 3-10) Input to the linear projection layer (Linear Projection) for pixel-level segmentation prediction. The feature map is changed to H*W*CLASS, where CLASS is the number of classification categories, and the final output is obtained and named .

[0046] Preferably, according to the present invention, a comprehensive loss function method is adopted in training, and the Dice loss function and the cross-entropy loss function are used in combination; the Dice loss function alleviates the problem of data imbalance in binary classification; the Dice loss function has the formula:

[0047] = ;

[0048] where |M| and |N| represent the areas of the segmentation result and the label, and |M∩N| represents the area of the overlapping part of the segmentation result and the label. The cross-entropy loss function optimizes pixel-level classification. The cross-entropy loss function has the formula:

[0049] ;

[0050] where y and p are the true value and the predicted value when y = 0|1, p∈(0,1) respectively;

[0051] The total loss function of the medical image segmentation model based on efficient dual attention is expressed as:

[0052] ;

[0053] where is the balance coefficient, and the trade-off between precision and recall is optimized by adjusting the parameter .

[0054] Furthermore, step 5) includes the following steps:

[0055] Input the test set D2 into the optimized medical image segmentation model based on efficient dual attention; obtain the final medical image segmentation result map and name it .

[0056] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the medical image segmentation method based on efficient dual attention are implemented.

[0057] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the medical image segmentation method based on efficient dual attention are implemented.

[0058] The medical image segmentation system based on efficient dual attention includes:

[0059] A medical image data set acquisition module, configured to: perform data preprocessing and divide the obtained data set into a training set and a test set;

[0060] Medical image segmentation model construction module, configured to: construct a medical image segmentation model based on efficient dual attention;

[0061] Medical image segmentation model training module, configured to: input a training set into the medical image segmentation model based on efficient dual attention, and output a feature map; train the medical image segmentation model based on efficient dual attention to obtain an optimized medical image segmentation model based on efficient dual attention;

[0062] Medical image segmentation module, configured to: input a test set into the optimized medical image segmentation model based on efficient dual attention, and output a final medical image segmentation result.

[0063] The beneficial effects of the present invention are:

[0064] 1) The present invention designs the FFRM module as a skip connection in the network, which can more accurately locate the target in complex backgrounds, effectively fuse semantic features between high-level and low-level layers, thereby extracting more significant features and retaining semantic spatial information.

[0065] 2) The present invention designs the MNCA bottleneck module, which combines attribute convolution, normalized channel attention mechanism and depthwise separable convolution (DSConv) to enhance the interdependence between channels. It makes full use of the complementarity of global region and local edge information to maintain global context information.

[0066] 3) The present invention proposes a new symmetric dual-branch encoder-decoder network architecture, which is constructed with Dual Transformer blocks as basic units. By combining spatial attention and channel attention, it can capture local features and global features simultaneously, thereby improving the feature extraction ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a schematic structural diagram of a medical image segmentation model based on efficient dual attention;

[0068] Figure 2 It is a schematic structural diagram of the Dual Transformer module;

[0069] Figure 3 It is a schematic structural diagram of the MNCA bottleneck module;

[0070] Figure 4 It is a schematic structural diagram of the FFRM module;

[0071] Figure 5 It is a schematic flow diagram of the medical image segmentation method based on efficient dual attention of the present invention;

[0072] Figure 6Schematic diagram of the results after processing using the method of the present invention and the existing model. Specific implementation manners

[0073] The present invention will be further defined below in conjunction with the accompanying drawings of the specification and embodiments, but is not limited thereto.

[0074] Embodiment 1

[0075] A medical image segmentation method based on efficient dual attention, that is, a medical image segmentation method based on efficient dual attention, as Figure 5 shown, includes:

[0076] 1) Obtain a medical image dataset, perform data preprocessing, and divide the obtained dataset into a training set and a test set;

[0077] 2) Construct a medical image segmentation model based on efficient dual attention;

[0078] 3) Input the training set into the medical image segmentation model based on efficient dual attention, and output a feature map;

[0079] 4) Train the medical image segmentation model based on efficient dual attention to obtain an optimized medical image segmentation model based on efficient dual attention;

[0080] 5) Input the test set into the optimized medical image segmentation model based on efficient dual attention, and output the final medical image segmentation result.

[0081] Embodiment 2

[0082] [[ID=3'4]]The medical image segmentation method based on efficient dual attention according to Embodiment 1 is different in that:

[0083] Step 1) includes the following steps:

[0084] 1-1) Data collection and preprocessing: Use the publicly available Synapse dataset; apply different data augmentation methods to the Synapse dataset, including horizontal flipping, vertical flipping, Gaussian noise, blurring, and random brightness contrast, to increase the number of samples, and obtain a dataset named Data; the Synapse dataset, that is, the Synapse multi-organ segmentation dataset, contains 3,779 axial abdominal clinical CT images of 30 patients. This dataset contains 8 abdominal organs (aorta, gallbladder, left kidney, right kidney, liver, pancreas, spleen, and stomach). The present invention adjusts the resolution of all images to 512×512.

[0085] 1-2) Data division: Divide the dataset Data into a training set D1 and a test set D2 according to a ratio of 9:1.

[0086] The medical image segmentation model based on efficient dual attention includes an efficient dual attention module, namely the Dual Transformer module, an encoder module, a decoder module, a multi-scale normalized channel attention module, namely the MNCA bottleneck module, and a feature fusion residual module, namely the FFRM module; as Figure 1 shown;

[0087] as Figure 2 shown, the Dual Transformer module includes spatial attention and channel attention;

[0088] In the Dual Transformer module, spatial attention is first performed, and then channel attention is performed; by combining spatial attention and channel attention, this dual attention block can capture local features and global features simultaneously, thereby improving the feature extraction ability of the model. At the same time, through residual connection and normalization operations. The model can be trained more stably.

[0089] Figure 2 In (a), it is spatial attention. In spatial attention, the dimension of the input feature matrix X is n×d, where n is the sequence length and d is the feature dimension; a query matrix Q (Query), a key matrix K (Key), and a value matrix V (Value) are generated from the input feature matrix X, and the dimension of each matrix is n×di, where di is the dimension; the transpose operation is performed on the key matrix K to obtain K T , and the dimension becomes dk*n; calculate the dot product of V and K T , to obtain the attention weight matrix, and then multiply the attention weight matrix by Q to obtain the spatial attention output; connect the spatial attention output with the input X through residual connection, and then process it through normalization (Norm) and a feed-forward network (FFN) to obtain the final spatial attention output; the feed-forward network, FFN, is a key component in the Transformer architecture. It receives the feature vector output by the self-attention mechanism, performs non-linear transformation and feature extraction, and enhances the model's expression and learning ability. In the Dual Transformer module, the input and output dimensions of the FFN are the same, and internally, it realizes feature interaction and mapping through two-layer linear transformation and activation functions, helping the model better understand the data semantics and context relationship.

[0090] Figure 2 In (b), it is channel attention. In channel attention, the input feature matrix X has a dimension of n×d; a query matrix Q1, a key matrix K1, and a value matrix V1 are generated from the input X, and the dimension of each matrix is n×di, where di is the dimension; the transpose operation is performed on the key matrix K1 to obtain K1 T / β, where β is a scaling factor used to stabilize training; calculate Q1 and K1 TThe dot product of / β is used to obtain the attention weight matrix, and then the attention weight matrix is multiplied by V1 to obtain the channel attention output. The channel attention output is connected with the input X through a residual connection, and then processed through normalization (Norm) and a feed-forward network (FFN) to obtain the final channel attention output.

[0091] The decoder module from top to bottom includes a first upsampling module, a second upsampling module, and a third upsampling module, all of which include two consecutive Dual Transformer blocks and a Patch Expanding layer, i.e., a block expansion layer.

[0092] The first downsampling module, the second downsampling module, and the third downsampling module all include two consecutive Dual Transformer blocks and a Patch Merging layer, i.e., a patch merging layer.

[0093] First, through the Patch Partition layer and the Linear Embedding layer, the Patch Partition layer divides the image into multiple non-overlapping image patches (patches); the Linear Embedding layer maps the feature vector of each image patch to an embedding vector of a fixed dimension through a linear transformation (fully connected layer), divides the input image into non-overlapping blocks of size 4×4, and maps the non-overlapping blocks to a new feature dimension C (W / 4 × H / 4 × C).

[0094] Next, representation learning is carried out through two consecutive Dual Transformer blocks; at this time, two consecutive Dual Transformer blocks are used for representation learning in the encoder. After being processed by the first Dual Transformer block, the shape of the feature matrix is still N×d, but the feature representation is richer and more abstract. After being processed by the second Dual Transformer block, the shape of the feature matrix is still N×d, but the feature representation is deeper and more abstract, providing a higher-quality feature input for the subsequent Patch Merging or Patch Expanding layer.

[0095] The internal processing process of each Dual Transformer block:

[0096] The input is the embedded feature matrix from the previous layer, with a shape of N×d, where N is the number of image patches and d is the embedding dimension. The processing procedure is as follows: perform self-attention calculation on the input features to capture the global dependencies between features. Each attention head calculates independently, and then the results are concatenated and linearly transformed to obtain a new feature representation; perform a residual connection between the output of the self-attention mechanism and the input features, and perform normalization processing through a normalization layer to enhance the stability and convergence speed of the model; perform a non-linear transformation on the normalized features to further extract and enhance features. The FFN usually consists of two linear layers and an activation function to increase the expressive power of the model; perform a residual connection between the output of the FFN and the input features, and perform normalization processing through the normalization layer again to obtain the final feature representation. The feature matrix after being processed by the Dual Transformer block still has a shape of N×d, but the feature representation is richer and more abstract. During this process, the feature dimension and resolution remain unchanged.

[0097] Subsequently, through the Patch Merging layer that performs 2-fold downsampling, while reducing the resolution, the feature dimension is increased. This process is repeated three times in the encoder to gradually refine the feature representation of the image.

[0098] The processing procedure of the Patch Merging layer is as follows:

[0099] The input is the feature matrix output by the Dual Transformer block, with a shape of N×d. The processing procedure: group adjacent multiple image patches; fuse the feature of each image patch within the group, usually through linear transformation and dimensionality reduction operations, to merge the features of multiple patches into one feature vector; through the above fusion operation, the resolution of the feature map is reduced. The feature matrix after being processed by the Patch Merging layer has a shape of N'×2d, where N' is the number of merged image patches, and the feature dimension is increased to 2d.

[0100] In the MNCA bottleneck module, the dimension and resolution of the feature map remain consistent. The MNCA bottleneck module is divided into two main steps. It includes:

[0101] First, as shown in (a) of Figure 3 , it is a multi-residual multi-scale strategy. Through a depthwise separable convolution module (DASConv) with expansion ratios of 1, 3, 5, and 7 respectively, the receptive field is expanded to extract rich multi-scale feature information;

[0102] DASConv is a depthwise separable convolution, including depth convolution and pointwise convolution; depth convolution performs convolution operations on each input channel separately, and pointwise convolution performs linear combination between channels on the result of depth convolution;

[0103] Figure 3 Figure (a) shows four DASConvs with different dilation rates: 1, 3, 5, and 7. The dilation rate determines the sampling interval of the convolutional kernel on the feature map. The larger the dilation rate, the wider the actual coverage of the convolutional kernel and the larger the receptive field.

[0104] Dilation rate 1: The convolutional kernel samples the feature map at an interval of 1, covering a small local area and suitable for capturing fine-grained features.

[0105] Dilation rate 3: The convolutional kernel samples the feature map at an interval of 3, covering a larger area and suitable for capturing medium-scale features.

[0106] Dilation rate 5: The convolutional kernel samples the feature map at an interval of 5, covering a larger area and suitable for capturing larger-scale features.

[0107] Dilation rate 7: The convolutional kernel samples the feature map at an interval of 7, covering the largest area and suitable for capturing global features.

[0108] The feature maps output by the four DASConv modules with different dilation rates are concatenated to obtain a feature map that fuses multi-scale features.

[0109] In addition, DASConv is fused with the residual channel Fin to extract features of various sizes of objects; the fused feature map and the input feature map Fin are fused through a residual connection to enhance the feature expression ability and the network convergence performance.

[0110] Secondly, as shown in Figure 3 Figure (b), channel attention based on normalization is further adopted to enhance the correlation between feature channels; the BN function is used to perform batch normalization on DASout to suppress unimportant weights, so that NAMout contains more important feature weight information. Then, the channel weight Wr is multiplied by NAMout to enforce the dependence relationship between feature channels;

[0111] Finally, the final output of the MNCA bottleneck module is obtained through the Sigmoid activation function.

[0112] The decoder module from top to bottom includes a first upsampling module, a second upsampling module, and a third upsampling module, all of which include two consecutive Dual Transformer blocks and a Patch Expanding layer, i.e., a block expansion layer;

[0113] Inspired by U-Net and Swin-Unet, a decoder based on a symmetric encoder was designed. The decoder structure is similar to the encoder and is also constructed based on two consecutive Dual Transformer blocks. The Patch Expanding layer upsamples the extracted depth features by a factor of 2 and reshapes the adjacent dimensional feature maps into higher-resolution feature maps; correspondingly, the feature dimension is reduced to half of the original dimension, gradually restoring the feature maps to the original image size;

[0114] The encoder module and the decoder module are connected by the FFRM module. The skip connection fuses the features of the encoder with the depth features restored from upsampling, thus reducing the loss of spatial data caused by downsampling.

[0115] As Figure 4 shown, the FFRM module includes a deep feature map branch m and a shallow feature map branch n, which perform the same feature processing operations in parallel. Before fusion, the FFRM module performs an inverse residual operation on the low- and high-level feature maps respectively, and then sums them to finally obtain the output feature map m + n. It retains more semantic information and restores more spatial details at the same time.

[0116] The FFRM module aims to fuse the deep feature map branch m and the shallow feature map branch n to achieve feature fusion and refinement. The following are the specific structures of these two branches and the fusion implementation process:

[0117] Deep feature map branch m:

[0118] 1×1 convolution: Adjust the number of channels and perform feature transformation on the input deep feature map m. Use a 1×1 convolution kernel, followed by the ReLU6 activation function, to reduce the computational amount and maintain the richness of features.

[0119] 3×3 depthwise separable convolution (Dwise 3×3): Further extract local features and enhance the feature expression ability. Use a 3×3 depthwise separable convolution kernel, followed by the ReLU6 activation function, to reduce the computational amount while maintaining the local perception ability of features.

[0120] 1×1 convolution: Adjust the number of channels and perform a linear transformation on the features again. Use a 1×1 convolution kernel and do not use an activation function to maintain the linear combination ability of features.

[0121] Shallow feature map branch (n):

[0122] 1×1 convolution: Adjust the number of channels and perform feature transformation on the input shallow feature map n. Use a 1×1 convolution kernel, followed by the ReLU6 activation function, to reduce the computational amount and maintain the richness of features.

[0123] 3×3 Depthwise Separable Convolution (Dwise 3×3): Further extract local features and enhance the feature expression ability. Use a 3×3 depthwise separable convolution kernel, followed by a ReLU6 activation function, which reduces the computational amount while maintaining the local perception ability of features.

[0124] 1×1 Convolution: Adjust the number of channels and perform a linear transformation on the features again. Use a 1×1 convolution kernel without an activation function to maintain the linear combination ability of features.

[0125] The same feature processing operations are performed in parallel on two branches. The processed high-level features and low-level features are directly added together. High-level features provide semantic information (such as the target contour), and low-level features supplement details (such as texture), and the two complement each other. The inverted residual structure enhances the diversity and non-linear expression ability of features through a channel expansion-compression strategy.

[0126] In step 3), the training set is input into the medical image segmentation model based on efficient dual attention, and a feature map is output; it includes the following steps:

[0127] 3-1) Abbreviate the i-th preprocessed image data in the training set D1 Input it into the medical image segmentation model based on efficient dual attention. After downsampling through the Patch Partition layer and the Liear Enbedding layer, the image is reduced to a size of H / 4*W / 4*C, where H and W are the height and width of the original image, and C is the number of channels, and a feature map is output ;

[0128] 3-2) Input into the first downsampling module; First, perform representation learning through two DualTransformer blocks. During this process, the feature dimension and resolution remain unchanged, and a feature map is output ; Subsequently, through the Patch Merging layer that performs 2-fold downsampling, while reducing the resolution, the feature dimension is increased, and the image size is reduced from W / 4*H / 4*C to W / 8*H / 8*2C, and a feature map is output ;

[0129] 3-3) Input into the second downsampling module; First, perform representation learning through two DualTransformer blocks. During this process, the feature dimension and resolution remain unchanged, and a feature map is output ; Subsequently, through the Patch Merging layer that performs 2x downsampling, while reducing the resolution, the feature dimension is increased, and the image size is reduced from W / 8*H / 8*2C to W / 16*H / 16*4C, and the output is the feature map ;

[0130] 3-4) Input into the third downsampling module; First, it passes through two DualTransformer blocks for representation learning. During this process, the feature dimension and resolution remain unchanged, and the output is the feature map ; Subsequently, through the Patch Merging layer that performs 2x downsampling, while reducing the resolution, the feature dimension is increased, and the image size is reduced from W / 16*H / 16*4C to W / 32*H / 32*8C, and the output is the feature map ;

[0131] 3-5) Input into the MNCA bottleneck module. The feature dimension and resolution remain unchanged, and the output is the feature map ;

[0132] 3-6) Input into the third upsampling module; First, it passes through the Patch Expanding layer to perform 2x upsampling on the extracted depth features and reshapes the adjacent dimension feature maps into higher-resolution feature maps. The image size is expanded from H / 32*W / 32*8C to H / 16*W / 16*4C, and the output is the feature map ; Second, it passes through the FFRM module and adds it to the obtained from 3-4) to reduce the spatial information loss caused by downsampling, and the output is the feature map [[ID=,35]]; Finally, it passes through two Dual Transformer blocks for representation learning. During this process, the feature dimension and resolution remain unchanged, and the output is the feature map ;

[0133] 3-7) Input into the second upsampling module; First, it passes through the Patch Expanding layer to perform 2x upsampling on the extracted depth features and reshapes the adjacent dimension feature maps into higher-resolution feature maps. The image size is expanded from H / 16*W / 16*4C to H / 8*W / 8*2C, and the output is the feature map ; Second, it passes through the FFRM module and adds it to the obtained from 3-3)Add to reduce the loss of spatial information caused by downsampling, and output the feature map ; Finally, Perform representation learning through two Dual Transformer blocks. During this process, the feature dimension and resolution remain unchanged, and the output is the feature map ;

[0134] 3-8) Input into the first upsampling module; First, perform 2x upsampling on the extracted depth features through the Patch Expanding layer, and reshape the adjacent dimension feature maps into higher-resolution feature maps. The image size expands from H / 8*W / 8*2C to H / 4*W / 4*C, and the output is the feature map ; Second, Pass through the FFRM module, and add it to the obtained from 3-2) to reduce the loss of spatial information caused by downsampling, and the output is the feature map ; Finally, Perform representation learning through two Dual Transformer blocks. During this process, the feature dimension and resolution remain unchanged, and the output is the feature map ;

[0135] 3-9) Input into the Patch Expanding layer to perform 2x upsampling on the extracted depth features, and reshape the adjacent dimension feature maps into higher-resolution feature maps. The image size expands from H / 4*W / 4*C to H / *W / *C, and the output is the feature map ;

[0136] 3-10) Input into the linear projection layer (Linear Projection) for pixel-level segmentation prediction. Change the feature map to H*W*CLASS, where CLASS is the number of classification categories, and obtain the final output and name it .

[0137] Due to the limited scale of the medical image dataset, and the area to be segmented in the image only accounts for a small part of the entire image. These problems may lead to overfitting of the model during training. In training, an integrated loss function method is adopted, which combines the Dice loss function and the cross-entropy loss function; the Dice loss function alleviates the problem of data imbalance in binary classification; the Dice loss function The formula is:

[0138] = ;

[0139] Among them, |M| and |N| represent the areas of the segmentation result and the label, and |M∩N| represents the area of the overlapping part between the segmentation result and the label. The closer the intersection is to 1, the more accurate the prediction result. The cross-entropy loss function optimizes pixel-level classification. The cross-entropy loss function has the following formula:

[0140] ;

[0141] where y and p are the true value and the predicted value when y = 0|1, p∈(0,1);

[0142] To make full use of the complementary advantages of the Dice loss function and the cross-entropy loss, these two loss functions are linearly combined, and the coefficient is balanced with the value of the loss function. The total loss function of the medical image segmentation model based on efficient dual attention is expressed as:

[0143] ;

[0144] where is the balance coefficient, and the trade-off between precision and recall is optimized by adjusting the parameter .

[0145] During the training process of the medical image segmentation model based on efficient dual attention, the programming language of the medical image segmentation model based on efficient dual attention is Python 3.9, the deep learning framework is PyTorch, and it is trained on four NVIDIA A100 GPUs with 40GB of memory each. The training configuration includes a batch size of 24, an SGD optimizer momentum of 0.9, a base learning rate of 0.1, a weight decay of 0.0001, and it is trained for 400 epochs using the cross-entropy loss and the Dice loss ( = 0.5· + 0.5· ). To evaluate the performance of the proposed medical image segmentation model based on efficient dual attention, the average Dice similarity coefficient (DSC) and the average Hausdorff distance (HD) are used as evaluation metrics. The DSC value ranges from 0 to 1, and the larger the value, the better the performance. The smaller the HD value, the better the performance. Finally, the optimized medical image segmentation model based on efficient dual attention is obtained.

[0146] Step 5) includes the following steps:

[0147] Input the test set D2 into the optimized medical image segmentation model based on efficient dual attention; obtain the final medical image segmentation result map and name it .

[0148] Figure 6 It is a visualization schematic diagram of the results processed by using the method of the present invention and the existing model; by comparison, it can be seen that the segmentation performance of the model of the present invention has been improved. On the Synapse dataset, especially for the gallbladder, kidney, liver and spleen, the overall segmentation map is smoother and more natural.

[0149] Example 3

[0150] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the medical image segmentation method based on efficient dual attention described in Example 1 or 2.

[0151] Example 4

[0152] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, it implements the steps of the medical image segmentation method based on efficient dual attention described in Example 1 or 2.

[0153] Example 5

[0154] A medical image segmentation system based on efficient dual attention includes:

[0155] A medical image dataset acquisition module, configured to: perform data preprocessing and divide the obtained dataset into a training set and a test set;

[0156] A medical image segmentation model construction module, configured to: construct a medical image segmentation model based on efficient dual attention;

[0157] A medical image segmentation model training module, configured to: input the training set into the medical image segmentation model based on efficient dual attention, and output a feature map; train the medical image segmentation model based on efficient dual attention to obtain an optimized medical image segmentation model based on efficient dual attention;

[0158] A medical image segmentation module, configured to: input the test set into the optimized medical image segmentation model based on efficient dual attention, and output the final medical image segmentation result.

[0159] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A medical image segmentation method based on efficient double attention, characterized in that, Including: 1) Obtain a medical image dataset, perform data preprocessing, and divide the obtained dataset into a training set and a test set; 2) Construct a medical image segmentation model based on efficient dual attention; 3) Input the training set into the medical image segmentation model based on efficient dual attention, and output a feature map; 4) Train the medical image segmentation model based on efficient dual attention to obtain an optimized medical image segmentation model based on efficient dual attention; 5) Input the test set into the optimized medical image segmentation model based on efficient dual attention, and output the final medical image segmentation result; The medical image segmentation model based on efficient dual attention includes an efficient dual attention module, namely the Dual Transformer module, an encoder module, a decoder module, a multi-scale normalized channel attention module, namely the MNCA bottleneck module, and a feature fusion residual module, namely the FFRM module; The Dual Transformer module includes spatial attention and channel attention; In the Dual Transformer module, spatial attention is first performed, and then channel attention is performed; by combining spatial attention and channel attention, local features and global features are captured simultaneously. At the same time, through residual connection and normalization operations; The execution process in the MNCA bottleneck module is as follows: First, expand the receptive field through a depthwise separable convolution module, namely DASConv, with expansion ratios of 1, 3, 5, and 7 respectively, and extract rich multi-scale feature information; DASConv is a depthwise separable convolution, including depth convolution and pointwise convolution; depth convolution performs convolution operations on each input channel separately, and pointwise convolution performs linear combinations between channels on the result of depth convolution; In addition, fuse DASConv with the residual channel Fin to extract features of various sizes of objects; Secondly, further adopt channel attention based on normalization to enhance the correlation between feature channels; use the BN function to perform batch normalization on DASout to suppress unimportant weights, and then multiply the channel weight Wr and NAMout to enforce the dependence relationship between feature channels; Finally, obtain the final output of the MNCA bottleneck module through the Sigmoid activation function.

2. The medical image segmentation method based on efficient dual attention according to claim 1, characterized in that, Step 1) includes the following steps: 1-1) Data collection and preprocessing: Use the Synapse dataset; apply different data augmentation methods to the Synapse dataset, including horizontal flipping, vertical flipping, Gaussian noise, blurring, and random brightness contrast, to increase the number of samples, and obtain a dataset named Data; 1-2) Data division: Divide the dataset Data into a training set D1 and a test set D2.

3. The medical image segmentation method based on efficient dual attention according to claim 1, wherein In spatial attention, the dimension of the input feature matrix X is n×d, where n is the sequence length and d is the feature dimension; generate a query matrix Q, a key matrix K, and a value matrix V from the input feature matrix X, and the dimension of each matrix is n×di, where di is the dimension; Perform a transpose operation on the key matrix K to obtain K T , and the dimension becomes dk*n; calculate the dot product of V and K T to obtain the attention weight matrix, then multiply the attention weight matrix by Q to obtain the spatial attention output; perform a residual connection between the spatial attention output and the input X, and then process it through normalization and a feed-forward network to obtain the final spatial attention output; In channel attention, the input feature matrix X has a dimension of n×d. Query matrix Q1, key matrix K1, and value matrix V1 are generated from the input X, and each matrix has a dimension of n×di, where di is the dimension. Perform a transpose operation on the key matrix K1 to obtain K1 T / β, where β is a scaling factor, and calculate Q1 and K1 T / β to obtain the attention weight matrix. Then multiply the attention weight matrix by V1 to obtain the channel attention output; perform a residual connection between the channel attention output and the input X, and then process it through normalization and a feed-forward network to obtain the final channel attention output; The encoder module includes a Patch Partition layer (i.e., a block partitioning layer), a Linear Embedding layer (i.e., a linear embedding layer), a first downsampling module, a second downsampling module, and a third downsampling module. The first downsampling module, the second downsampling module, and the third downsampling module each include two consecutive DualTransformer blocks and a Patch Merging layer (i.e., a patch merging layer). First, through the Patch Partition layer and the Linear Embedding layer, the Patch Partition layer divides the image into multiple non-overlapping image patches. The Linear Embedding layer maps the feature vector of each image patch to an embedding vector of a fixed dimension through a linear transformation, divides the input image into non-overlapping patches of size 4×4, and maps the non-overlapping patches to a new feature dimension C. Next, perform representation learning through two consecutive Dual Transformer blocks. Subsequently, through the Patch Merging layer that performs 2-fold downsampling, while reducing the resolution, the feature dimension is increased. This process is repeated three times in the encoder to gradually refine the feature representation of the image.

4. The medical image segmentation method based on efficient dual attention according to claim 1, characterized in that The decoder module includes a first upsampling module, a second upsampling module, and a third upsampling module from top to bottom, and each includes two consecutive Dual Transformer blocks and a Patch Expanding layer (i.e., a block expanding layer). The Patch Expanding layer performs 2-fold upsampling on the extracted depth features and reshapes the adjacent dimension features into a higher-resolution feature map. Correspondingly, the feature dimension is reduced to half of the original dimension, gradually restoring the feature map to the original image size. The encoder module and the decoder module are connected through the FFRM module. The FFRM module includes a deep feature map branch m and a shallow feature map branch n, which perform the same feature processing operations in parallel. Before fusion, the FFRM module performs an inverse residual operation on the low- and high-level feature maps respectively, and then performs summation to finally obtain the output feature map m + n.

5. The medical image segmentation method based on efficient dual attention according to claim 1, characterized in that In step 3), the training set is input into the medical image segmentation model based on efficient dual attention, and a feature map is output. The steps are as follows: 3-1) Abbreviate the preprocessed image data of the i-th image in the training set D1 as D i Input it into the medical image segmentation model based on efficient dual attention. After downsampling through the Patch Partition layer and the Linear Embedding layer, the image is reduced to a size of H / 4 * W / 4 * C, where H and W are the height and width of the original image, and C is the number of channels, and the output is the feature map D down ; 3-2) Input D down into the first downsampling module; D down First, perform representation learning through two Dual Transformer blocks and output a feature map Subsequently, through the Patch Merging layer that performs 2x downsampling, while reducing the resolution, increase the feature dimension. The image size is reduced from W / 4*H / 4*C to W / 8*H / 8*2C, and a feature map is output 3-3) Input into the second downsampling module; First, perform representation learning through two Dual Transformer blocks, and output the feature map Subsequently, through the Patch Merging layer that performs 2x downsampling, while reducing the resolution, increase the feature dimension. The image size is reduced from W / 8*H / 8*2C to W / 16*H / 16*4C, and the output feature map is obtained 3 - 4) Input into the third downsampling module; First, perform representation learning through two Dual Transformer blocks, and output the feature map Subsequently, through the Patch Merging layer that performs 2x downsampling, while reducing the resolution, increase the feature dimension. The image size is reduced from W / 16 * H / 16 * 4C to W / 32 * H / 32 * 8C, and the output feature map is obtained (3 - 5) Input into the MNCA bottleneck module, keeping the feature dimension and resolution unchanged, and obtaining the feature map D as the output down4 ; 3 - 6) Input D down4 into the third upsampling module; D down4 First, pass through the Patch Expanding layer to perform 2x upsampling on the extracted depth features, and reshape the adjacent - dimension feature maps into higher - resolution feature maps. The image size expands from H / 32 * W / 32 * 8C to H / 16 * W / 16 * 4C, and the output is a feature map Secondly, pass through the FFRM module, and add it to the one obtained from 3 - 4) to reduce the spatial information loss caused by downsampling, and the output is a feature map Finally, perform representation learning through two Dual Transformer blocks. During this process, the feature dimension and resolution remain unchanged, and the output is a feature map 3 - 7) Input into the second upsampling module; First, pass through the Patch Expanding layer to perform 2x upsampling on the extracted depth features, and reshape the adjacent - dimension feature maps into higher - resolution feature maps. The image size expands from H / 16 * W / 16 * 4C to H / 8 * W / 8 * 2C, and the output is a feature map Secondly, pass through the FFRM module, and add it to the one obtained from 3 - 3) to reduce the spatial information loss caused by downsampling, and the output is a feature map Finally, perform representation learning through two Dual Transformer blocks. During this process, the feature dimension and resolution remain unchanged, and the output is a feature map 3 - 8) Input into the first upsampling module; First, perform 2x upsampling on the extracted depth features through the Patch Expanding layer, and reshape the adjacent - dimension feature maps into higher - resolution feature maps. The image size expands from H / 8*W / 8*2C to H / 4*W / 4*C, and the output is a feature map Secondly, pass through the FFRM module, and add it to the obtained from 3 - 2) to reduce the spatial information loss caused by downsampling, and the output is a feature map Finally, perform representation learning through two Dual Transformer blocks. During this process, the feature dimension and resolution remain unchanged, and the output is a feature map 3-9) Input into the Patch Expanding layer to perform 2x upsampling on the extracted depth features, and reshape the adjacent dimensional feature maps into higher-resolution feature maps. The image size is expanded from H / 4*W / 4*C to H / *W / *C, and the output is the feature map D up ; 3 - 10) Input D up into the linear projection layer for pixel - level segmentation prediction; transform the feature map into H*W*CLASS, where CLASS is the number of classification categories, obtain the final output and name it 6. The medical image segmentation method based on efficient dual attention according to any one of claims 1-4, characterized in that, In the training, a comprehensive loss function method is adopted, which combines the Dice loss function and the cross-entropy loss function; the Dice loss function alleviates the problem of data imbalance in binary classification; the formula of the Dice loss function L dice is as follows: Among them, |M| and |N| represent the areas of the segmentation result and the label, |M∩N| represents the area of the overlapping part of the segmentation result and the label, and the cross-entropy loss function optimizes pixel-level classification. The formula for the cross-entropy loss function L ce is as follows: Among them, y and p are the true value and the predicted value when y = 0|1, p∈(0,1), respectively. The total loss function Loss of the medical image segmentation model based on efficient dual attention is expressed as: Loss=(1-μ)L dice +μL ce ; Among them, μ is the balance coefficient, and the trade-off between precision and recall is optimized by adjusting the parameter μ.

7. The medical image segmentation method based on efficient dual attention according to any one of claims 1-4, characterized in that, Step 5) includes the following steps: Input the test set D2 into the optimized medical image segmentation model based on efficient dual attention; obtain the final medical image segmentation result map and name it D finaloutput .

8. A medical image segmentation system based on efficient dual attention, characterized in that, Including: A medical image dataset acquisition module, which is configured to perform data preprocessing and divide the obtained dataset into a training set and a test set. Medical Image Segmentation Model Construction Module, configured to: construct a medical image segmentation model based on efficient dual attention; Medical Image Segmentation Model Training Module, configured to: input the training set into the medical image segmentation model based on efficient dual attention, and output a feature map; Train the medical image segmentation model based on efficient dual attention to obtain an optimized medical image segmentation model based on efficient dual attention; Medical Image Segmentation Module, configured to: input the test set into the optimized medical image segmentation model based on efficient dual attention, and output the final medical image segmentation result; The medical image segmentation model based on efficient dual attention includes an efficient dual attention module, namely the Dual Transformer module, an encoder module, a decoder module, a multi-scale normalized channel attention module, namely the MNCA bottleneck module, and a feature fusion residual module, namely the FFRM module; The Dual Transformer module includes spatial attention and channel attention; In the Dual Transformer module, spatial attention is first performed, and then channel attention is performed; by combining spatial attention and channel attention, local features and global features are captured simultaneously. At the same time, through residual connection and normalization operations; The execution process in the MNCA bottleneck module is as follows: First, expand the receptive field through a depthwise separable convolution module, namely DASConv, with expansion ratios of 1, 3, 5, and 7 respectively, and extract rich multi-scale feature information; DASConv is a depthwise separable convolution, including depthwise convolution and pointwise convolution; depthwise convolution performs convolution operations on each input channel separately, and pointwise convolution performs a linear combination between channels on the result of depthwise convolution; In addition, fuse DASConv with the residual channel Fin to extract features of various sizes of objects; Secondly, further adopt normalization-based channel attention to enhance the correlation between feature channels; use the BN function to perform batch normalization on DASout to suppress unimportant weights, and then multiply the channel weight Wr and NAMout to enforce the dependency relationship between feature channels; Finally, obtain the final output of the MNCA bottleneck module through the Sigmoid activation function.

Images