A medical image segmentation method based on convolution and deformable self-attention mechanism

Abstract

This invention relates to the field of medical image segmentation and provides a medical image segmentation method based on convolution and deformable self-attention mechanisms. The method includes: S1: preprocessing and data augmentation of CT images; S2: constructing a U-Net architecture model, Med-CaDA, based on convolution and deformable self-attention mechanisms; S3: applying the Dice loss function to the U-Net architecture model constructed in step S2; S4: training the U-Net architecture model constructed in step S2 using the Adam optimization algorithm; and S5: measuring segmentation accuracy using two metrics: Dice score and 95% Hausdorff distance. This invention constructs a U-Net architecture segmentation model and proposes a CaDA block based on convolution and deformable self-attention mechanisms. This retains the advantages of convolution in extracting local information while utilizing the ability of deformable self-attention mechanisms to capture global dependencies, achieving good medical image segmentation accuracy with relatively low computational cost.

Description

Technical Field

[0001] This invention belongs to the field of medical image segmentation technology, specifically relating to a medical image segmentation method based on convolution and deformable self-attention mechanism. Background Technology

[0002] With the continuous advancement of medical imaging technology and computer vision technology, medical image analysis has become an indispensable tool and technique in medical research and clinical disease diagnosis. As an important component of medical image analysis, medical image segmentation plays a crucial role in clinical diagnosis and treatment.

[0003] In recent years, the fully convolutional neural network architecture U-Net has dominated medical image segmentation. U-Net has achieved great success in various medical image segmentation tasks by constructing an asymmetric encoder-decoder architecture with skip connections. The encoder consists of a series of convolutional layers and downsampling layers to extract deep features; the decoder upsamples the deep features to the same size as the input and fuses features of different scales obtained from the encoder process through skip connections to compensate for the spatial information loss caused by the downsampling process. Inspired by the design of U-Net, methods such as Res-UNet, Att-UNet, U-Net++, and UNet3+ have been developed for organ lesion segmentation, and their excellent performance demonstrates the strong feature learning ability of convolutional neural networks.

[0004] However, the lack of ability to capture long-range dependencies makes it difficult for convolutional neural network-based methods to meet the segmentation accuracy requirements of medical applications. Transformers have achieved breakthroughs in certain areas of computer vision, resulting in a series of high-performance methods such as ViT, PVT, and Swing Transformers. Inspired by this, Transformers have gradually been applied to the field of medical image segmentation. Recently, the Deformable Attention Transformer, by designing a deformable self-attention mechanism, overcomes the drawback of methods like PVT and Swing Transformers potentially losing key relevant information, achieving state-of-the-art accuracy on the ImageNet dataset. Summary of the Invention

[0005] This invention provides a medical image segmentation method based on convolution and deformable self-attention mechanism, aiming to solve the problem that convolutional neural networks lack the ability to capture long-distance dependencies.

[0006] This invention is implemented as follows: a medical image segmentation method based on convolution and deformable self-attention mechanism, comprising the following steps:

[0007] Step S1: Preprocess and augment the CT images;

[0008] Step S2: Construct the U-Net architecture model Med-CaDA based on convolution and deformable self-attention mechanism;

[0009] Step S3: Apply the Dice loss function to the U-Net architecture model constructed in step S2;

[0010] Step S4: Train the U-Net architecture model constructed in step S2 using the Adam optimization algorithm;

[0011] Step S5: Use two metrics, Dice Score and 95% Hausdorff distance, to measure segmentation accuracy.

[0012] Furthermore, in step S1, the preprocessing uses the Z-score method for standardization, and the data augmentation includes padding, random cropping, random flipping, and random intensity shifting.

[0013] Furthermore, in step S2, the U-Net architecture model consists of an encoder, a decoder, and skip connections. The encoder includes an embedding layer, a CaDA block composed of convolution and deformable self-attention mechanisms, and a downsampling layer. The decoder includes an upsampling layer, a CaDA block, and an extension layer.

[0014] Furthermore, the Embedding layer is used to map the input image to a multi-dimensional space, with an output size of [missing value].

[0015] Furthermore, the CaDA block is used for local and global information extraction, and includes an inverted residual module composed of convolutions and a deformable self-attention mechanism. The bottleneck residual module consists of two 1×1×1 convolutions and a depthwise separable convolution, used for local information extraction and ensuring that the input and output feature map sizes are consistent. The bottleneck residual module can be represented as:

[0016] Bottleneck(X)=Conv(F(Conv(X))) (1)

[0017] F(X)=DWConv(X)+X (2)

[0018] Conv() represents convolution, DWConv() represents depthwise separable convolution, and BatchNorm normalization and GELU activation function are omitted.

[0019] Deformable self-attention mechanisms are a variant of self-attention mechanisms. They achieve global dependency capture without consuming a large amount of computation. Assuming that its input is H×W×D×C, its output is also H×W×D×C, and the input and output dimensions are consistent.

[0020] Furthermore, the downsampling layer and the upsampling layer are respectively a 2×2×2 convolution and a 2×2×2 deconvolution.

[0021] Furthermore, the extended layer employs a 4×4×4 deconvolution with a stride of 4 to transform the feature map from... Restore the original image to H×W×D×K, where K is the final number of segmentation categories.

[0022] Furthermore, in step S3, the Dice loss function is implemented by the following formula:

[0023]

[0024] |A∩B| represents the intersection between A and B, where |A| and |B| represent the number of elements in A and B, respectively.

[0025] Furthermore, in step S4, the learning rate of the Adam optimization algorithm is set to 1e-4.

[0026] Furthermore, in step S5, the Dice score is:

[0027]

[0028] The 95% Hausdorff distance index is:

[0029]

[0030] The distance(a,b) function calculates the distance between points a and b, where A and B are two irregular regions.

[0031] Compared with existing technologies, the advantages of this invention are as follows: This invention provides a medical image segmentation method based on convolution and deformable self-attention mechanisms, which fully extracts local and global contextual information by utilizing convolution and Transformer attention mechanisms. This method significantly improves the accuracy of medical image segmentation while maintaining low computational cost. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the method steps of the present invention;

[0033] Figure 2 This is a diagram of the Med-CaDA model, the U-Net architecture model used in this invention.

[0034] Figure 3 This is a diagram of the CaDA block model in this invention;

[0035] Figure 4 This is a flowchart of the deformable self-attention mechanism in this invention; Detailed Implementation

[0036] To better understand the above-mentioned objectives, technical solutions, and effects of the present invention, the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. It should be noted that many specific details are set forth in the following description to provide a thorough understanding of the present invention; however, the present invention may also be implemented in other ways different from those described below. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0037] Please see Figure 1 This invention provides a technical solution: a medical image segmentation method based on convolution and deformable self-attention mechanism, comprising the following steps:

[0038] Step S1: Preprocess and augment the CT images;

[0039] In this embodiment, preprocessing uses Z-score normalization to enhance the distinction between foreground and background. The formula is as follows:

[0040]

[0041] Where μ is the mean of all sample data and σ is the standard deviation of all sample data.

[0042] Data augmentation employs techniques including padding, random cropping, random flipping, and random intensity shifting. Specifically, an image with an input size of 240×240×155 is padded to 240×240×160, then randomly cropped to a size of 128×128×128. Finally, random flipping and random intensity shifting are applied sequentially to address the issue of insufficient data volume and improve the model's generalization ability.

[0043] Step S2: Construct the U-Net architecture model Med-CaDA based on convolution and deformable self-attention mechanism;

[0044] In this embodiment, the U-Net architecture model Med-CaDA is as follows: Figure 2 As shown, it consists of an encoder, a decoder, and skip connections. The encoder includes an embedding layer, a CaDA block composed of convolution and deformable self-attention mechanism, and a downsampling layer. The decoder includes an upsampling layer, a CaDA block, and an extension layer.

[0045] The Embedding layer is used to map the original image with an input size of H×W×D×4 to a multidimensional space, with an output size of H×W×D×4. It sequentially includes 3×3×3 convolutions with stride of 2 and padding of 1, 3×3×3 convolutions with stride of 1 and padding of 1, 3×3×3 convolutions with stride of 2 and padding of 1, and 3×3×3 convolutions with stride of 1 and padding of 1.

[0046] CaDA blocks, such as Figure 3 As shown, this is used for local and global information extraction, and includes a bottleneck residual module composed of convolutions and a deformable self-attention mechanism. The bottleneck residual module consists of a 1×1×1 convolution with a stride of 1, a 3×3×3 depthwise separable convolution, and a 1×1×1 convolution with a stride of 1, used for local information extraction and ensuring that the input and output feature map sizes are consistent. The first 1×1×1 convolution is responsible for halving the dimensions, and the second 1×1×1 convolution is responsible for doubling the dimensions.

[0047] The bottleneck residual module can be represented as:

[0048] Bottleneck(X)=Conv(F(Conv(X))) (2)

[0049] F(X)=DWConv(X)+X (3)

[0050] Here, Conv() represents convolution, DWConv() represents depthwise separable convolution, and the first Conv() is used to reduce the dimension to the original. (In practical applications, R=4). The second Conv() function restores the reduced dimensionality, thereby reducing the number of parameters. Additionally, BatchNorm normalization and the GELU activation function are omitted.

[0051] Deformable self-attention mechanisms are a variant of self-attention mechanisms. They capture global dependencies without consuming a large amount of computation. Assuming that its input is H×W×D×C, its output is also H×W×D×C, and the input and output dimensions are consistent.

[0052] like Figure 4 As shown, specifically: for input X, firstly generate a set of grid points P∈H. g ×W g ×D g ×3, using the Query key Q through the Offset network to learn the offset ΔP of grid point P (to prevent the generated ΔP from being too large, controllable parameters s and tanh() are used to control the size of ΔP), and then bilinear interpolation is used to obtain the sampling point X on the original image X. sampled Finally, based on the sampling points, key and value keys are generated and combined with the query key to calculate self-attention, and X is output. O .

[0053] It can be represented as:

[0054] Q = XW q (4)

[0055] ΔP=stanh(Offset(Q)) (5)

[0056] X sampled =BI(X,P+ΔP) (6)

[0057] K = X sampled W k V = X sampled W v (7)

[0058]

[0059] X o =ZW o (9)

[0060] In the formula, Offset() is an offset learning network composed of depthwise separable convolutions and 1×1×1 convolutions, used to generate an offset learning network of size H. g ×W g ×D g The offset ΔP is ×3, and BI represents bilinear interpolation.

[0061] The downsampling layer and upsampling layer are a 2×2×2 convolution with a stride of 2 and a 2×2×2 deconvolution with a stride of 2, respectively. The downsampling layer transforms the feature map from... Cascaded downsampling Upsampling transforms the feature map from Cascaded upsampling back size.

[0062] The extended layer uses a 4×4×4 deconvolution with a stride of 4 to transform the feature map from... Restore the original image to H×W×D×K, where K is the final number of segmentation categories.

[0063] Step S3: Apply the Dice loss function to the U-Net architecture model constructed in step S2;

[0064] In this embodiment, the Dice loss function is implemented by the following formula:

[0065]

[0066] |A∩B| represents the intersection between A and B, where |A| and |B| represent the number of elements in A and B, respectively.

[0067] During training, the Dice loss function can assess the degree of difference between the prediction and the actual value, and adjust the network weights to make the prediction more accurate.

[0068] Step S4: Train the U-Net architecture model constructed in step S2 using the Adam optimization algorithm;

[0069] In this implementation, the Adam optimization algorithm has a learning rate of 1e-4, 800 training epochs, and a batch size of 4. L2 regularization is used to mitigate overfitting, and the weight decay is set to 1e-5. Furthermore, a cosine learning rate decay strategy is employed to ensure that the training process reaches the optimal solution.

[0070] Step S5: Use two metrics, Dice Score and 95% Hausdorff distance, to measure segmentation accuracy.

[0071] In this implementation, the Dice score is:

[0072]

[0073] The 95% Hausdorff distance index is:

[0074]

[0075] The distance(a,b) function calculates the distance between points a and b, where A and B are two irregular regions.

[0076] This invention is the first to apply the deformable self-attention mechanism Transformer to medical image segmentation tasks, and proposes a CaDA block by combining it with a residual convolution module. This ensures both the model's ability to extract local information and captures global dependencies. Finally, experiments on the BraTS2020 dataset demonstrate that, compared to the classic U-Net architecture, this model significantly improves both performance metrics in the ET, WT, and TC regions. Compared to state-of-the-art methods, this model achieves comparable segmentation results with lower computational cost. The segmentation results are compared in Table 1 below:

[0077] Table 1 Comparison of segmentation results

[0078]

[0079] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A medical image segmentation method based on convolution and deformable self-attention mechanism, characterized in that: The method comprises the following steps: Step S1: preprocessing and data enhancement on the CT image; Step S2: constructing a Med-CaDA model based on a convolution and a deformable self-attention mechanism, the Med-CaDA model being composed of an encoder, a decoder and a skip connection, the encoder including an Embedding layer, a CaDA block composed of a convolution and a deformable self-attention mechanism, a down-sampling layer, the decoder including an up-sampling layer, a CaDA block, an expansion layer, the CaDA block being used for local-global information extraction and containing a bottleneck residual module composed of two convolutions and a depthwise separable convolution, for local information extraction and ensuring that the input and output feature map sizes are consistent; the bottleneck residual module can be expressed as: (1) (2) wherein denotes a convolution, denotes a depthwise separable convolution; Step S3: using a Dice loss function for the U-Net architecture model constructed in step S2; Step S4: training the U-Net architecture model constructed in step S2 by using an Adam optimization algorithm; Step S5: using two indexes of Dice Score and 95% Hausdorff distance to measure the segmentation accuracy; The deformable self-attention mechanism Transformer specifically comprises: for input X, first generating a set of grid points learning the offset of the grid points P through the Offset network via the Query key Q Then, the bilinear interpolation is used to obtain the sampling points on the original image X Finally, the Key and Value keys are generated according to the sampling points, and the self-attention is calculated together with the Query key to output .

2. The medical image segmentation method based on convolution and deformable self-attention mechanism according to claim 1, characterized in that: In step S1, the preprocessing is standardized by using a Z-score method, and the data enhancement includes padding, random cropping, random flipping, and random intensity shift.

3. The medical image segmentation method based on convolution and deformable self-attention mechanism according to claim 2, characterized in that: The Embedding layer is used to map the input image to a multi-dimensional space, and the output size is .

4. The medical image segmentation method based on convolution and deformable self-attention mechanism according to claim 1, characterized in that: The down-sampling layer and the up-sampling layer are respectively convolution and deconvolution.

5. The medical image segmentation method based on convolution and deformable self-attention mechanism according to claim 3, characterized in that: The extended layer adopts Deconvolution restores the feature map from the original image where K is the final number of segmentation classes.

6. The medical image segmentation method based on convolution and deformable self-attention mechanism according to claim 1, characterized in that: The step S3 is realized by the following formula: (3) denotes the intersection between A and B, and denotes the number of elements in A and B, respectively.

7. The medical image segmentation method based on convolution and deformable self-attention mechanism according to claim 1, characterized in that: The learning rate of the Adam optimization algorithm in step S4 is set to 1e-4.

8. The medical image segmentation method based on convolution and deformable self-attention mechanism according to claim 1, characterized in that: The Dice score index in step S5 is: (4) The 95% Hausdorff distance index is: (5) wherein is the distance between two points, A and B, of two irregular regions.

Images