A two-dimensional medical image segmentation method and system

Abstract

This invention belongs to the field of image segmentation and provides a method and system for two-dimensional medical image segmentation. The method includes acquiring and preprocessing a two-dimensional medical image to be segmented; performing image segmentation based on the preprocessed two-dimensional medical image using a pre-trained two-dimensional medical image segmentation model. To effectively utilize convolutional neural networks to extract local features from the image and fuse semantic information from deep low-resolution layers with local information from shallow high-resolution layers, this two-dimensional medical image segmentation model uses a convolutional attention enhancement module and skip connections with attention gating units to fuse and refine multi-level features. This simultaneously suppresses irrelevant regions and highlights salient features, thereby extracting effective local features from the image. By fusing high-resolution features from different scales of the encoder, global and local relationships between pixels are captured, mitigating spatial information loss caused by downsampling.

Description

Technical Field

[0001] This invention belongs to the field of image segmentation technology, specifically relating to a two-dimensional medical image segmentation method and system. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Medical image segmentation extracts key information from images of specific tissues and is a crucial step in visualizing medical images and assisting doctors in diagnosis and treatment planning. The task of medical image segmentation is to perform pixel-by-pixel classification and separate different organs or lesion regions within an image.

[0004] Pure convolutional neural networks (CNNs) possess outstanding capabilities in multi-scale representation and capturing local semantic and textural information, making them widely used in medical image segmentation tasks. Inspired by the FCN structure and encoder-decoder model, Ronneberger et al. developed the U-Net model for biomedical image segmentation. The U-Net model's extensive application in medical segmentation stems from its multiple convolutional upsampling and downsampling modules, as well as skip connections that propagate high-resolution contextual information within the network. Several U-Net versions with improvements to the backbone network or skip connections have demonstrated good performance on various medical image segmentation datasets, such as Residual U-Net, U-Net++, and U-Net 3+. For pure convolutional neural network-based medical image segmentation methods, while CNN-based methods offer satisfactory performance, their limitations in learning long-range dependencies between pixels result in insufficient capture of the geometric and structural information present in medical image data. To overcome this limitation, some works have incorporated attention modules into their architectures to enhance the extraction of local features. While these attention-based methods have achieved good performance, a mechanism for learning long-distance dependencies between pixels is still needed to extend the performance of existing CNN-based U-Net variants when the receptive field of convolution operations is limited.

[0005] Recent advances in Vision Transformers have to some extent overcome the limitations of segmentation models in capturing long-range dependencies. Transformers use self-attention to learn the correlations between all input tokens, enabling them to capture long-range dependencies. More specifically, Dosovitskiy et al. introduced the Vision Transformer model to alleviate the shortcomings of convolutional neural networks in capturing long-range semantic dependencies. It transforms the image into fixed-size blocks before flattening it into vectors and embeds Transformer modules through positional encoding. For medical image segmentation methods based on Vision Transformers: Transformers overcome the limitations in capturing long-range dependencies to some extent, but their use of self-attention limits their ability to learn local (contextual) relationships between pixels.

[0006] Combining Transformer with U-Net models has become a new research direction. For example, TransUNet and MedT are both image segmentation models that utilize the advantages of U-Net and Transformer. Transformer can capture long-range dependencies through self-attention, but lacks the ability to learn local (contextual) relationships between pixels. Previous works have attempted to overcome this problem by embedding convolutional layers in the encoder or decoder modules, but sometimes feature inconsistencies occur. For medical image segmentation methods based on a hybrid Transformer and CNN architecture: although models like TransUNet and MedT can learn local (contextual) information between pixels to some extent, they cannot effectively combine learning global-local context to enhance feature representation, and their decoders cannot correctly aggregate the multi-level features generated by the encoder. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a two-dimensional medical image segmentation method and system. This invention can suppress irrelevant regions and highlight salient features through our convolutional attention enhancement module, which is more effective in extracting local features from images compared to pure convolutional networks. The proposed skip connections with attention gating units can fuse high-resolution features from different scales of the encoder, successfully capturing global and local relationships between pixels to mitigate spatial information loss caused by downsampling.

[0008] According to some embodiments, the first aspect of the present invention provides a two-dimensional medical image segmentation system, which adopts the following technical solution:

[0009] A two-dimensional medical image segmentation system, comprising:

[0010] The image acquisition and processing module is configured to acquire and preprocess the two-dimensional medical image to be segmented.

[0011] The image segmentation module is configured to perform image segmentation based on pre-processed two-dimensional medical images using a pre-trained two-dimensional medical image segmentation model.

[0012] The step of using a pre-trained two-dimensional medical image segmentation model for image segmentation includes:

[0013] Convolutional feature maps are obtained by performing convolution and convolutional attention enhancement operations on preprocessed 2D medical images. The convolutional feature maps are then downsampled three times, and local information in the downsampled feature images is captured using convolutional attention enhancement to obtain encoded feature maps at different scales. The encoded feature maps after the third downsampling are then processed by Transformer to obtain encoded feature representation maps.

[0014] The encoded feature map is upsampled and then concatenated with the encoded feature map after a second downsampling. The concatenated result is then decoded using a fusion method of skip connections and attention gating units to obtain the first decoded feature map.

[0015] Using the same method, the decoded feature map is concatenated with the encoded feature map at different scales and then decoded twice to obtain decoded feature maps at different scales. The matrices are then added together to obtain the image segmentation result.

[0016] Furthermore, the acquisition and preprocessing of the two-dimensional medical image to be segmented specifically includes:

[0017] Obtain the two-dimensional medical image to be segmented;

[0018] The two-dimensional medical image to be segmented is randomly scaled and cropped;

[0019] Perform random flipping and rotation operations on the cropped 2D medical image;

[0020] The preprocessed two-dimensional medical image is obtained.

[0021] Furthermore, the method of capturing local information in the downsampled feature image using convolutional attention enhancement specifically involves:

[0022] Channel attention is used to perform channel dimensionality reduction and channel restoration operations on the downsampled feature maps after max pooling and average pooling of the image, respectively. The results are then concatenated and Hadamard inner product is performed with the original downsampled feature image.

[0023] Spatial attention is used to capture the maximum and average information in the feature image after channel attention processing;

[0024] By performing convolution operations, the maximum and average information captured by spatial attention are sequentially subjected to convolution operations, normalization, and activation function operations at different scales to obtain local information in the downsampled feature image.

[0025] Furthermore, the convolutional attention enhancement operation specifically includes:

[0026] CAA(x)=ConvBlock(SA(CA(x));

[0027] In the formula, x is the input tensor, CAA(x) represents the convolutional attention enhancement operation on x, CA(x) is the channel attention processing, and SA(x) is the spatial attention processing;

[0028] ConvBlock(x)=σ(BN(C1(σ(BN(C2(C2(σ(BN(C1(x)))))))));

[0029] In the formula, σ(x) is the ReLU activation layer, BN(x) represents the batch normalization operation, C1(x) is a 1×1 convolutional layer, and C2(x) is a 3×3 convolutional layer.

[0030] Furthermore, the channel attention processing specifically includes:

[0031]

[0032] In the formula, σ2(x) is the Sigmoid activation function, Pmax(x) and Pavg(x) represent adaptive max pooling and adaptive average pooling, respectively, C1(x) is a 1×1 convolutional layer that reduces the channel dimension by a factor of 16, σ1 is a ReLU activation layer, and C2(x) is a 1×1 convolutional layer that restores the channels to their original channel dimensions. The inner product of Hadama;

[0033] Spatial attention processing, specifically:

[0034]

[0035] In the formula, σ(x) is the Sigmoid activation function, Cmax(x) and Cavg(x) represent the maximum and average values ​​obtained along the channel dimension, respectively, and C(x) is a 3×3 convolutional layer with a padding of 1.

[0036] Furthermore, the process of concatenating the decoded feature map with encoded feature maps of different scales using the same method, followed by two decoding operations to obtain decoded feature maps of different scales, and then performing matrix addition to obtain the image segmentation result, includes:

[0037] The first decoded feature map is processed by convolutional attention enhancement and then upsampled by a factor of 2. It is then concatenated with the encoded feature map after the first downsampling. The concatenated result is then decoded by using skip connections and attention gating units to obtain the second decoded feature map.

[0038] The second decoded feature map is processed using convolutional attention enhancement and then upsampled by a factor of two before being concatenated with the convolutional feature map. The concatenated result is then decoded using skip connections and attention gating units to obtain the third decoded feature map. Figure 2 The fourth decoded feature map is obtained after upsampling.

[0039] The result of upsampling the first decoded feature map by eight times, and the second decoded feature map Figure 4 The result of upsampling, the third decoding feature Figure 2 The upsampling result and the fourth decoded feature map are added together to obtain the image segmentation result.

[0040] Furthermore, the decoder utilizes a fusion of skip connections and attention gating units to decode the concatenated features, specifically as follows:

[0041] AG(g,x)=x*σ2(BN(C(σ1(BN(Cg(g)+BN(Cx(x))))))));

[0042] In the formula, σ1(x) and σ2(x) correspond to the ReLU activation function and the Sigmoid activation function, respectively; Cg(x), Cx(x) and C(x) represent the channel 1×1 convolution operation; BN(x) is the batch normalization operation, and g and x are the upsampled feature map in the decoder and the feature map from the encoder in the skip connection, respectively.

[0043] According to some embodiments, the second aspect of the present invention provides a two-dimensional medical image segmentation method, which adopts the following technical solution:

[0044] A two-dimensional medical image segmentation method, comprising:

[0045] Acquire the two-dimensional medical image to be segmented and perform preprocessing;

[0046] Based on the preprocessed two-dimensional medical image, image segmentation is performed using a pre-trained two-dimensional medical image segmentation model.

[0047] The step of using a pre-trained two-dimensional medical image segmentation model for image segmentation includes:

[0048] Convolutional feature maps are obtained by performing convolution and convolutional attention enhancement operations on preprocessed 2D medical images. The convolutional feature maps are then downsampled three times, and local information in the downsampled feature images is captured using convolutional attention enhancement to obtain encoded feature maps at different scales. The encoded feature maps after the third downsampling are then processed by Transformer to obtain encoded feature representation maps.

[0049] The encoded feature map is upsampled and then concatenated with the encoded feature map after a second downsampling. The concatenated result is then decoded using a fusion method of skip connections and attention gating units to obtain the first decoded feature map.

[0050] Using the same method, the decoded feature map is concatenated with the encoded feature map at different scales and then decoded twice to obtain decoded feature maps at different scales. The matrices are then added together to obtain the image segmentation result.

[0051] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium.

[0052] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0053] Acquire the two-dimensional medical image to be segmented and perform preprocessing;

[0054] Based on the preprocessed two-dimensional medical image, image segmentation is performed using a pre-trained two-dimensional medical image segmentation model.

[0055] The step of using a pre-trained two-dimensional medical image segmentation model for image segmentation includes:

[0056] Convolutional feature maps are obtained by performing convolution and convolutional attention enhancement operations on preprocessed 2D medical images. The convolutional feature maps are then downsampled three times, and local information in the downsampled feature images is captured using convolutional attention enhancement to obtain encoded feature maps at different scales. The encoded feature maps after the third downsampling are then processed by Transformer to obtain encoded feature representation maps.

[0057] The encoded feature map is upsampled and then concatenated with the encoded feature map after a second downsampling. The concatenated result is then decoded using a fusion method of skip connections and attention gating units to obtain the first decoded feature map.

[0058] Using the same method, the decoded feature map is concatenated with the encoded feature map at different scales and then decoded twice to obtain decoded feature maps at different scales. The matrices are then added together to obtain the image segmentation result.

[0059] According to some embodiments, a fourth aspect of the present invention provides a computer device.

[0060] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor performing the following steps:

[0061] Acquire the two-dimensional medical image to be segmented and perform preprocessing;

[0062] Based on the preprocessed two-dimensional medical image, image segmentation is performed using a pre-trained two-dimensional medical image segmentation model.

[0063] The step of using a pre-trained two-dimensional medical image segmentation model for image segmentation includes:

[0064] Convolutional feature maps are obtained by performing convolution and convolutional attention enhancement operations on preprocessed 2D medical images. The convolutional feature maps are then downsampled three times, and local information in the downsampled feature images is captured using convolutional attention enhancement to obtain encoded feature maps at different scales. The encoded feature maps after the third downsampling are then processed by Transformer to obtain encoded feature representation maps.

[0065] The encoded feature map is upsampled and then concatenated with the encoded feature map after a second downsampling. The concatenated result is then decoded using a fusion method of skip connections and attention gating units to obtain the first decoded feature map.

[0066] Using the same method, the decoded feature map is concatenated with the encoded feature map at different scales and then decoded twice to obtain decoded feature maps at different scales. The matrices are then added together to obtain the image segmentation result.

[0067] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0068] To effectively extract local features from images using convolutional neural networks and fuse semantic information from deep, low-resolution layers with local information from shallow, high-resolution layers, this invention employs a convolutional attention enhancement module and skip connections with attention gating units to fuse and refine multi-level features. A hybrid architecture of Transformer and CNN is used as the backbone network, and attention-based convolutional modules aggregate multi-level features while suppressing irrelevant regions and highlighting salient features, thereby extracting effective local features from the image. The combination of skip connections and attention gating units allows this module to fuse high-resolution features from different scales of the encoder, capturing both global and local relationships between pixels to mitigate spatial information loss caused by downsampling. Attached Figure Description

[0069] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0070] Figure 1 This is a flowchart of a two-dimensional medical image segmentation method according to an embodiment of the present invention;

[0071] Figure 2 This is a flowchart of the training of the two-dimensional medical image segmentation model in an embodiment of the present invention;

[0072] Figure 3 This is a framework diagram of a two-dimensional medical image segmentation model in an embodiment of the present invention;

[0073] Figure 4 This is a visualization comparison chart of the two-dimensional medical image segmentation model and other models in this embodiment of the invention. Detailed Implementation

[0074] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0075] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0076] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0077] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0078] Example 1

[0079] This embodiment provides a two-dimensional medical image segmentation system, including:

[0080] The image acquisition and processing module is configured to acquire and preprocess the two-dimensional medical image to be segmented.

[0081] The image segmentation module is configured to perform image segmentation based on pre-processed two-dimensional medical images using a pre-trained two-dimensional medical image segmentation model.

[0082] The step of using a pre-trained two-dimensional medical image segmentation model for image segmentation includes:

[0083] Convolutional feature maps are obtained by performing convolution and convolutional attention enhancement operations on preprocessed 2D medical images. The convolutional feature maps are then downsampled three times, and local information in the downsampled feature images is captured using convolutional attention enhancement to obtain encoded feature maps at different scales. The encoded feature maps after the third downsampling are then processed by Transformer to obtain encoded feature representation maps.

[0084] The encoded feature map is upsampled and then concatenated with the encoded feature map after a second downsampling. The concatenated result is then decoded using a fusion method of skip connections and attention gating units to obtain the first decoded feature map.

[0085] Using the same method, the decoded feature map is concatenated with the encoded feature map at different scales and then decoded twice to obtain decoded feature maps at different scales. The matrices are then added together to obtain the image segmentation result.

[0086] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0087] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0088] Example 2

[0089] like Figure 1 As shown, this embodiment provides a two-dimensional medical image segmentation method. This embodiment uses the application of this method to a server as an example for illustration. It is understood that this method can also be applied to a terminal, or to a system including a terminal, server, and system, and is implemented through interaction between the terminal and server. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communication, middleware services, domain name services, CDN security services, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein. In this embodiment, the method includes the following steps:

[0090] Step S01: Acquire the two-dimensional medical image to be segmented and perform preprocessing;

[0091] Step S02: Based on the preprocessed two-dimensional medical image, perform image segmentation using a pre-trained two-dimensional medical image segmentation model;

[0092] In step S02, the image segmentation using a pre-trained two-dimensional medical image segmentation model includes:

[0093] Step S021: Based on the preprocessed two-dimensional medical image, perform convolution operation and convolution attention enhancement operation to obtain convolution feature map, and perform three downsampling operations on the convolution feature map, and use convolution attention enhancement to capture local information in the downsampled feature image to obtain coded feature maps at different scales; then perform Transformer processing on the coded feature map after the third downsampling to obtain coded feature representation map;

[0094] Step S022: The upsampled encoded feature map is concatenated with the downsampled encoded feature map, and the concatenated result is decoded by using skip connections and attention gating units to obtain the first decoded feature map;

[0095] Step S023: Using the same method, concatenate the decoded feature map with the encoded feature map at different scales and perform two decoding operations to obtain decoded feature maps at different scales. Then, perform matrix addition to obtain the image segmentation result.

[0096] Specifically, in step S01, the acquisition and preprocessing of the two-dimensional medical image to be segmented includes:

[0097] Obtain the two-dimensional medical image to be segmented;

[0098] The two-dimensional medical image to be segmented is randomly scaled and cropped;

[0099] Perform random flipping and rotation operations on the cropped 2D medical image;

[0100] The preprocessed two-dimensional medical image is obtained.

[0101] In step S021, the method of capturing local information in the downsampled feature image using convolutional attention enhancement specifically involves:

[0102] Channel attention is used to perform channel dimensionality reduction and channel restoration operations on the feature maps after max pooling and average pooling of the image, respectively. The results are then concatenated and Hadamard inner product is performed with the original downsampled feature image to determine which features in the image are important.

[0103] Spatial attention is used to capture the maximum and average information in the feature image after channel attention processing, thereby determining where the feature image is meaningful;

[0104] By performing convolution operations, the maximum and average information captured by spatial attention are sequentially subjected to convolution operations, normalization, and activation function operations at different scales to obtain local information in the downsampled feature image.

[0105] The convolutional attention enhancement operation is specifically as follows:

[0106] CAA(x)=ConvBlock(SA(CA(x));

[0107] In the formula, x is the input tensor, CAA(x) represents the convolutional attention enhancement operation on x, CA(x) is the channel attention processing, and SA(x) is the spatial attention processing;

[0108] ConvBlock(x)=σ(BN(C1(σ(BN(C2(C2(σ(BN(C1(x)))))))));

[0109] In the formula, σ(x) is the ReLU activation layer, BN(x) represents the batch normalization operation, C1(x) is a 1×1 convolutional layer, and C2(x) is a 3×3 convolutional layer.

[0110] The channel attention processing specifically includes:

[0111]

[0112] In the formula, σ2(x) is the Sigmoid activation function, Pmax(x) and Pavg(x) represent adaptive max pooling and adaptive average pooling, respectively, C1(x) is a 1×1 convolutional layer that reduces the channel dimension by a factor of 16, σ1 is a ReLU activation layer, and C2(x) is a 1×1 convolutional layer that restores the channels to their original channel dimensions. The inner product of Hadama;

[0113] Spatial attention processing, specifically:

[0114]

[0115] In the formula, σ(x) is the Sigmoid activation function, Cmax(x) and Cavg(x) represent the maximum and average values ​​obtained along the channel dimension, respectively, and C(x) is a 3×3 convolutional layer with a padding of 1.

[0116] In a specific embodiment, in step S023, the process of concatenating the decoded feature map with encoded feature maps of different scales using the same method, performing two decoding operations to obtain decoded feature maps of different scales, and then adding the matrices to obtain the image segmentation result includes:

[0117] The first decoded feature map is processed by convolutional attention enhancement and then upsampled by a factor of 2. It is then concatenated with the encoded feature map after the first downsampling. The concatenated result is then decoded by using skip connections and attention gating units to obtain the second decoded feature map.

[0118] The second decoded feature map is processed using convolutional attention enhancement and then upsampled by a factor of two before being concatenated with the convolutional feature map. The concatenated result is then decoded using skip connections and attention gating units to obtain the third decoded feature map. Figure 2 The fourth decoded feature map is obtained after upsampling.

[0119] The result of upsampling the first decoded feature map by eight times, and the second decoded feature map Figure 4 The result of upsampling, the third decoding feature Figure 2 The upsampling result and the fourth decoded feature map are added together to obtain the image segmentation result.

[0120] The decoder uses a fusion of skip connections and attention gating units to decode the concatenated features, specifically:

[0121] AG(g,x)=x*σ2(BN(C(σ1(BN(Cg(g)+BN(Cx(x))))))));

[0122] In the formula, σ1(x) and σ2(x) correspond to the ReLU activation function and the Sigmoid activation function, respectively; Cg(x), Cx(x) and C(x) represent the channel 1×1 convolution operation; BN(x) is the batch normalization operation, and g and x are the upsampled feature map in the decoder and the feature map from the encoder in the skip connection, respectively.

[0123] To effectively utilize convolutional neural networks to extract local features from images and fuse semantic information from deep, low-resolution layers with local information from shallow, high-resolution layers, this embodiment proposes the ConvolutionalAttention Augmented TransUNet (CAA-TransUNet) model, a two-dimensional medical image segmentation model. This model uses convolutional attention enhancement modules and skip connections with attention-gated units to fuse and refine multi-level features. By using a hybrid architecture of Transformer and CNN as the backbone network and aggregating multi-level features using attention-based convolutional modules, CAA-TransUNet can capture both global and local (contextual) relationships between pixels.

[0124] (1) A novel convolutional attention enhancement module is proposed to enhance global and local features. This module can simultaneously suppress irrelevant regions and highlight salient features, thereby extracting effective local features from images. (2) A skip connection with attention gating unit is proposed, which combines skip connections with attention gating unit. This module can fuse high-resolution features from different scales of the encoder, thereby capturing global and local relationships between pixels to mitigate spatial information loss caused by downsampling, so as to aggregate feature information of corresponding stages in the encoder and decoder. (3) A multi-stage feature loss aggregation method is proposed to achieve faster convergence speed and better performance. The model of this embodiment is evaluated on three public datasets: ACDC, MoNuSeg, and GlaS. Experimental results show that the model proposed in this embodiment significantly outperforms the baseline method.

[0125] like Figure 2 As shown, the training process for the two-dimensional medical image segmentation model is as follows:

[0126] Step 1. Training Data Acquisition: Three publicly available medical image segmentation datasets—ACDC, MoNuSeg, and GlaS—were selected as training data. The Automated Heart Diagnosis Challenge dataset (ACDC) was split into 70 training samples, 10 validation samples, and 20 test samples. The gland segmentation dataset GlaS contains 85 training images and 80 test images. The cell nucleus segmentation dataset MoNuSeg contains 30 images with approximately 2000 nucleus boundary annotations.

[0127] Step 2. Data Preprocessing and Augmentation: First, the original images and ground truth segmentation images in the dataset are randomly scaled and cropped to a size of 224×224. Second, the cropped training images and their corresponding ground truth segmentation images are randomly flipped and rotated with a 50% probability. This not only effectively compensates for the limited number of training images in the medical dataset and improves the model's robustness, but also enhances the model's ability to resist overfitting.

[0128] Step 3. As Figure 3 The diagram shows the framework of a 2D medical image segmentation model based on a hybrid Transformer and CNN architecture. It consists of two parts: the first part is a CNN-Transformer hybrid encoder with convolutional attention enhancement; the second part is a decoder composed of skip connections with attention-gated units. The specific implementations of these two parts are as follows:

[0129] CNN-Transformer hybrid architecture encoder with convolutional attention enhancement:

[0130] The original input image is 3×224×224 in size, where 3 represents the number of channels in the feature image, and 224×224 represents the height and width of the feature image. The input image x1 is processed using a 7×7 convolutional kernel with a stride of 2 and padding of 3, and then passed through a BatchNorm layer, ReLU activation function mapping, and a convolutional attention enhancement module to obtain a feature map x2 of size 128×112×112.

[0131] The feature map x2 is downsampled using a 3×3 convolutional kernel with a stride of 2 and a padding of 1. Then, it is mapped through a BatchNorm layer and a ReLU activation function. The output is fed into our self-designed convolutional attention enhancement module to obtain a feature map x3 of size 256×56×56. This approach can better capture local information in the image.

[0132] The feature map x3 is downsampled using a 3×3 convolution kernel with a stride of 2 and a padding of 1, and then passed through a BatchNorm layer, a ReLU layer, and a convolutional attention enhancement module to obtain a feature map x4 of size 512×28×28.

[0133] The feature map x4 is downsampled using a 3×3 convolutional kernel with a stride of 2 and a padding of 1, and then passed through a BatchNorm layer, a ReLU layer, and a convolutional attention enhancement module to obtain a feature map x5 of size 1024×14×14.

[0134] The 1024×14×14 feature map x5 is positionally encoded and then fed into a Transformer with vit_transformer_dim of 1024 and patch_size of 2. The result is transformed into a 1024×14×14 feature map x6.

[0135] The convolutional attention enhancement module in the above operation consists of channel attention, spatial attention, and convolutional block (ConvBlock), as shown in equation (1):

[0136] CAA(x)=ConvBlock(SA(CA(x)) (1)

[0137] (1) In the formula, x is the input tensor, CAA(x) represents the convolutional attention enhancement operation on x, CA(x) is the channel attention processing, and SA(x) is the spatial attention processing.

[0138] Formula (2) defines the channel attention CA(x):

[0139]

[0140] In equation (2), σ2(x) is the Sigmoid activation function, Pmax(x) and Pavg(x) represent adaptive max pooling and adaptive average pooling, respectively, C1(x) is a 1×1 convolutional layer that reduces the channel dimension by a factor of 16, σ1 is a ReLU activation layer, and C2(x) is a 1×1 convolutional layer that restores the channels to their original channel dimensions. It is the inner product of Hadama.

[0141] Formula (3) defines the spatial attention SA(x):

[0142]

[0143] (3) In the formula, σ(x) is the Sigmoid activation function, Cmax(x) and Cavg(x) represent the maximum value and average value obtained along the channel dimension, respectively, and C(x) is a 3×3 convolutional layer with a padding of 1.

[0144] Formula (4) defines ConvBloc(x):

[0145] CoonvBlock(x)=σ(BN(C1(σ(BN(C2(σ(BN(C1(x)))))))) (4)

[0146] (4) In the formula, σ(x) is the ReLU activation layer, BN(x) represents the batch normalization operation, C1(x) is the 1×1 convolutional layer, and C2(x) is the 3×3 convolutional layer.

[0147] A decoder consisting of skip connections with attention-gated units:

[0148] The feature map x6 obtained from the encoder is passed through the convolutional attention enhancement module. The result is upsampled by 2 times while keeping the number of channels unchanged. It is then concatenated with the feature map x4 in the encoder and sent to the attention gating unit. The result is then passed through the convolutional attention enhancement module to obtain a feature map x7 of size 256×28×28.

[0149] The feature map x7 is upsampled by a factor of two while keeping the number of channels the same. It is then concatenated with the feature map x3 in the encoder and fed into the attention gating unit. The result is then processed by the convolutional attention enhancement module to obtain a feature map x8 of size 128×56×56.

[0150] Feature map x8 is upsampled by a factor of two while keeping the number of channels unchanged. It is then concatenated with feature map x2 in the encoder and fed into the attention gating unit. The result is then processed by the convolutional attention enhancement module to obtain feature map x9 of size 64×112×112. Feature map x9 is then processed by a linear layer to obtain feature map x10 of size 64×224×224.

[0151] The results of upsampling feature map x7 by 8 times, feature map x8 by 4 times, feature map x9 by 2 times, and feature map x10 are added together. Finally, the predicted map x11 is obtained by passing it through a linear layer.

[0152] Formula (5) defines the attention gating unit:

[0153] AG(g,x)=x*σ2(BN(C(σ1(BN(Cg(g)+BN(Cx(x)))))))) (5)

[0154] (5) In the formula, σ1(x) and σ2(x) correspond to the ReLU activation function and the Sigmoid activation function, respectively. Cg(x), Cx(x) and C(x) represent the channel 1×1 convolution operation; BN(x) is the batch normalization operation, and g and x are the upsampled feature map in the decoder and the feature map from the encoder in the skip connection, respectively.

[0155] Step 4. Multi-stage feature loss aggregation method. The loss function is used to calculate the error between the model's predicted value and the actual segmented image. We use a combination of the cross-entropy loss function and the Dice loss function as the model's loss function. The outputs of the four prediction heads are calculated separately, multiplied by their respective weights, and then summed to obtain the final loss.

[0156] loss x =0.5×CE_Loss(x)+0.5×DICE_Loss(x) (6)

[0157] Loss = α × loss x7 +β×loss x8 +γ×lossx9 +ζ×loss x10 (7)

[0158] Equation (6) defines that for each feature map x in each stage, we use 0.5 times CE_Loss and 0.5 times Dice_Loss as the loss of that feature map. Equation (7) means that the loss is calculated for the predicted maps after upsampling the feature maps x7, x8, x9, and x10 in the four stages, and then each is multiplied by a weight and added together to obtain the final loss of the model. Here, we take α = β = γ = ζ = 1.

[0159] Step 5. Use the AdamW optimizer and set the initial learning rate to 0.001 and the weight decay to 1e-4. Setting the weight decay coefficient can prevent the model from overfitting, and adaptive learning rate adjustment can speed up the convergence of the model.

[0160] Step 6. Model Training and Testing: Following the description in Step 2, preprocess and augment the training images. The resulting training images are then fed into a CNN-Transformer hybrid architecture encoder with convolutional attention enhancement, and then into a decoder composed of skip connections with attention-gated units to obtain the final prediction map. The loss between the predicted map and the ground truth segmentation map is calculated using the loss function designed in Step 4. Finally, the gradient is updated using the AdamW optimizer from Step 5. Each training iteration contains 8 samples, and a total of 200 training iterations are performed. For the GlaS and MoNoSeg datasets, average IoU and Dice score are used as evaluation metrics; for the ACDC dataset, only the Dice score is used.

[0161] The comparative models used in the experiment are the current mainstream medical image segmentation models TransUNet and Swin-Unet. Table 1 shows the experimental data comparison with other models, and the visualization comparison with other models can be found in [link to visualization]. Figure 4 .

[0162] Table 1. Experimental comparison results of this method with other models.

[0163]

[0164]

[0165] This embodiment uses a hybrid Transformer and CNN architecture for medical image segmentation. The proposed method introduces a novel convolutional attention enhancement module that suppresses irrelevant regions and highlights salient features, extracting more effective local features from images compared to pure convolutional networks. The proposed skip connections with attention-gated units fuse high-resolution features from different scales of the encoder, successfully capturing global and local relationships between pixels to mitigate spatial information loss caused by downsampling. Combining this with a multi-stage feature loss aggregation method and selecting optimal network parameters, optimization algorithms, and learning rates results in faster convergence and better performance.

[0166] Example 3

[0167] This embodiment provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the following steps:

[0168] Acquire the two-dimensional medical image to be segmented and perform preprocessing;

[0169] Based on the preprocessed two-dimensional medical image, image segmentation is performed using a pre-trained two-dimensional medical image segmentation model.

[0170] The step of using a pre-trained two-dimensional medical image segmentation model for image segmentation includes:

[0171] Convolutional feature maps are obtained by performing convolution and convolutional attention enhancement operations on preprocessed 2D medical images. The convolutional feature maps are then downsampled three times, and local information in the downsampled feature images is captured using convolutional attention enhancement to obtain encoded feature maps at different scales. The encoded feature maps after the third downsampling are then processed by Transformer to obtain encoded feature representation maps.

[0172] The encoded feature map is upsampled and then concatenated with the encoded feature map after a second downsampling. The concatenated result is then decoded using a fusion method of skip connections and attention gating units to obtain the first decoded feature map.

[0173] Using the same method, the decoded feature map is concatenated with the encoded feature map at different scales and then decoded twice to obtain decoded feature maps at different scales. The matrices are then added together to obtain the image segmentation result.

[0174] Example 4

[0175] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor performs the following steps:

[0176] Acquire the two-dimensional medical image to be segmented and perform preprocessing;

[0177] Based on the preprocessed two-dimensional medical image, image segmentation is performed using a pre-trained two-dimensional medical image segmentation model.

[0178] The step of using a pre-trained two-dimensional medical image segmentation model for image segmentation includes:

[0179] Convolutional feature maps are obtained by performing convolution and convolutional attention enhancement operations on preprocessed 2D medical images. The convolutional feature maps are then downsampled three times, and local information in the downsampled feature images is captured using convolutional attention enhancement to obtain encoded feature maps at different scales. The encoded feature maps after the third downsampling are then processed by Transformer to obtain encoded feature representation maps.

[0180] The encoded feature map is upsampled and then concatenated with the encoded feature map after a second downsampling. The concatenated result is then decoded using a fusion method of skip connections and attention gating units to obtain the first decoded feature map.

[0181] Using the same method, the decoded feature map is concatenated with the encoded feature map at different scales and then decoded twice to obtain decoded feature maps at different scales. The matrices are then added together to obtain the image segmentation result.

[0182] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0183] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0184] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0185] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0186] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0187] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A two-dimensional medical image segmentation system, characterized in that, include: The image acquisition and processing module is configured to acquire and preprocess the two-dimensional medical image to be segmented. The image segmentation module is configured to perform image segmentation based on pre-processed two-dimensional medical images using a pre-trained two-dimensional medical image segmentation model. The step of using a pre-trained two-dimensional medical image segmentation model for image segmentation includes: Convolutional feature maps are obtained by performing convolution and convolutional attention enhancement operations on preprocessed 2D medical images. The convolutional feature maps are then downsampled three times, and local information in the downsampled feature images is captured using convolutional attention enhancement to obtain encoded feature maps at different scales. The encoded feature maps after the third downsampling are then processed by Transformer to obtain encoded feature representation maps. The encoded feature map is upsampled and then concatenated with the encoded feature map after a second downsampling. The concatenated result is then decoded using a fusion method of skip connections and attention gating units to obtain the first decoded feature map. Using the same method, the decoded feature map is concatenated with the encoded feature maps at different scales, and then decoded twice to obtain decoded feature maps at different scales. These decoded feature maps are then matrix-added to obtain the image segmentation result, including: The first decoded feature map is processed by convolutional attention enhancement and then upsampled by a factor of 2. It is then concatenated with the encoded feature map after the first downsampling. The concatenated result is then decoded by using skip connections and attention gating units to obtain the second decoded feature map. The second decoded feature map is processed using convolutional attention enhancement and then upsampled by a factor of two before being concatenated with the convolutional feature map. The concatenated result is then decoded using skip connections and attention gating units to obtain the third decoded feature map, specifically: ； In the formula, σ 1( x )and σ 2( x These correspond to the ReLU activation function and the Sigmoid activation function, respectively. Cg ( x ), Cx ( x )and C ( x () indicates a 1×1 channel convolution operation; BN ( x This is a batch normalization operation. g and x These are the upsampled feature map in the decoder and the feature map from the encoder in the skip connections, respectively. The fourth decoded feature map is obtained by upsampling the third decoded feature map by a factor of two. The image segmentation result is obtained by performing a matrix addition operation on the results of upsampling the first decoded feature map by eight times, the second decoded feature map by four times, the third decoded feature map by two times, and the fourth decoded feature map.

2. The two-dimensional medical image segmentation system as described in claim 1, characterized in that, The acquisition and preprocessing of the two-dimensional medical image to be segmented specifically includes: Obtain the two-dimensional medical image to be segmented; The two-dimensional medical image to be segmented is randomly scaled and cropped; Perform random flipping and rotation operations on the cropped 2D medical image; The preprocessed two-dimensional medical image is obtained.

3. The two-dimensional medical image segmentation system as described in claim 1, characterized in that, The method of capturing local information in downsampled feature images using convolutional attention enhancement specifically involves: Channel attention is used to perform channel dimensionality reduction and channel restoration operations on the downsampled feature maps after max pooling and average pooling of the image, respectively. The results are then concatenated and Hadamard inner product is performed with the original downsampled feature image. Spatial attention is used to capture the maximum and average information in the feature image after channel attention processing; By performing convolution operations, the maximum and average information captured by spatial attention are sequentially subjected to convolution operations, normalization, and activation function operations at different scales to obtain local information in the downsampled feature image.

4. The two-dimensional medical image segmentation system as described in claim 3, characterized in that, The convolutional attention enhancement operation is specifically as follows: ； In the formula, x For the input tensor, CAA ( x ) indicates to x Perform convolutional attention enhancement operations. CA ( x This is for channel attention processing. SA ( x This is for spatial attention processing; ； In the formula, σ ( x () is the ReLU activation layer. BN ( x () indicates a batch normalization operation. C 1( x () is a 1×1 convolutional layer. C 2( x () is a 3×3 convolutional layer.

5. A two-dimensional medical image segmentation system as described in claim 4, characterized in that, The channel attention processing specifically includes: ； In the formula, σ 2( x ) is the Sigmoid activation function. Pmax ( x )and Pavg ( x ) represent adaptive max pooling and adaptive average pooling, respectively. C 1( x This is a 1×1 kernel-sized convolutional layer that reduces the channel dimension by a factor of 16. σ 1 is a ReLU activation layer. C 2( x () is a 1×1 convolutional layer that restores the channels to their original channel dimensions. The inner product of Hadama; Spatial attention processing, specifically: ； In the formula, σ ( x ) is the Sigmoid activation function. Cmax ( x )and Cavg ( x () represent the maximum and average values ​​obtained along the channel dimension, respectively. C ( x ) is a 3×3 convolutional layer with padding of 1.

6. A two-dimensional medical image segmentation method, characterized in that, include: Acquire the two-dimensional medical image to be segmented and perform preprocessing; Based on the preprocessed two-dimensional medical image, image segmentation is performed using a pre-trained two-dimensional medical image segmentation model. The step of using a pre-trained two-dimensional medical image segmentation model for image segmentation includes: Convolutional feature maps are obtained by performing convolution and convolutional attention enhancement operations on preprocessed 2D medical images. The convolutional feature maps are then downsampled three times, and local information in the downsampled feature images is captured using convolutional attention enhancement to obtain encoded feature maps at different scales. The encoded feature maps after the third downsampling are then processed by Transformer to obtain encoded feature representation maps. The encoded feature map is upsampled and then concatenated with the encoded feature map after a second downsampling. The concatenated result is then decoded using a fusion method of skip connections and attention gating units to obtain the first decoded feature map. Using the same method, the decoded feature map is concatenated with the encoded feature maps at different scales, and then decoded twice to obtain decoded feature maps at different scales. These decoded feature maps are then matrix-added to obtain the image segmentation result, including: The first decoded feature map is processed by convolutional attention enhancement and then upsampled by a factor of 2. It is then concatenated with the encoded feature map after the first downsampling. The concatenated result is then decoded by using skip connections and attention gating units to obtain the second decoded feature map. The second decoded feature map is processed using convolutional attention enhancement and then upsampled by a factor of two before being concatenated with the convolutional feature map. The concatenated result is then decoded using skip connections and attention gating units to obtain the third decoded feature map, specifically: ； In the formula, σ 1( x )and σ 2( x These correspond to the ReLU activation function and the Sigmoid activation function, respectively. Cg ( x ), Cx ( x )and C ( x () indicates a 1×1 channel convolution operation; BN ( x This is a batch normalization operation. g and x These are the upsampled feature map in the decoder and the feature map from the encoder in the skip connections, respectively. The fourth decoded feature map is obtained by upsampling the third decoded feature map by a factor of two. The image segmentation result is obtained by performing a matrix addition operation on the results of upsampling the first decoded feature map by eight times, the second decoded feature map by four times, the third decoded feature map by two times, and the fourth decoded feature map.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, The processor executes the following steps on the program: Acquire the two-dimensional medical image to be segmented and perform preprocessing; Based on the preprocessed two-dimensional medical image, image segmentation is performed using a pre-trained two-dimensional medical image segmentation model. The step of using a pre-trained two-dimensional medical image segmentation model for image segmentation includes: Convolutional feature maps are obtained by performing convolution and convolutional attention enhancement operations on preprocessed 2D medical images. The convolutional feature maps are then downsampled three times, and local information in the downsampled feature images is captured using convolutional attention enhancement to obtain encoded feature maps at different scales. The encoded feature maps after the third downsampling are then processed by Transformer to obtain encoded feature representation maps. The encoded feature map is upsampled and then concatenated with the encoded feature map after a second downsampling. The concatenated result is then decoded using a fusion method of skip connections and attention gating units to obtain the first decoded feature map. Using the same method, the decoded feature map is concatenated with the encoded feature maps at different scales, and then decoded twice to obtain decoded feature maps at different scales. These decoded feature maps are then matrix-added to obtain the image segmentation result, including: The first decoded feature map is processed by convolutional attention enhancement and then upsampled by a factor of two. It is then concatenated with the encoded feature map after the first downsampling. The concatenated result is decoded by using skip connections and attention gating units to obtain the second decoded feature map. The second decoded feature map is processed using convolutional attention enhancement and then upsampled by a factor of two before being concatenated with the convolutional feature map. The concatenated result is then decoded using skip connections and attention gating units to obtain the third decoded feature map, specifically: ； In the formula, σ 1( x )and σ 2( x These correspond to the ReLU activation function and the Sigmoid activation function, respectively. Cg ( x ), Cx ( x )and C ( x () indicates a 1×1 channel convolution operation; BN ( x This is a batch normalization operation. g and x These are the upsampled feature map in the decoder and the feature map from the encoder in the skip connections, respectively. The fourth decoded feature map is obtained by upsampling the third decoded feature map by a factor of two. The image segmentation result is obtained by performing a matrix addition operation on the results of upsampling the first decoded feature map by eight times, the second decoded feature map by four times, the third decoded feature map by two times, and the fourth decoded feature map.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor performs the following steps: Acquire the two-dimensional medical image to be segmented and perform preprocessing; Based on the preprocessed two-dimensional medical image, image segmentation is performed using a pre-trained two-dimensional medical image segmentation model. The step of using a pre-trained two-dimensional medical image segmentation model for image segmentation includes: Convolutional feature maps are obtained by performing convolution and convolutional attention enhancement operations on preprocessed 2D medical images. The convolutional feature maps are then downsampled three times, and local information in the downsampled feature images is captured using convolutional attention enhancement to obtain encoded feature maps at different scales. The encoded feature maps after the third downsampling are then processed by Transformer to obtain encoded feature representation maps. The encoded feature map is upsampled and then concatenated with the encoded feature map after a second downsampling. The concatenated result is then decoded using a fusion method of skip connections and attention gating units to obtain the first decoded feature map. Using the same method, the decoded feature map is concatenated with the encoded feature maps at different scales, and then decoded twice to obtain decoded feature maps at different scales. These decoded feature maps are then matrix-added to obtain the image segmentation result, including: The first decoded feature map is processed by convolutional attention enhancement and then upsampled by a factor of two. It is then concatenated with the encoded feature map after the first downsampling. The concatenated result is decoded by using skip connections and attention gating units to obtain the second decoded feature map. The second decoded feature map is processed using convolutional attention enhancement and then upsampled by a factor of two before being concatenated with the convolutional feature map. The concatenated result is then decoded using skip connections and attention gating units to obtain the third decoded feature map, specifically: ； In the formula, σ 1( x )and σ 2( x These correspond to the ReLU activation function and the Sigmoid activation function, respectively. Cg ( x ), Cx ( x )and C ( x () indicates a 1×1 channel convolution operation; BN ( x This is a batch normalization operation. g and x These are the upsampled feature map in the decoder and the feature map from the encoder in the skip connections, respectively. The fourth decoded feature map is obtained by upsampling the third decoded feature map by a factor of two. The image segmentation result is obtained by performing a matrix addition operation on the results of upsampling the first decoded feature map by eight times, the second decoded feature map by four times, the third decoded feature map by two times, and the fourth decoded feature map.

Images