Medical image segmentation method and system based on self-attention and information bottleneck

Abstract

The application provides a medical image segmentation method and system based on self-attention and information bottleneck, and relates to the field of medical image segmentation. The method comprises the following steps: obtaining an original medical image; extracting features to obtain a preliminary feature map, and inputting the preliminary feature map into a MultiIB-Transformer coding part and a skip connection SC-MRF part; in the MultiIB-Transformer coding part, a first IB block is used to compress irrelevant features, and a second IB block is used to compress irrelevant features again to obtain a first output feature map; in the skip connection SC-MRF part, semantic information in a low-resolution feature map is transferred to a high-resolution feature map to obtain a second output feature map; and the first output feature map and the second output feature map are spliced, and the spliced feature map is input into a pre-trained prediction network for image segmentation. The application reduces the parameter quantity of the model, improves the sensitivity of the model to key features, compresses noise, and improves the accuracy of model segmentation.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of medical image segmentation, and particularly relates to a medical image segmentation method and system based on self-attention and information bottleneck. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] When doctors diagnose and operate diseases, accurate medical image segmentation results are essential tools. With the continuous development of computer vision technology, many researchers are working hard to propose new neural network models to achieve faster and more accurate medical image segmentation. Among them, the U-Net model is a representative convolutional neural network model that has shown good performance on some popular medical image datasets. With the successful application of the U-Net model on many medical image datasets, many researchers have begun to improve the U-Net model. For example, Res-UNet uses residual blocks to accelerate convergence and increase network depth, thereby improving segmentation results. Attention-UNet introduces attention mechanisms to improve the representation ability of feature maps, thereby further improving segmentation accuracy. 3D U-Net is a three-dimensional version of the U-Net model that can handle volumetric data. U-Net++ and Dense-UNet further optimize the segmentation results by increasing skip connections and dense connections, etc. In addition, other convolutional neural network models are constantly emerging, providing new possibilities for the development of medical image segmentation.

[0004] Although convolutional neural networks perform well in image processing, they have certain limitations in extracting global context information due to the size limitations of their convolution kernels. This makes it challenging to model long-range relationships. In contrast, the Transformer model excels in extracting global information. Therefore, some researchers in medical image analysis have begun to explore the application of Transformers in this field. TransUNet is the first model to successfully apply Transformers to medical image segmentation. It is based on the U-Net model and adds 12 Transformer layers in the encoder to alleviate the limitations of convolution operations. In this way, TransUNet can better capture global information and long-term dependencies, thereby improving the accuracy of medical image segmentation.

[0005] Although the TransUNet model provides a new idea for the development of medical image segmentation models, the parameter quantity of the TransUNet model is three times that of the U-Net model, and more computing resources are required for model training. The root cause of this problem is that researchers add more Transformer layers in the model, which leads to an explosion of the parameter quantity. Moreover, the data set related to medical images is usually small in scale, with only a few hundred or a few thousand samples. When the parameter of the model is large and the data set is small, the risk of overfitting increases. This is not only a problem of the TransUNet model, but also a problem faced by other medical image segmentation models based on the Transformer. In addition, the skip connection in the TransUNet ignores the semantic gap between the decoder and the encoder, which may reduce the accuracy of the prediction results of the model. SUMMARY

[0006] To overcome the deficiencies of the prior art described above, the present application provides a medical image segmentation method and system based on self-attention and information bottleneck, which further modifies the Transformer layer and the skip connection of the TransUNet model, aiming to reduce the parameter quantity of the model, improve the sensitivity of the model to key features, and compress noise, so as to improve the accuracy of the model for abdominal multi-organ segmentation and solve at least one technical problem existing in the background art.

[0007] To achieve the above-mentioned purpose, one or more embodiments of the present application provide the following technical solutions:

[0008] The present application provides a medical image segmentation method based on self-attention and information bottleneck.

[0009] The medical image segmentation method based on self-attention and information bottleneck comprises the following steps:

[0010] Obtaining an original medical image;

[0011] Inputting the original medical image into a medical image segmentation model based on Transformer and information bottleneck, extracting features of the original medical image by using an encoder module to obtain a preliminary feature map, and inputting the preliminary feature map into a MultiIB-Transformer encoding part and a skip connection SC-MRF part, respectively;

[0012] In the MultiIB-Transformer encoding part, global semantic information is obtained by using a multi-head self-attention mechanism, and then a first IB block connected after the multi-head self-attention mechanism is used to compress irrelevant features. The compressed features are sequentially input into a normalization layer and an MLP layer, and a second IB block connected after the MLP layer is used to compress irrelevant features again to obtain a first output feature map;

[0013] In the skip connection SC-MRF part, semantic information in the low-resolution feature map is transferred to the high-resolution feature map through a convolution operation to obtain a second output feature map;

[0014] The first output feature map and the second output feature map are input into a decoder module for splicing, and the spliced feature map is input into a pre-trained prediction network for image segmentation.

[0015] Preferably, the MultiIB-Transformer encoding part comprises, in sequence, a normalization layer, a multi-head self-attention mechanism, a first IB block, a normalization layer, an MLP layer, and a second IB block, and the first IB block and the second IB block are added in a residual form.

[0016] Preferably, the objective function of the IB block is:

[0017]

[0018] where β is a Lagrange multiplier, ranging from 0 to 1; E represents the mathematical expectation value, KL represents the kullback leibler divergence; q(y|z) is a distribution that can be learned from the network; r(z) is usually a standard Gaussian distribution.

[0019] Preferably, in the skip connection SC-MRF part, semantic information in the low-resolution feature map is transferred to the medium-resolution feature map through a first convolution operation, and semantic information in the medium-resolution feature map is transferred to the high-resolution feature map through a second convolution operation to obtain a second output feature map.

[0020] Preferably, the bottom layer feature map is first upsampled, and then the upsampled bottom layer feature map is concatenated with the intermediate layer feature map, and the two are fused using a first 1x1 convolution, and the feature map after the first fusion is added to the intermediate layer feature map.

[0021] The intermediate layer feature map to which the feature map after the first fusion is added is upsampled, and is connected with the top layer feature map, and a second 1x1 convolution is used to fuse the top layer feature map and the intermediate layer feature map to which the feature map after the first fusion is added, and the feature map after the second fusion is added to the top layer feature map.

[0022] Preferably, the mathematical expression of the intermediate layer feature map is:

[0023] F middle-layer =Conv 1×1 [Cat(F middle-layer ,Up(F bottom-layer ))]+F middle-layer

[0024] Preferably, the top layer feature map mathematical expression is:

[0025] F top-layer =Conv 1×1 [Cat(F top-layer , Up(F middle-layer ))]+F top-layer

[0026] The second aspect of the present application provides a medical image segmentation system based on self-attention and information bottleneck.

[0027] The medical image segmentation system based on self-attention and information bottleneck comprises:

[0028] An image acquisition module configured to acquire an original medical image;

[0029] An encoder feature extraction module configured to input the original medical image into a medical image segmentation model based on Transformer and information bottleneck, extract features of the original medical image by using an encoder module to obtain a preliminary feature map, and input the preliminary feature map into a MultilB-Transformer encoding part and a skip connection SC-MRF part respectively;

[0030] A MultilB-Transformer encoding module configured to, in the MultilB-Transformer encoding part, obtain global semantic information by using a multi-head self-attention mechanism, then compress irrelevant features by using a first IB block connected after the multi-head self-attention mechanism, input the compressed features into a normalization layer and an MLP layer in sequence, and compress irrelevant features again by using a second IB block connected after the MLP layer to obtain a first output feature map;

[0031] A skip connection SC-MRF module configured to, in the skip connection SC-MRF part, transfer semantic information in a low-resolution feature map to a high-resolution feature map by convolution operation to obtain a second output feature map;

[0032] A prediction module configured to input the first output feature map and the second output feature map into a decoder module for splicing, input the spliced feature map into a pre-trained prediction network, and perform image segmentation.

[0033] The third aspect of the present application provides a computer readable storage medium having a program stored thereon, and the program is executed by a processor to realize the steps in the medical image segmentation method based on self-attention and information bottleneck according to the first aspect of the present application.

[0034] The fourth aspect of the present application provides an electronic device, comprising a memory, a processor and a program stored in the memory and executable on the processor, wherein the processor implements the steps in the medical image segmentation method based on self-attention and information bottleneck according to the first aspect of the present application when executing the program.

[0035] The one or more technical solutions above have the following beneficial effects:

[0036] The present application provides a medical image segmentation method and system based on self-attention and information bottleneck, designs a MultiIB-Transformer module, adds multiple information bottleneck (IB) blocks in a Transformer layer, and through multiple compressions, can help the model to filter out as much irrelevant features as possible, only keep relevant features, and improve robustness. It is the first medical image segmentation model that combines multiple information bottleneck blocks with a Transformer layer, improves the sensitivity of the model to key features, compresses noise, and improves the accuracy of the model for abdominal multi-organ segmentation.

[0037] In addition, the present application designs a lightweight multi-resolution fusion module, which only needs two 1x1 convolution operations to help the high-resolution feature map obtain rich semantic information, thereby well relieving the semantic gap.

[0038] The advantages of the additional aspects of the present application will be partially given in the following description, partially will become obvious from the following description, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and the illustrative embodiments of the present application and their description serve the purpose of explaining the present application. They do not constitute an inappropriate limitation on the present application.

[0040] Figure 1 The method flowchart of the first embodiment.

[0041] Figure 2 The functional principle block diagram of the MultiIB-Transformer module of the first embodiment.

[0042] Figure 3 The functional principle block diagram of the SC-MRF module of the first embodiment. DETAILED DESCRIPTION

[0043] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the present application belongs.

[0044] It should be noted that the terms used herein are only intended to describe specific embodiments and are not intended to limit the exemplary embodiments according to the present application.

[0045] In the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0046] Embodiment one

[0047] The embodiment discloses a medical image segmentation method based on self-attention and information bottleneck.

[0048] As shown in the medical image segmentation method based on self-attention and information bottleneck, comprising the following steps: Figure 1 Obtaining an original medical image;

[0049] Inputting the original medical image into a medical image segmentation model based on Transformer and information bottleneck, extracting features of the original medical image by using an encoder module to obtain a preliminary feature map, and inputting the preliminary feature map into a MultiIB-Transformer encoding part and a skip connection SC-MRF part respectively;

[0050] In the MultiIB-Transformer encoding part, global semantic information is obtained by using a multi-head self-attention mechanism, and then a first IB block connected after the multi-head self-attention mechanism is used to compress irrelevant features, and the compressed features are sequentially input into a normalization layer and an MLP layer, and a second IB block connected after the MLP layer is used to compress irrelevant features again to obtain a first output feature map;

[0051] In the skip connection SC-MRF part, semantic information in a low-resolution feature map is transferred to a high-resolution feature map by convolution operation to obtain a second output feature map;

[0052] The first output feature map and the second output feature map are input into a decoder module for splicing, and the spliced feature map is input into a pre-trained prediction network for image segmentation.

[0053] Specifically, the embodiment provides a model for medical image segmentation based on Transformer and information bottleneck, which comprises:

[0054] The CNN encoding part performs three times of down-sampling on the input abdominal CT image, and the resolutions are 1 / 2, 1 / 4 and 1 / 8 of the input image respectively, the purpose is to extract features and retain high-resolution spatial information.

[0055]

[0056] ​The MultiIB-Transformer encoding part decomposes the feature map obtained from the previous step through three downsampling steps into 16×16 blocks, forming a sequence of image blocks. This sequence is then positionally encoded, and global semantic information is obtained through the multi-head self-attention mechanism provided by the Transformer for feature extraction. For example... Figure 2 As shown, we choose to add IB blocks at two locations in the Transformer. Since Multi-Head Self-Attention (MSA) is a crucial part of feature extraction in the Transformer layer, we choose to add an IB block after MSA to compress irrelevant features. The second location is after the MLP layer. Multiple compressions help the model filter out irrelevant features as much as possible, retaining only relevant features and improving robustness. To prevent relevant features from being compressed during training, which could lead to a decrease in model segmentation accuracy, we choose to add these two IB blocks as residuals.

[0057] The SC-MRF section with skip connections has the following structure: Figure 3 As shown, semantic information in low-resolution feature maps is continuously transferred to high-resolution feature maps. This helps high-resolution feature maps to simultaneously possess both texture and semantic information. It can make the segmentation of organ or lesion boundaries more accurate and alleviate semantic gaps.

[0058] In the decoding part, the feature maps obtained by downsampling the CNN encoding block three times are concatenated with the feature maps of the same resolution size in the decoding part. The purpose is to make up for the spatial detail information lost during the encoding process and help the decoding block achieve a fine segmentation result. Then, upsampling is performed.

[0059] The prediction part is used to segment the input medical image using a pre-trained prediction network.

[0060] Our research shows that adding more Transformer layers does not significantly improve segmentation accuracy. On the contrary, we found that even adding just one Transformer layer yields better segmentation results than TransUNet, thus significantly reducing the number of parameters.

[0061] In addition, to reduce the risk of model overfitting due to the small number of samples in the medical image dataset, we propose the MultiIB-Transformer module. This module combines the information bottleneck (IB) theory with the Transformer. The function of the information bottleneck theory is to help the model compress noise during training and reduce the risk of overfitting. We choose to add IB blocks in two positions in the Transformer. Since the multi-head self-attention mechanism (MSA) is the key part of feature extraction in the Transformer layer, we choose to add an IB block after the MSA to compress irrelevant features. The second position is after the MLP layer. By multiple compression, it can help the model filter out as many irrelevant features as possible, only keep relevant features, and improve robustness. To prevent relevant features from being compressed during training and cause the model segmentation accuracy to drop, we choose to add these two IB blocks in the form of a residual.

[0062] IB theory suggests that imposing constraints on the model can make the model only learn key features, compress irrelevant features, and reduce the risk of overfitting. The constraint function is:

[0063] L IB (Z) = I(Y; Z) - βI(X; Z)

[0064] Z can be regarded as a latent feature, Y is the output, and X is the input. β is a Lagrange multiplier, ranging from [0, 1], used to control the compression rate. I(X; Z) and I(Y; Z) represent mutual information.

[0065]

[0066] In the above formula, p(y) represents the probability, p(y,z) represents the joint probability, and p(y|z) represents the conditional probability. However, in reality, data is distributed in high dimensions, which makes it very difficult to calculate p(y|z) in the above formula. To solve this problem, I use the idea of variational information bottleneck (VIB). The specific implementation method is to approximate p(y|z) with q(y|z) and approximate p(z) with r(z). Since the Kullback Leibler Divergence between two distributions is always positive, an approximate solution can be obtained.

[0067] I(Y; Z) - βI(X; Z) ≥ ∫dxdydzp(x)p(y|z)p(z|x)logq(y|z)

[0068]

[0069] By using the empirical distribution The approximate result p(x, y) = p(y|x)p(x) is obtained.

[0070]

[0071] In the above formula, N represents the number of samples, r(z) represents the probability, p(z|x n ) and q(y n |z) represent conditional probabilities. Then Monte Carlo sampling is used, and it is assumed that p(z|x) obeys a Gaussian distribution, the mean and variance of which can be learned in the network. Thus, the final objective function can be obtained:

[0072]

[0073] where E represents the mathematical expectation value, and KL represents the kullback leibler divergence. p Φ is considered to be a parameterized multivariate Gaussian encoder. q(y|z) is a distribution that can be learned from the network. r(z) is usually selected as a standard Gaussian distribution.

[0074] By applying the IB theory, overfitting can be effectively prevented, and the robustness of the model can be improved. This is of great significance to the model trained on a medical image dataset with very few samples.

[0075] We also propose a skip connection based on multi-resolution fusion, called SC-MRF module. This new skip connection can help high-resolution feature maps obtain more rich semantic information, thus well relieving the semantic gap.

[0076] First, the bottom layer feature map is up-sampled, and then concatenated with the intermediate layer feature map. A 1x1 convolution is used for feature fusion, and the number of channels is kept consistent with the intermediate layer feature map. The fused feature map is added to the intermediate layer feature map. The intermediate layer feature map enriches the semantic information while keeping the resolution unchanged. The mathematical expression is:

[0077] F middle-layer =Conv 1×1 [Cat(F middle-layer ,Up(F bottom-layer ))]+F middle-layer

[0078] In the above formula, F middle-layer and F bottom-layer represent the intermediate layer and bottom layer feature maps respectively, Conv 1×1 represents a 1x1 convolution operation, Cat represents a concatenation operation on the feature map, and Up represents up-sampling of the feature map.

[0079] Then the obtained intermediate layer feature map is up-sampled and connected with the top layer feature map. A 1x1 convolution is used to fuse the feature maps, and the channel number of the fused feature map is the same as that of the top layer feature map. Then it is added to the top layer feature map. Therefore, the top layer feature map obtains the semantic information provided by the intermediate layer and bottom layer feature maps while keeping the resolution unchanged. The mathematical expression is:

[0080] F top-layer =Conv 1×1 [Cat(F top-layer ,Up(F mniddle-layer ))]+F top-layer

[0081] In the above formula, F top-layer and F middle-layer represent the top layer and intermediate feature maps respectively, Conv 1×1 represents a 1x1 convolution operation, Cat represents a feature map splicing operation, and Up represents an up-sampling operation on the feature map.

[0082] The performance of the model proposed in this embodiment will be verified through experiments as follows:

[0083] (1) In order to better illustrate the excellent segmentation performance of the MultilB-TransUNet model proposed by us, we made a comparative experiment with the existing advanced segmentation model, and the results are shown in Tables 1 and 2.

[0084] Hereinafter, the data set used and the evaluation index commonly used in medical image segmentation will be briefly introduced.

[0085] Synapse multi-organ segmentation data set: This data set consists of 30 CT scans of 8 abdominal organs, including left kidney, right kidney, aorta, spleen, gallbladder, liver, spleen, pancreas and stomach. A total of 3779 axial enhanced abdominal clinical CT images were obtained. 18 samples (2212 axial slices) were randomly selected as the training set, and the remaining 12 samples (1567 axial slices) were used as the test set.

[0086] Breast UltraSound lmages data set: This data set consists of 780 breast cancer ultrasound images and corresponding segmentation images of 600 female patients. Among them, 210 are malignant, 437 are benign, and 133 are normal. We use this data set to prove that MultilB-TransUNet still performs well on ultrasound images. We only use benign and malignant images, a total of 647. And the data set is randomly divided into three 80-20 splits.

[0087] Evaluation index: DSC evaluation index can judge the similarity between two sets, and its value range is 0 to 1. When the value is 0, the segmentation effect is the worst, and when the value is 1, the segmentation effect is the best. HD evaluation index is to calculate the distance between two sets, and the smaller the value, the higher the similarity between the two sets. IoU is a measure of the positional accuracy of the segmentation result.

[0088] Table 1

[0089]

[0090] From Table 1, we can see that the DSC and HD evaluation indexes of our MultiIB-TransUNet are 81.83% and 20.04mm. Compared with the TransUNet model, the improvement ranges are 4.35% and 11.65mm respectively. In addition, compared with the 93.23M parameters of the TransUNet model, the parameters of our MultiIB-TransUNet model are only 22.30M, which is significantly reduced. This shows that the MultiIB-TransUNet model is more competitive than TransUNet in overall organ segmentation, edge segmentation and computing resource occupation.

[0091] The segmentation results of each organ are also better than TransUNet. Among them, the accuracy of the segmentation results of the two large organs of stomach and spleen is improved by 7.62% and 4.75%. For the segmentation of small organs such as pancreas, kidney (right) and gallbladder, the improvement of segmentation accuracy is more obvious, increasing by 11.83%, 4.37% and 3.12% respectively. This shows that our MultiIB-TransUNet model has more advantages in learning the features of small organs.

[0092] Compared with UCTransNet, AFTer-UNet, CTC-Net and TransUNet+ models, our model achieves better results in DSC, HD and Param three evaluation indexes. Compared with HiFormer-L model, our MultiIB-TransUNet model is more competitive than HiFormer-L model in DSC and Param two evaluation indexes, although it is slightly insufficient in HD evaluation index.

[0093] Table 2

[0094]

[0095] From Table 2, it can be seen that compared with TransUNet, the segmentation results of MultiIB-TransUNet on F1 score and IoU are improved by 1.48% and 0.83% respectively. Compared with the newly proposed UNeXt, these two evaluation indicators are improved by 1.41% and 0.8%. The results show that MultiIB-TransUNet has good generalization ability.

[0096] (ii) In addition, we use the Synapse multi-organ segmentation dataset to study the impact of adding residual information bottleneck (IB) blocks (MultiIB-Transformer module) at different positions of the Transformer layer on the segmentation performance of the model, and the impact of the residual path on the segmentation performance of the model. The contents of our experiments are as follows:

[0097] (a) Transformer layer without information bottleneck block (baseline).

[0098] (b) Information bottleneck block is added after MSA of the Transformer layer.

[0099] (c) Information bottleneck block is added after MLP of the Transformer layer.

[0100] (d) Add non-residual information bottleneck block after MSA and MLP of the Transformer layer.

[0101] (e) Add residual information bottleneck block after MSA and MLP of the Transformer layer.

[0102] As can be seen from Table 3, adding IB block after MSA can improve the segmentation results. This is because MSA is a key part of Transformer feature extraction. Inputting the extracted features into the IB block can compress noise and help the model learn more relevant features. Adding IB block after MLP also has a positive impact on segmentation results. This is because MLP is located at the end of the Transformer layer, and adding IB block after it is equivalent to performing complete filtering on the features extracted by the entire Transformer layer to compress irrelevant features. When adding IB block after both MSA and MLP, the segmentation accuracy will further improve. This is because the ability of an IB block to compress noise is limited, and irrelevant features can be better compressed through multiple IB blocks.

[0103] Table 3

[0104]

[0105] In addition, the IB block with residual can improve the model segmentation result better than the IB block without residual. This is because, during the training process, the IB block may compress the relevant features, resulting in a decrease in the accuracy of the model segmentation result. After adding the residual path, it can help the model to compress as much as possible the irrelevant features and retain the relevant features during the training process.

[0106] (III) To verify the effectiveness of the SC-MRF module, we designed a comparative experiment. The experimental objects include:

[0107] (I) The original TransUNet model.

[0108] (II) The TransUNet model with only one Transformer layer (baseline model).

[0109] (III) Adding the SC-MRF module to the baseline model.

[0110] (IV) Adding the SC-MRF and MultiIB-Transformer modules to the baseline model, which is our proposed MultiIB-TransUNet model.

[0111] As can be seen from Table 4, our three models are superior to the TransUNet model in terms of HD and DSC evaluation indicators and the number of parameters. Compared with the baseline, after adding the SC-MRF module, better segmentation results are obtained. However, the number of parameters does not increase significantly, which means that the SC-MRF module we proposed is a simple and effective multi-resolution fusion module. Although the number of parameters of the MultiIB-TransUNet model increases compared with the baseline. However, the DSC and HD evaluation indicators are significantly improved, which means that when the SC-MRF module is combined with the MultiIB-Transformer module, the segmentation performance can be further improved.

[0112] Table 4

[0113]

[0114] Example Two

[0115] The embodiment discloses a medical image segmentation system based on self-attention and information bottleneck.

[0116] The medical image segmentation system based on self-attention and information bottleneck comprises:

[0117] The image acquisition module is configured to acquire an original medical image.

[0118] The encoder feature extraction module is configured to: input the original medical image into a medical image segmentation model based on a Transformer and an information bottleneck, extract features of the original medical image by using an encoder module to obtain a preliminary feature map, and input the preliminary feature map into a MultiIB-Transformer encoding part and a skip connection SC-MRF part respectively.

[0119] The MultiIB-Transformer encoding module is configured to: in the MultiIB-Transformer encoding part, obtain global semantic information by using a multi-head self-attention mechanism, then compress irrelevant features by using a first IB block connected after the multi-head self-attention mechanism, input the compressed features into a normalization layer and an MLP layer in sequence, and compress irrelevant features again by using a second IB block connected after the MLP layer to obtain a first output feature map.

[0120] The skip connection SC-MRF module is configured to: in the skip connection SC-MRF part, transfer semantic information in a low-resolution feature map to a high-resolution feature map by convolution operation to obtain a second output feature map.

[0121] The prediction module is configured to: input the first output feature map and the second output feature map into a decoder module for splicing, and input the spliced feature map into a pre-trained prediction network for image segmentation.

[0122] Embodiment three

[0123] An object of the embodiment is to provide a computer-readable storage medium.

[0124] A computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps in the medical image segmentation method based on self-attention and information bottleneck according to Embodiment 1 of the present disclosure.

[0125] Embodiment four

[0126] An object of the embodiment is to provide an electronic device.

[0127] An electronic device includes a memory, a processor, and a program stored on the memory and executable on the processor, and the processor executes the program to implement the steps in the medical image segmentation method based on self-attention and information bottleneck according to Embodiment 1 of the present disclosure.

[0128] The steps involved in the apparatuses of the above embodiments two, three and four correspond to the method of embodiment one, and the specific implementation can refer to the relevant description of embodiment one. The term "computer readable storage medium" should be understood as including a single medium or multiple media of one or more instruction sets; it should also be understood as including any medium capable of storing, encoding or carrying a set of instructions for execution by a processor and causing the processor to perform any of the methods in the present application.

[0129] Those skilled in the art should understand that each module or step of the present application described above can be realized by a general computer device, alternatively, they can be realized by program codes executable by a computing device, so that they can be stored in a storage device for execution by a computing device, or they can be respectively manufactured into each integrated circuit module, or a plurality of modules or steps among them can be manufactured into a single integrated circuit module to realize. The present application is not limited to any specific combination of hardware and software.

[0130] Although the specific embodiments of the present application are described above in combination with the drawings, it is not a limitation on the protection scope of the present application, and those skilled in the art should understand that various modifications or changes made by those skilled in the art on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A medical image segmentation method based on self-attention and information bottleneck, characterized in that, Includes the following steps: Acquire raw medical images; The original medical image is input into a medical image segmentation model based on Transformer and information bottleneck. The encoder module is used to extract the features of the original medical image to obtain a preliminary feature map. The preliminary feature map is then input into the MultiIB-Transformer encoding part and the skip connection SC-MRF part, respectively. In the MultiIB-Transformer encoding part, the multi-head self-attention mechanism is used to obtain global semantic information. Then, the first IB block connected after the multi-head self-attention mechanism is used to compress irrelevant features. The compressed features are sequentially input into the normalization layer and the MLP layer. The second IB block connected after the MLP layer is used to compress irrelevant features again to obtain the first output feature map. In the skip connection SC-MRF part, semantic information in the low-resolution feature map is transferred to the high-resolution feature map through convolution operation to obtain the second output feature map; The first and second output feature maps are input into the decoder module for concatenation. The concatenated feature map is then input into the pre-trained prediction network for image segmentation.

2. The medical image segmentation method based on self-attention and information bottleneck as described in claim 1, characterized in that, The MultiIB-Transformer encoding part includes a normalization layer, a multi-head self-attention mechanism, a first IB block, a normalization layer, an MLP layer, and a second IB block connected in sequence. The first IB block and the second IB block are added in the form of residuals.

3. The medical image segmentation method based on self-attention and information bottleneck as described in claim 2, characterized in that, The objective function of the IB block is: Where β is a Lagrange multiplier in the range [0,1]; E represents the expected value; KL represents the Kullbackleibler divergence; q(y|z) is a distribution that can be learned from the network; and r(z) is a standard Gaussian distribution.

4. The medical image segmentation method based on self-attention and information bottleneck as described in claim 1, characterized in that, In the skip connection SC-MRF part, the semantic information in the low-resolution feature map is transferred to the medium-resolution feature map through the first convolution operation, and then the semantic information in the medium-resolution feature map is transferred to the high-resolution feature map through the second convolution operation to obtain the second output feature map.

5. The medical image segmentation method based on self-attention and information bottleneck as described in claim 4, characterized in that, First, the bottom feature map is upsampled. Then, the upsampled bottom feature map is concatenated with the intermediate feature map. The two are then fused using the first 1×1 convolution. The feature map after the first fusion is then added to the intermediate feature map. The intermediate layer feature map with the first fusion feature map is upsampled and concatenated with the top layer feature map. A second 1×1 convolution is used to fuse the top layer feature map and the intermediate layer feature map with the first fusion feature map. The feature map with the second fusion is then added to the top layer feature map.

6. The medical image segmentation method based on self-attention and information bottleneck as described in claim 4, characterized in that, The mathematical expression for the intermediate layer feature map is: 。 7. The medical image segmentation method based on self-attention and information bottleneck as described in claim 4, characterized in that, The mathematical expression for the top-level feature map is: 。 8. A medical image segmentation system based on self-attention and information bottleneck, characterized in that: include: The image acquisition module is configured to acquire raw medical images; The encoder feature extraction module is configured to: input the original medical image into the medical image segmentation model based on Transformer and information bottleneck, extract the features of the original medical image using the encoder module to obtain a preliminary feature map, and input the preliminary feature map into the MultiIB-Transformer encoding part and the skip connection SC-MRF part respectively; The MultiIB-Transformer encoding module is configured to: in the MultiIB-Transformer encoding part, use the multi-head self-attention mechanism to obtain global semantic information, and then use the first IB block connected after the multi-head self-attention mechanism to compress irrelevant features. The compressed features are then input into the normalization layer and the MLP layer in sequence. The second IB block connected after the MLP layer is used to compress irrelevant features again to obtain the first output feature map. The skip connection SC-MRF module is configured to: in the skip connection SC-MRF part, transfer the semantic information in the low-resolution feature map to the high-resolution feature map through convolution operation to obtain the second output feature map; The prediction module is configured to: input the first output feature map and the second output feature map into the decoder module for concatenation, and input the concatenated feature map into the pre-trained prediction network for image segmentation.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by a processor, the program implements the steps in the medical image segmentation method based on self-attention and information bottleneck as described in any one of claims 1-7.

10. An electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the medical image segmentation method based on self-attention and information bottleneck as described in any one of claims 1-7.

Images