Medical image segmentation method based on multiple extended recursive attention

Abstract

A medical image segmentation method based on multi-extended recursive attention belongs to the technical field of image segmentation, and comprises the following steps: a coding part: coding input features by a multi-extended recursive attention block and a feature merging layer; in the decoding part, firstly, the feature data of the multi-extended recursive attention block encoding part are input into a decoder; secondly, the decoder gradually recovers the original resolution of the image through up-sampling, and semantic information obtained through up-sampling and features output by the encoder of the first part are subjected to feature interaction through jump connection; thirdly, inputting the interacted features into the decoding part of the recursive attention block, and restoring the original information of the image to the maximum extent; and finally, through a linear mapping layer, obtaining a pixel-level segmentation prediction result according to the extracted image feature marks. By adopting the method, the model can show better overall performance and robustness in a segmentation task, and the method has important significance in practical application in the fields of medical images and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of image segmentation, and in particular relates to a medical image segmentation method based on multi-extended recursive attention. Background Art

[0002] Medical image segmentation plays a vital role in modern medicine, especially in the fields of clinical surgery, disease diagnosis, treatment planning, and clinical quantification. The goal of medical image segmentation is to accurately delineate "interesting" areas such as abnormal regions and organs so that the boundaries of lesions can be quickly and accurately identified and the extent of lesions can be accurately assessed. The importance of this process lies in the fact that even small segmentation errors may lead to adverse outcomes in clinical practice and increased risks for subsequent computer-aided diagnosis. In addition, manually drawing the location of lesions and their boundaries is time-consuming and sometimes even impractical in various imaging modalities. The final segmentation results may also be affected by the clinician's personal experience, preferences, and expertise. Therefore, the use of accurate and reliable image segmentation technology is key to ensuring accuracy and efficiency in clinical practice.

[0003] With the rapid development of deep learning technology, many methods based on convolutional neural networks (CNNs) have been proposed to handle medical image segmentation tasks. Among them, U-Net is a classic architecture that realizes feature extraction and reconstruction through the connection between the encoder and the decoder, and has been proven to perform well in segmentation tasks. In addition, a series of improved versions have emerged, such as U-Net++, Attention U-Net, V-Net, etc., which have shown good performance on different medical datasets. However, although these CNN-based methods perform well in learning semantic information, due to the local characteristics of the convolution operation, existing technologies often find it difficult to capture global context information, especially when long-range dependencies are involved. Some studies have tried to improve this problem by introducing convolutional layers, self-attention mechanisms, and image pyramids, but these methods have not made breakthrough progress in significantly improving the ability to model long-range dependencies.

[0004] With the introduction of Visual Transformer (ViT) in recent years, people have begun to re-examine the design of segmentation models. ViT has attracted much attention for its excellent performance and scalability, but its attention mechanism may not be able to handle objects of interest with different shapes and sizes in medical images. Especially in shallow networks, the existing self-attention mechanism still has certain limitations in capturing the characteristics of target objects. Summary of the invention

[0005] The main purpose of the present invention is to overcome the shortcomings of the prior art. The present invention uses BRAU-Net++ as a benchmark segmentation and provides a medical image segmentation method based on multi-extended recursive attention. Multi-extended recursive attention is used to enhance the features of target objects with different appearances, and a feature interaction module is used to fuse low-level and high-level features to improve performance.

[0006] The design concept of the present invention is as follows: first, the spatial features of different receptive field ranges are captured by using deep separable convolutional layers with different expansion rates to capture spatial features of a series of receptive field sizes, so that the difference between the segmented object and the background can be modeled in more detail, and its ability to distinguish different organs is enhanced; second, when calculating the similarity graph, multi-scale context is introduced, and a recursive mechanism is used to enhance the representation ability of the features, so as to more effectively capture the features of target objects with different appearances; third, a feature interaction module is introduced to reorganize the jump connection to better fuse the output features of the encoder and decoder. In short, this fused feature not only contains the information of low-level and high-level features, but also can effectively improve the segmentation performance.

[0007] The present invention is achieved through the following technical solutions: A medical image segmentation method based on multi-extended recursive attention, comprising the following steps: S1. Select a medical image segmentation dataset and preprocess the training set in the dataset, including the following steps: S1-1, preprocess the medical image segmentation dataset, and then resize all preprocessed images to 224×224 pixel images as input images; S1-2, divide the input image obtained in step S1-1 into overlapping small blocks, obtain feature maps, and project the feature dimension of each feature map to an arbitrary dimension C through two 3×3 convolutional layers; S2, the encoder consists of several multi-extended recursive attention blocks and several feature merging layers. First, the feature map is encoded through several multi-extended recursive attention blocks, and then enters the feature merging layer to reduce the input spatial resolution, and multi-scale information and global information features are extracted from the feature map, including the following steps: S2-1, input the feature map preprocessed in step S1 into the continuous multi-expanded recursive attention block to perform representation learning; In the multi-expanded recurrent attention block: S2-1-1. Input features along the channel dimension Divided into four different heads, generating , R is the color channel, R=3, H, W are the spatial dimensions, and C is the feature channel; then, each head undergoes four depth-separable dilated convolutions with different dilation rates d1, d2, d3, and d4, respectively, to obtain , the formula is as follows: ; (1) In formula (1), They represent the features obtained by applying depthwise separable dilated convolution to the heads corresponding to d1 to d4, where d1=1, d2=2, d3=3, and d4=4; the operation DDWConv() represents depthwise separable dilated convolution; S2-1-2. Recursive attention includes a recursive process and multi-scale attention. First, the features after the depthwise separable dilated convolution are input into the multi-scale attention, which uses 1 × 1 convolution to apply linear projection. Then, three convolutions d5, d6, and d7 with different dilation rates but shared kernels are applied to capture the multi-scale context, and the group number is set to the feature channel number, where d5=1, d6=3, and d7=5. Finally, the parallel features of different scales are weighted and summed, and the SiLU self-calibration mechanism is used to determine the weight of each scale by the activation strength. The formula is as follows: ; (2) In formula (2), ; (3) ; (4) In formula (2) to formula (4), is the feature map, is the weight value; H and W are spatial dimensions, C is the feature channel, K is the convolution kernel size, r is the convolution kernel expansion rate, and g is the number of convolution kernel groups; S2-1-3, after the multi-scale attention is completed, it enters the recursive attention, and then increases the depth of the recursive attention without increasing the parameters, and then forms the recursive attention with the multi-scale attention; the formula of the recursive attention is as follows: ; (5) ; (6) ; (7) In formula (5) to (7), t is the step length, is the hidden state; MDA is a nonlinear activation function, the initial hidden state is a linear function combining the input and hidden state, is the projection weight; S2-2, the input features of the encoder are sent to the feature merging layer with a 3×3 convolutional layer for downsampling after passing through multiple multi-expanded recursive attention blocks, reducing the input spatial resolution while increasing the number of channels, and performing 2x downsampling at the same time, reducing the number of tags by half and increasing the feature dimension by 2x; S3. The decoder is composed of multiple multi-extended recursive attention blocks and several upsampling layers. The features output by the multi-extended recursive attention blocks in the encoder are connected through jumps, and then interact with the output of the extended recursive attention blocks in the decoder to decode the feature map information, including the following steps: S3-1, in the decoder, the feature map after the multi-expanded recursive attention block in the encoder of step S2 is input into the upsampling layer, that is, the resolution of the feature map is increased by 2 times, and the dimension of the feature map is reduced by half, and it is reshaped into a feature map with higher resolution; S3-2. Let the output of the multi-extended recursive attention block in the encoder be , the output of the multi-expanded recurrent attention block in the decoder is , construct a skip connection by connecting the outputs of the encoder and decoder, and connect the higher resolution feature maps in the upsampling layer through the skip connection; S3-2-1. Use global average pooling to aggregate spatial information into a channel-level global feature and generate ; Then, one-dimensional convolution with different parameters is used to achieve x 1 and x 2 feature embedding, the formula is as follows: ; (8) ; (9) ; (10) ; (11) In equations (8) to (11), C1D1 and C1D2 represent two different 1×1 convolutional layers; S3-2-2. Regarding x 1 and x 2's internal features, the attention weight is obtained through the Sigmoid activation function: ; (12) ; (13) The fine features are obtained by multiplying the input features by the attention weights: ; (14) ; (15) In formula (14) to formula (15), Indicates element-related products; S3-2-3. For x 1 and x 2 feature interaction, x 1 Embed and x 2 Embed Connect to form features , use the Softmax activation function to generate the attention weights of the connection dimension , which represents the dynamic guidance of the network to the interactive correlation relationship, and its formula is: ; (16) x 1 and x The fine features of the interaction between 2 are obtained as follows: ; (17) ; (18) In formula (17) to formula (18), Represent the first row vector and the second row vector of W respectively; S3-3. x 1 Internal features and x 1. x Add the features between 2 to get the fusion feature x 3; S4, segmentation prediction: The last layer of the decoder performs 4x upsampling and outputs a feature map with a resolution of H×W. The upsampled feature labels are passed to the linear mapping layer, and the final pixel-level segmentation prediction results are output to complete the medical image segmentation based on multi-extended recursive attention.

[0008] The beneficial effects of the present invention are: With respect to the shortcomings of the prior art, the present invention uses the BRAU-Net++ framework as a basis to design a multi-extended recursive attention to improve the framework. In addition, a new skip connection is introduced to improve the performance. The introduction of this multi-extended recursive attention method brings significant advantages and positive effects to the segmentation model design: 1) By using deep separable convolutional layers with different dilation rates to capture spatial features of different receptive fields, this method can model the difference between segmented objects and background in more detail. This fine-grained modeling enables the model to more accurately distinguish different organs or objects, thereby enhancing the discriminative ability of the segmentation model; 2) The introduction of multi-scale context and recursive mechanism to calculate similarity graphs helps to enhance the representation ability of features. In this way, the model can more effectively capture the features of target objects with different appearances, improving the model's ability to segment objects of interest of various shapes and sizes; 3) The feature interaction module is introduced to reorganize the jump connection, which effectively fuses the output features of the encoder and decoder. This fused feature not only contains the information of low-level and high-level features, but also can effectively improve the segmentation performance.

[0009] In summary, the use of the present invention can enable the model to show better overall performance and robustness in segmentation tasks, which is of great significance for practical applications in fields such as medical imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In the description of the drawings of the invention patent, in order to more clearly show the embodiments or technical solutions in the prior art, we will briefly introduce the necessary drawings. These drawings only represent some embodiments of the invention, and those skilled in the art may still obtain other drawings based on these drawings to realize more technical solutions without making creative efforts.

[0011] Figure 1 It is the overall flow chart of the present invention; Figure 2 The flowchart of the improved multi-expanded recursive attention module of the present invention; Figure 3 A flowchart of the multi-scale attention inside the improved multi-extended recursive attention module of the present invention; Figure 4 This is a flow chart of the feature interaction module of the jump connection in the present invention; Figure 5 A comparison chart of the segmentation effect of the present invention and other existing segmentation algorithms on the general data set Synapse; Figure 6 This is a comparison chart of the effects of the present invention and other segmentation algorithms on the ISIC-2018 dataset. DETAILED DESCRIPTION

[0012] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0013] The present invention proposes a medical image segmentation method based on multi-extended recursive attention, such as Figure 1 As shown, it mainly includes two parts: The first part, the encoding part, encodes the input features by multiple extended recursive attention blocks and feature merging layers: first, the public medical image segmentation dataset is preprocessed; second, the input image is divided into overlapping small blocks, and the feature dimension of each small block is projected to the required dimension through the convolution layer; third, the feature map is input into the encoding part of the multiple extended recursive attention blocks for global feature extraction; finally, downsampling is performed through the feature merging layer; The second part, the decoding part, firstly, inputs the feature data of the encoding part of the multi-extended recursive attention block into the decoder; secondly, the decoder gradually restores the original resolution of the image by upsampling, and interacts the semantic information obtained by upsampling with the features of the output of the encoder in the first part through jump connections to obtain more low-level and high-level feature information; thirdly, the interacted features are input into the decoding part of the recursive attention block to restore the original information of the image to the greatest extent; finally, through a linear mapping layer, the pixel-level segmentation prediction result is obtained according to the extracted image feature labels.

[0014] like Figures 1 to 4 A medical image segmentation method based on multi-extended recursive attention is shown, comprising the following steps: A medical image segmentation method based on multi-extended recursive attention, comprising the following steps: S1. Select a medical image segmentation dataset. In this specific implementation, the Synapse dataset is selected: a dataset for multi-organ segmentation, which comes from the MICCAI 2015 Multi-Atlas Abdomen Labeling Challenge. The training set contains 2212 axial enhanced abdominal CT images. The training set in the dataset is preprocessed, including the following steps: S1-1. Preprocess the medical image segmentation dataset (including data format conversion, cropping, resampling and standardization in sequence), and then resize all preprocessed images to 224×224 pixel images as input images; S1-2, divide the input image obtained in step S1-1 into overlapping small blocks, obtain feature maps, and project the feature dimension of each feature map to an arbitrary dimension C through two 3×3 convolutional layers; S2, the encoder consists of several multi-extended recursive attention blocks and several feature merging layers. First, the feature map is encoded through several multi-extended recursive attention blocks, and then enters the feature merging layer to reduce the input spatial resolution, and multi-scale information and global information features are extracted from the feature map, including the following steps: S2-1, input the feature map preprocessed in step S1 into the continuous multi-expanded recursive attention block to perform representation learning; In the multi-expanded recurrent attention block: S2-1-1. Input features along the channel dimension Divided into four different heads, generating , R is the color channel, R=3, H, W are the spatial dimensions, and C is the feature channel; then, each head undergoes four depth-separable dilated convolutions with different dilation rates d1, d2, d3, and d4, respectively, to obtain , the formula is as follows: ; (1) In formula (1), They represent the features obtained by applying depthwise separable dilated convolution to the heads corresponding to d1 to d4, where d1=1, d2=2, d3=3, and d4=4; the operation DDWConv() represents depthwise separable dilated convolution; S2-1-2. Recursive attention includes a recursive process and multi-scale attention. First, the features after the depthwise separable dilated convolution are input into the multi-scale attention, which uses 1 × 1 convolution to apply linear projection. Then, three convolutions d5, d6, and d7 with different dilation rates but shared kernels are applied to capture the multi-scale context, and the group number is set to the feature channel number, where d5=1, d6=3, and d7=5. Finally, the parallel features of different scales are weighted and summed, and the SiLU self-calibration mechanism is used to determine the weight of each scale by the activation strength. The formula is as follows: ; (2) In formula (2), ; (3) ; (4) In formula (2) to formula (4), is the feature map, is the weight value; H and W are spatial dimensions, C is the feature channel, K is the convolution kernel size, r is the convolution kernel expansion rate, and g is the number of convolution kernel groups; S2-1-3, after the multi-scale attention is completed, it enters the recursive attention, and then increases the depth of the recursive attention without increasing the parameters, and then forms the recursive attention with the multi-scale attention; the formula of the recursive attention is as follows: ; (5) ; (6) ; (7) In formula (5) to (7), t is the step length, is the hidden state; MDA is a nonlinear activation function, the initial hidden state is a linear function combining the input and hidden state, is the projection weight; S2-2, the input features of the encoder are sent to the feature merging layer with a 3×3 convolutional layer for downsampling after passing through multiple multi-expanded recursive attention blocks, reducing the input spatial resolution while increasing the number of channels, and performing 2x downsampling at the same time, reducing the number of tags by half and increasing the feature dimension by 2x; S3. The decoder is composed of multiple multi-extended recursive attention blocks and several upsampling layers. The features output by the multi-extended recursive attention blocks in the encoder are connected through jumps, and then interact with the output of the extended recursive attention blocks in the decoder to decode the feature map information, including the following steps: S3-1, in the decoder, the feature map after the multi-expanded recursive attention block in the encoder of step S2 is input into the upsampling layer, that is, the resolution of the feature map is increased by 2 times, and the dimension of the feature map is reduced by half, and it is reshaped into a feature map with higher resolution; S3-2. Let the output of the multi-extended recursive attention block in the encoder be , the output of the multi-expanded recurrent attention block in the decoder is , construct a skip connection by connecting the outputs of the encoder and decoder, and connect the higher resolution feature maps in the upsampling layer through the skip connection; S3-2-1. Use global average pooling to aggregate spatial information into a channel-level global feature and generate ; Then, one-dimensional convolution with different parameters is used to achieve x 1 and x 2 feature embedding, the formula is as follows: ; (8) ; (9) ; (10) ; (11) In equations (8) to (11), C1D1 and C1D2 represent two different 1×1 convolutional layers; S3-2-2. Regarding x 1 and x 2's internal features, the attention weight is obtained through the Sigmoid activation function: ; (12) ; (13) The fine features are obtained by multiplying the input features by the attention weights: ; (14) ; (15) In formula (14) to formula (15), Indicates element-related products; S3-2-3. For x 1 and x 2 feature interaction, x 1 Embed and x 2 Embed Connect to form features , use the Softmax activation function to generate the attention weights of the connection dimension , which represents the dynamic guidance of the network to the interactive correlation relationship, and its formula is: ; (16) x 1 and x The fine features of the interaction between 2 are obtained as follows: ; (17) ; (18) In formula (17) to formula (18), Represent the first row vector and the second row vector of W respectively; S3-3. x 1 Internal features and x 1. x Add the features between 2 to get the fusion feature x 3; S4, segmentation prediction: The last layer of the decoder performs 4x upsampling and outputs a feature map with a resolution of H×W. The upsampled feature labels are passed to the linear mapping layer, and the final pixel-level segmentation prediction results are output to complete the medical image segmentation based on multi-extended recursive attention.

[0015] We conducted preliminary testing experiments on the medical image segmentation scheme based on multi-extended recursive attention proposed in this paper. We used the Pytorch framework to conduct experiments on a server with NVIDIA TITAN RTX4090. The software platform was Pycharm, and the multi-extended recursive attention scheme was implemented in Python.

[0016] In this specific implementation, the comparison chart of the segmentation effects of other existing segmentation algorithms on the Synapse dataset is first tested, and then the comparison chart with other segmentation algorithm effects on the ISIC-2018 dataset is tested.

[0017] Figure 5 In the figure, Ours is the name of the segmentation method proposed in the present invention, Ground Truth represents manual annotation, TransUnet, Swin-Unet, MISSformer, VM-Unet, and BRAU-Net++ are the names of trackers proposed by other scholars in recent years, and the English on the left represents the colors of different organs. Figure 5 Some qualitative results of different methods on the Synapse dataset are given. Figure 5It can be seen that the method of the present invention generates smooth segmentation maps of the gallbladder, left kidney and pancreas, which indicates that multi-extended recursive attention may be good at capturing the features of small targets, and the present invention can better learn global and long-range semantic information, thereby producing better segmentation results.

[0018] Figure 6 The segmentation effect of the present invention and the advanced segmentation algorithms Swin-Unet, VM-Unet, and BRAU-Net++ in recent years on the ISIC-2018 dataset is compared. Figure 6 It can be seen that the segmentation method of the present invention obtains better boundary segmentation prediction than other methods, and the segmented mask contour is closer to the manual annotation value.

[0019] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A medical image segmentation method based on multi-extended recursive attention, comprising the following steps: S1. Select a medical image segmentation dataset and preprocess the training set in the dataset, including the following steps: S1-1, preprocess the medical image segmentation dataset, and then resize all preprocessed images to 224×224 pixel images as input images; S1-2, divide the input image obtained in step S1-1 into overlapping small blocks, obtain feature maps, and project the feature dimension of each feature map to an arbitrary dimension C through two 3×3 convolutional layers; S2, the encoder consists of several multi-extended recursive attention blocks and several feature merging layers. First, the feature map is encoded through several multi-extended recursive attention blocks, and then enters the feature merging layer to reduce the input spatial resolution, and multi-scale information and global information features are extracted from the feature map, including the following steps: S2-1, input the feature map preprocessed in step S1 into the continuous multi-expanded recursive attention block to perform representation learning; In the multi-expanded recurrent attention block: S2-1-1. Input features along the channel dimension Divided into four different heads, generating , R is the color channel, R=3, H, W are the spatial dimensions, and C is the feature channel; then, each head undergoes four depth-separable dilated convolutions with different dilation rates d1, d2, d3, and d4, respectively, to obtain , the formula is as follows: ；（1） In formula (1), They represent the features obtained by applying depthwise separable dilated convolution to the heads corresponding to d1 to d4, where d1=1, d2=2, d3=3, and d4=4; the operation DDWConv() represents depthwise separable dilated convolution; S2-1-2. Recursive attention includes a recursive process and multi-scale attention. First, the features after the depthwise separable dilated convolution are input into the multi-scale attention, which uses 1 × 1 convolution to apply linear projection. Then, three convolutions d5, d6, and d7 with different dilation rates but shared kernels are applied to capture the multi-scale context, and the group number is set to the feature channel number, where d5=1, d6=3, and d7=5. Finally, the parallel features of different scales are weighted and summed, and the SiLU self-calibration mechanism is used to determine the weight of each scale by the activation strength. The formula is as follows: ；（2） In formula (2), ；（3） ；（4） In formula (2) to formula (4), is the feature map, is the weight value; H and W are spatial dimensions, C is the feature channel, K is the convolution kernel size, r is the convolution kernel expansion rate, and g is the number of convolution kernel groups; S2-1-3, after the multi-scale attention is completed, it enters the recursive attention, and then increases the depth of the recursive attention without increasing the parameters, and then forms the recursive attention with the multi-scale attention; the formula of the recursive attention is as follows: ；（5） ；（6） ；（7） In formula (5) to (7), t is the step length, is the hidden state; MDA is a nonlinear activation function, the initial hidden state is a linear function combining the input and hidden state, is the projection weight; S2-2, the input features of the encoder are sent to the feature merging layer with a 3×3 convolutional layer for downsampling after passing through multiple multi-expanded recursive attention blocks, reducing the input spatial resolution while increasing the number of channels, and performing 2x downsampling at the same time, reducing the number of tags by half and increasing the feature dimension by 2x; S3. The decoder is composed of multiple multi-extended recursive attention blocks and several upsampling layers. The features output by the multi-extended recursive attention blocks in the encoder are connected through jumps, and then interact with the output of the extended recursive attention blocks in the decoder to decode the feature map information, including the following steps: S3-1, in the decoder, the feature map after the multi-expanded recursive attention block in the encoder of step S2 is input into the upsampling layer, that is, the resolution of the feature map is increased by 2 times, and the dimension of the feature map is reduced by half, and it is reshaped into a feature map with higher resolution; S3-2. Let the output of the multi-extended recursive attention block in the encoder be , the output of the multi-expanded recurrent attention block in the decoder is , construct a skip connection by connecting the outputs of the encoder and decoder, and connect the higher resolution feature maps in the upsampling layer through the skip connection; S3-2-1. Use global average pooling to aggregate spatial information into a channel-level global feature and generate ; Then, one-dimensional convolution with different parameters is used to achieve x 1 and x 2 feature embedding, the formula is as follows: ；（8） ；（9） ；（10） ；（11） In equations (8) to (11), C1D1 and C1D2 represent two different 1×1 convolutional layers; S3-2-2. Regarding x 1 and x 2's internal features, and obtains the attention weight through the Sigmoid activation function: ；（12） ；（13） The fine features are obtained by multiplying the input features by the attention weights: ；（14） ；（15） In formula (14) to formula (15), Indicates element-related products; S3-2-3. For x 1 and x 2 feature interaction, x 1 Embed and x 2 Embed Connect to form features , use the Softmax activation function to generate the attention weights of the connection dimension , which represents the dynamic guidance of the network to the interactive correlation relationship, and its formula is: ；（16） x 1 and x The fine features of the interaction between 2 are obtained as follows: ；（17） ；（18） In formula (17) to formula (18), Represent the first row vector and the second row vector of W respectively; S3-3. x 1 Internal features and x 1. x Add the features between 2 to get the fusion feature x 3; S4, segmentation prediction: The last layer of the decoder performs 4x upsampling and outputs a feature map with a resolution of H×W. The upsampled feature labels are passed to the linear mapping layer, and the final pixel-level segmentation prediction results are output to complete the medical image segmentation based on multi-extended recursive attention.

Images