Medical image automatic segmentation method based on grouping multi-scale and cross-scale fusion mechanism

Abstract

The invention provides a medical image automatic segmentation method based on a grouping multi-scale and cross-scale fusion mechanism, and aims to solve the problem that the performance of a traditional Transform is limited due to changes of organ structures, textures and shapes in medical image segmentation. According to the method, features are preliminarily extracted through a convolution stem layer, and after the spatial dimension is reduced, a feature map is sent to an encoder comprising a local sensing module and a multi-scale sensing module. The local sensing module extracts local space information and provides inductive bias for the self-attention module, and the multi-scale sensing module captures multi-scale information through a grouping multi-scale and cross-scale fusion mechanism. The decoder combines a multi-scale convolution block and an attention module to recover features, and uses an attention-based feature fusion module in jump connection. Experimental results show that the method is good in performance on a public data set, multi-scale information can be fully captured, the medical image segmentation performance is improved, and the method has application prospects.

Description

Technical Field

[0001] The present invention relates to a medical image automatic segmentation method, in particular to a medical image automatic segmentation method based on grouping multi-scale and cross-scale fusion mechanism Background Art

[0002] Convolutional neural networks (CNNs) have proven to be very effective in medical image segmentation, thanks to their high efficiency in extracting spatial features. However, due to the limited receptive field size, traditional convolutions have limitations in capturing long-range dependencies and global context. In recent years, Transformers have shown good prospects in medical image segmentation due to their ability to model long-range dependencies and implicit spatial relationships. However, despite the significant progress made by Transformers, current research on Transformers mainly focuses on reducing the computational complexity of self-attention, while ignoring the multi-scale characteristics of objects within the same attention layer. For example, the PVT model, which has received much attention, proposes to merge spatial tokens in self-attention to reduce computational costs. However, this approach tends to merge too many tokens in a single self-attention layer, which results in a mixture of tokens from small objects and background noise. This oversight limits the performance of these methods when processing medical images containing objects of different sizes. The root cause is that current self-attention mechanisms usually use a fixed receptive field to generate tokens and maintain a consistent information granularity within a single attention layer. Therefore, they cannot capture features at different scales simultaneously. On the other hand, existing self-attention mechanisms usually rely on multi-head attention for group learning across channel dimensions. However, recent studies have shown that multi-head attention often exhibits significant redundancy, which inevitably reduces the efficiency of the model. To solve this problem, some studies have used the method of connecting multi-head attention heads in series, providing the output of one head as input to the next head. In this way, the flow of information between different heads can be promoted to generate head information with different features, but it also destroys the parallelism of multi-head attention and reduces computational efficiency. Therefore, it is necessary to design an appropriate structure that can well apply the advantages of the self-attention module, while being able to and fully capture the multi-scale feature information in the image to better cope with significant changes in organ structure, texture and shape. The present invention aims to propose a new method to overcome this difficulty and bring better reasoning results. Summary of the invention

[0003] The purpose of the present invention is to provide a method for automatic segmentation of medical images based on a grouped multi-scale and cross-scale fusion mechanism. The technical solution to achieve the purpose of the present invention is to extract features of sampled image blocks through a model constructed based on the self-attention layer idea of ​​a grouped multi-scale and cross-scale fusion mechanism, including the following steps:

[0004] Step 1: Preprocess the data based on the existing medical image dataset.

[0005] Step 2: Divide the processed data into training set, validation set and test set.

[0006] Step 3: Train the designed segmentation network model based on the training set:

[0007] Step 3.1: Use a multi-level encoder network to extract graphic features.

[0008] Step 3.2: In each encoder, the local perception module is first used to extract local spatial information and help the self-attention module obtain the inductive bias. Then, a multi-scale perception module based on grouped multi-scale and cross-scale fusion mechanisms is used to capture and fuse the multi-scale information in the feature image.

[0009] Step 3.3: Use the decoder structure to recover the features, and use the efficient fusion module to pass the detailed information in the encoder level.

[0010] Step 3.4: After calculating the loss based on the loss function, use the gradient descent algorithm to converge the model.

[0011] Step 4: Repeat step 3 to train the segmentation network model.

[0012] Step 5: Use the trained segmentation network to test the test data.

[0013] Compared with the prior art, the present invention has the following significant advantages:

[0014] (1) A local perception module consisting of a depthwise separable convolution and a feature space compression block is used to extract local spatial information and help the self-attention module obtain the inductive bias. Compared with traditional methods, it requires fewer parameters and has good performance.

[0015] (2) In order to fully capture the multi-scale information in the image, a cascaded self-attention mechanism based on group multi-scale and cross-scale fusion mechanism is proposed, which can fully capture and fuse the multi-scale information in the image.

[0016] (3) A decoding block consisting of a multi-scale convolutional block and spatial attention and channel attention is used. This can better adapt to the differences in morphology, structure, and size of different organs in medical images. At the same time, an efficient feature fusion module based on the idea of ​​attention is added to further enhance the recovery of feature information in the decoder.

[0017] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is the overall flow chart of the automatic medical image segmentation method based on the grouped multi-scale and cross-scale fusion mechanism of the present invention and the internal details.

[0019] Figure 2 This is the experimental result of comparing the model of the method of the present invention with several existing models on a certain skin melanoma segmentation dataset.

[0020] Figure 3 The model of the method of the present invention is compared with the results of several models on actual test data.

[0021] Figure 4 It is a summary of the specification of the present invention, which describes the overall process logic of the present invention. DETAILED DESCRIPTION

[0022] Combination Figure 1 The present invention provides a medical image automatic segmentation method based on group multi-scale and cross-scale fusion mechanism, comprising the following steps:

[0023] Step 1: Preprocess the data based on the existing medical image dataset.

[0024] Step 2: Divide the processed data into training set, validation set and test set.

[0025] Step 3: Train the designed segmentation network model based on the training set:

[0026] Step 3.1: Use a multi-level encoder network to extract graphic features.

[0027] Step 3.2: In each encoder, the local perception module is first used to extract local spatial information and help the self-attention module obtain the inductive bias. Then, a multi-scale perception module based on grouped multi-scale and cross-scale fusion mechanisms is used to capture and fuse the multi-scale information in the feature image.

[0028] Step 3.3: Use the decoder structure to recover the features, and use the efficient fusion module to pass the detailed information in the encoder level.

[0029] Step 3.4: After calculating the loss based on the loss function, use the gradient descent algorithm to converge the model.

[0030] Step 4: Repeat step 3 to train the segmentation network model.

[0031] Step 5: Use the trained segmentation network model to test the test data.

[0032] The present invention is further described in detail below with reference to examples:

[0033] The present invention uses medical magnetic resonance images or computer tomography images as input, and generates results after automatic segmentation of lesions through data preprocessing and feature extraction networks.

[0034] The process of the present invention is as follows Figure 1 shown.

[0035] (1) The ISIC2018 dataset used for training and testing of the present invention is a public skin melanoma segmentation dataset. The dataset contains 2694 dermoscopic images with segmentation mask labels. Following the official method, we divided the dataset into 1886 images for training and 808 images for testing. The annotation of each image includes the location of skin melanoma. At the same time, each case data is processed into a three-dimensional data slice with a size of 256 pixels * 256 pixels and a single feature channel as input. Since the amount of data involved in the training is small, in order to obtain a better model, we augmented the source data, including randomly selected image flipping, image rotation, adding Gaussian noise, blurring and other operations.

[0036] (2) Figure 1 (a) shows the model architecture diagram of the present invention.

[0037] First, the input image is subjected to convolution to perform preliminary feature extraction. The network structure consists of two simple convolutional layers, which reduces the input image size to 1 / 4 of the original image and the feature channel dimension to 128.

[0038] Feature Encoder: The encoder is mainly divided into two parts. The first part is constructed by depthwise separable convolution and feature space compression blocks, such as Figure 1 As shown in (e), in this module, feature space information is extracted through depthwise separable convolution. Compared with traditional convolution operations, depthwise separable convolution decomposes standard convolution into two separate steps: depthwise convolution and pointwise convolution. Its advantage is that it reduces the number of model parameters and computational complexity. It uses fewer parameters to implement convolution operations, thereby reducing the complexity of the model and improving computational efficiency. On the other hand, the channel dimension can be expanded and compressed through the feature compression block, thereby filtering out invalid channel feature information. The second part is a multi-scale perception module composed of a grouped multi-scale and cross-scale fusion mechanism, such as Figure 1 (e) and Figure 1As shown in (f), the module is a cascaded self-attention module, which consists of a grouped multi-scale attention module and a cross-scale attention module. The grouped multi-scale module directly groups feature maps along the channel and uses different patch sizes for different groups to generate feature vectors of different scales, and reduces channel redundancy within different groups. At the same time, it uses the traditional spatial self-attention mechanism to generate feature weight maps to capture features of different scales. The cross-scale attention module receives the output from the grouped multi-scale module, and uses channel self-attention to generate feature weights for feature groups of different scales, and reduces the redundancy of feature channels. Global pooling and a forward feedback network are then used to fuse feature masses from different scales and apply them to the query of each group, thereby achieving the effect of multi-scale feature fusion. The decoder structure uses multi-scale convolution blocks and post-processing modules based on spatial attention and channel attention to extract feature information. At the same time, high-order features are restored to low-dimensional feature maps through upsampling of convolution, and the efficient feature fusion module proposed in this paper is used to enhance information recovery. Finally, the segmentation result is generated after upsampling.

[0039] Efficient feature fusion module: The semantic gap between the encoder and decoder levels limits the segmentation performance. This method proposes an efficient feature fusion module based on the idea of ​​attention. By better fusing the information of the encoding layer and the decoding layer in space and channels to help the decoding layer recover information, to achieve this, we retain the jump connection part of the original U-Net structure before each downsampling layer and construct an additional efficient feature fusion module. Figure 1 As shown in (b), this module takes the output from the encoder and decoder, applies a feature compression block to learn channel features, and then uses convolution to fuse these channel features. We use a sigmoid function to generate a gating unit from the decoder output and multiply it with the former to dynamically adjust the spatial attention weights in an adaptive manner. Finally, this result is combined with the output of the original encoder to produce the final output, which helps stabilize the gradient and alleviate the problem of gradient disappearance.

[0040] Loss function: The method of the present invention uses the sum of the DICE index and the cross entropy as the loss function to perform model gradient descent convergence.

[0041] (3) Repeat step 3 to train the segmentation network model. The method of the present invention uses average intersection-over-union ratio, similarity coefficient, accuracy, sensitivity and specificity as indicators for judging the quality of the model.

[0042] (4) The trained segmentation network model is used to test the test data. The proposed method and several existing methods (UNet, Attention UNet, DAEFormer, TransFuse, MISSFormer, TransFuse, MALUNet, EGE-UNet) are compared on the test set. Based on the experimental results, the average intersection-over-union ratio, similarity coefficient, accuracy and sensitivity of the proposed method achieve good experimental results. The average intersection-over-union ratio of regional measurement of 81.07 is achieved on the ISIC2018 dataset, which is better than other Transformer-based methods such as DAEFormer and MISSFormer with pure Transformer, and TransFuse with multiple Transformer modules stacked in the deep layer of the encoder. The contours of the segmentation mask generated by our proposed method are more consistent with the standard results and are more resistant to noise interference, indicating that the model is more robust. Benefiting from the grouped multi-scale and cross-scale self-attention modules, the post-processing blocks using multi-scale convolution modules and spatial and channel attention mechanisms, and the efficient feature fusion module based on the idea of ​​attention, the proposed model can achieve better medical image segmentation results.

Claims

1. A method for automatic segmentation of medical images based on group multi-scale and cross-scale fusion mechanism, which is characterized by the following steps: Step 1: Preprocess the data based on the existing medical image dataset. Step 2: Divide the processed data into training set, test set and validation set. Step 3: Train the designed segmentation network model based on the training set: Step 3.1: Use a multi-level encoder network to extract graphic features. Step 3.2: In each encoder, the local perception module is first used to extract local spatial information and help the self-attention module obtain the inductive bias. Then, a multi-scale perception module based on grouped multi-scale and cross-scale fusion mechanisms is used to capture and fuse the multi-scale information in the feature image. Step 3.3: Use the decoder structure to recover the features, and use the efficient fusion module to pass the detailed information in the encoder level. Step 3.4: After calculating the loss based on the loss function, use the gradient descent algorithm to converge the model. Step 4: Repeat step 3 to train the segmentation network model. Step 5: Use the trained segmentation network to test the test data.

2. The method for automatic segmentation of medical images based on grouped multi-scale and cross-scale fusion mechanism according to claim 1, characterized in that: The specific method of training the network model described in step 3 is as follows: The network is a multi-stage network based on grouped multi-scale and cross-scale fusion mechanisms. First, the image is passed through the convolution stem module to initially extract feature information and reduce its spatial dimension. In each encoder, the local perception module is first used to extract local spatial information and help the self-attention module obtain the inductive bias. Then, a multi-scale perception module based on grouped multi-scale and cross-scale fusion mechanisms is used to capture and fuse the multi-scale information in the feature image. The decoder structure consists of a multi-scale convolution block for further capturing multi-scale features and a post-processing module based on spatial attention and channel attention. The high-dimensional features are restored to low-dimensional feature maps through convolution upsampling, and the encoder information is passed using an efficient fusion module to help restore the image information. Finally, the segmentation result is generated after upsampling.

3. The method for automatic segmentation of medical images based on grouped multi-scale and cross-scale fusion mechanism according to claim 1, characterized in that: The feature extraction module in each encoder of the segmentation network described in step 3.2 is as follows: The local perception module composed of deep separable convolution and feature space compression blocks is used to extract local spatial information. The deep separable convolution has the inductive characteristics of convolution on local positions, and has less computational complexity than ordinary convolution. The features are then passed through a multi-scale perception module composed of a group-based multi-scale and cross-scale fusion mechanism to extract and fuse multi-scale information in the feature image.

Images