A stroke image segmentation method based on multi-attention and dilated convolution
By adopting multi-attention and hollow convolution methods in stroke image segmentation, the problem of segmentation of stroke lesions is solved, and the segmentation accuracy is improved, especially in the prediction of small lesions and the prediction results in large lesions are more detailed.
Patent Information
- Application Number
- CN202410064036.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-17
- Publication Date
- 2025-03-28
- Estimated Expiration
- 2044-01-17
AI Technical Summary
The size of the stroke lesion area is large, the shape changes are diverse, and the appearance of normal tissues on the image is similar, resulting in poor performance of traditional image segmentation methods when distinguishing the lesion area from normal tissue.
Using a stroke image segmentation method based on multi-attention and hollow convolution, the network architecture consists of a downsampling module, a visual Transformer module, an upsampling module and a multi-hole convolution jump connection module. The extraction of detailed textures and global semantic information is enhanced by the multi-attention mechanism and hollow convolution.
It effectively solves the problem of segmentation of lesions in stroke image segmentation, improves the segmentation accuracy of the model, especially in the prediction of small lesions, and the prediction results in large lesions are more detailed.
Smart Images
Figure CN117876689B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of brain image segmentation based on deep learning, and in particular to a stroke image segmentation method based on multi-attention and dilated convolution. Background Art
[0002] Stroke is a common and serious disease. Its occurrence is related to the interruption of blood supply to the brain, resulting in brain tissue damage. Brain image segmentation plays an important role in the early diagnosis and treatment of stroke. Brain image segmentation is the process of extracting the area of interest in a medical image from the background. For stroke image segmentation, the key task is to accurately segment the lesion area, such as ischemic lesions or hemorrhagic areas, which helps doctors make more accurate diagnoses, assess the severity of the disease, and develop personalized treatment plans.
[0003] In recent years, with the development of deep learning technology, architectures such as UNet, DeepLab3+, and CLCI-Net have been widely used in medical image segmentation and have achieved good segmentation results. However, there are still great challenges in the image segmentation task of stroke lesion areas. First, the volume of stroke lesions varies greatly, from tiny local lesions to large-scale cerebral infarction areas, and the size and shape of lesions vary, making accurate segmentation of lesions more difficult. Secondly, the appearance of lesions and normal tissues in images is very similar. For example, the contrast between stroke lesions and normal brain tissues is low, and the texture features are similar, which leads to many traditional image segmentation methods performing poorly in distinguishing lesions from normal tissues. Summary of the invention
[0004] In view of the above technical problems, the present invention proposes a stroke image segmentation method based on multi-attention and dilated convolution. The network architecture of the present invention consists of four parts, including a downsampling module, a visual transformer module, an upsampling module and a multi-hole convolution jump connection module. The downsampling module extracts multi-level features from the image; the visual transformer module effectively captures global context information and establishes dependencies between various image regions; the multi-hole convolution jump connection module fuses the features extracted by convolutions with different dilation rates, so that the model can capture a wider range of context information and significantly expand the receptive field to adapt to the segmentation of different lesion areas; the upsampling module combines the multi-attention mechanism to decode and map deep features into segmentation label images. By combining the multi-attention mechanism and dilated convolution, the present invention enhances the model's extraction of detailed texture and global semantic information, and can adaptively extract important features in the image, thereby effectively solving the problems existing in the stroke image segmentation task.
[0005] The specific steps of the method of the present invention include:
[0006] S1. Data preprocessing: Divide the dataset into training set and test set, and use interpolation method to scale the images to be segmented in the dataset to a uniform spatial resolution.
[0007] S2. The images in the training set are preprocessed in step S1 and input into the downsampling module. The downsampling module consists of k convolution units, and each convolution unit i outputs feature F i ,i∈[1,k].
[0008] The first convolution unit is composed of a convolution layer and a maximum pooling layer cascaded.
[0009] The 2nd to kth convolution units, each convolution unit is composed of a number of convolution blocks, each convolution block is obtained by sequentially combining three convolution layers, and each convolution block adds a residual connection to connect the input residual of the convolution block to the output of the convolution block, and the residual connection is composed of a 1×1 convolution layer.
[0010] S3. The first k-1 different level features F extracted in step S2 i Input into the multi-hole convolution jump connection module respectively to extract features
[0011] The multi-atrous convolution jump connection module is composed of multi-branch atrous convolutions. By fusing the features extracted by convolutions with different atrous rates, the receptive field of the model is increased to adapt to the segmentation of lesion areas of different sizes. The specific steps include:
[0012] S31. Feature F i Input to the first branch, which consists of a convolution module with a convolution kernel size of 1×1 and a void rate of 1, and obtains the output feature
[0013] S32. Feature F i Input to the second branch, which consists of a convolution module with a convolution kernel size of 1×1 and a void rate of 1 and a convolution module with a convolution kernel size of 3×3 and a void rate of 3, and obtain the output feature
[0014] S33. Feature F i Input to the third branch, which consists of a convolution module with a convolution kernel size of 1×1 and a void rate of 1 and a convolution module with a convolution kernel size of 3×3 and a void rate of 5, and obtain the output feature
[0015] S34. Feature F i Input to the fourth branch, which is the identity mapping branch, and its output is feature F i .
[0016] S35. The output features of the first three branches above are cascaded in the channel dimension f cat (), and then through a 1×1 convolution f con1×1 () Adaptively compress the channel dimension to the number of channels of the input feature, and add the obtained feature to the feature output of the fourth branch, which is the output of the i-th skip connection module
[0017] S4. The feature F output by the kth convolution unit in the downsampling module k Perform linear mapping, then input it into the visual Transformer module, extract global features through n-layer Transformer encoder, and further perform convolution and deconvolution operations to obtain the output feature f k-1 .
[0018] S5. The feature f output from step S4 is k-1 and the features extracted by the k-1th multi-hole convolutional skip connection module in step S3 Cascading is performed in the channel dimension, and the cascaded features are input into the upsampling module to obtain the segmentation result of the input image.
[0019] The upsampling module is composed of k-1 attention units in cascade, each of which includes multiple attention modules and deconvolution layers connected in sequence; the input of the i-th attention unit is the feature f output by the previous module i and the features extracted by the i-th multi-hole convolutional jump connection module in step S3 The feature formed after cascading in the channel dimension, the output of the i-th attention unit is the feature f i-1 .
[0020] The specific steps of the attention unit include:
[0021] S51. Input the input features to the multi-attention module to extract global high-level semantic features and filter information, including the following steps:
[0022] S511. Input the features into a convolutional long short-term memory module (ConvLSTM) to extract feature F in the feature image.
[0023] S512. Input the feature F output in step S511 into the global context module, and calculate the global attention W through 1×1 convolution and softmax function. c .
[0024] S513. Global Attention W c Normalize it, multiply it with the feature F, and add it to the feature F after linear transformation to get the output F of the global context moduleGC .
[0025] S514. The feature F output from step S513 is GC Input to the channel attention module, and learn the weight of each channel through adaptive learning.
[0026] S52. Input the features output from step S514 into the deconvolution layer for feature upsampling, and enlarge its spatial resolution by two times.
[0027] Beneficial effects of the present invention: The present invention extracts multi-level features of the input image through the downsampling module, and transmits the multi-level features to the upsampling module through the multi-hole convolution jump connection module; the visual Transformer module effectively captures the global context information and establishes the dependency relationship between each image area; in the upsampling module, the features of the downsampling module and the features output by the previous module are cascaded in the channel dimension, and the cascaded features are adaptively fused through the multi-attention module; the features output by the upsampling module are subjected to convolution operation to obtain the segmentation prediction result. The present invention has the following benefits: 1. The multi-hole convolution jump connection module is designed to increase the receptive field of the model, so that it is suitable for the segmentation of lesion areas of different sizes; 2. The multi-attention module is designed, and by combining the global context attention and channel attention mechanism, a larger weight is given to the important information in the features, and unimportant information is filtered out, thereby improving the segmentation accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 Architecture diagram of stroke image segmentation based on multi-attention and dilated convolution;
[0029] Figure 2 Structure diagram of multi-hole convolutional jump connection module;
[0030] Figure 3 Global context and channel attention module structure diagram;
[0031] Figure 4 The effect diagram of the method of the present invention on segmentation of stroke lesion area. DETAILED DESCRIPTION
[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0033] As a preferred implementation form of the present invention, a stroke image segmentation method based on multi-attention and dilated convolution is provided, and its architecture is as follows: Figure 1 As shown, the following steps are included:
[0034] S1. Data preprocessing: The dataset is divided into a training set and a test set, and the images to be segmented are scaled to a uniform spatial resolution of 224*224 using an interpolation method.
[0035] S2. The images in the training set are preprocessed in step S1 and input into the downsampling module. The downsampling module consists of k convolution units, and each convolution unit i outputs feature F i ,i∈[1,k]; in this embodiment, k=4.
[0036] The first convolution unit is composed of a 7×7 convolution layer and a maximum pooling layer cascaded.
[0037] The 2nd to kth convolution units, each convolution unit is composed of several convolution blocks, each convolution block is obtained by sequentially combining three convolutions, and each convolution block adds a residual connection to connect the input residual of the convolution block to the output of the convolution block, and the residual connection is composed of a 1×1 convolution layer. In this embodiment, the second convolution unit is composed of 3 convolution blocks, the second convolution unit is composed of 4 convolution blocks, and the second convolution unit is composed of 6 convolution blocks. Each convolution block has three convolution layers, and the convolution kernel sizes are 1×1, 3×3 and 1×1 respectively.
[0038] S3. The first k-1 different level features F extracted in step S2 i Enter into Figure 2 The multi-hole convolution jump connection module shown in the figure extracts features
[0039] The multi-atrous convolution jump connection module is composed of multi-branch atrous convolutions. By fusing the features extracted by convolutions with different atrous rates, the receptive field of the model is increased to adapt to the segmentation of lesion areas of different sizes. The specific steps include:
[0040] S31. Feature F i Input to the first branch, which consists of a convolution module with a convolution kernel size of 1×1 and a void rate of 1, and obtains the output feature
[0041] S32. Feature F i Input to the second branch, which consists of a convolution module with a convolution kernel size of 1×1 and a void rate of 1 and a convolution module with a convolution kernel size of 3×3 and a void rate of 3, and obtain the output feature
[0042] S33. Feature F i Input to the third branch, which consists of a convolution module with a convolution kernel size of 1×1 and a void rate of 1 and a convolution module with a convolution kernel size of 3×3 and a void rate of 5, and obtain the output feature
[0043] S34. Feature F i Input to the fourth branch, which is the identity mapping branch, and its output is feature F i .
[0044] S35. The output features of the first three branches above are cascaded in the channel dimension f cat (), and then through a 1×1 convolution f con1×1 () Adaptively compress the channel dimension to the number of channels of the input feature, and add the obtained feature to the feature output of the fourth branch, which is the output of the i-th skip connection module The calculation formula is:
[0045]
[0046] S4. The feature F output by the kth convolution unit in the downsampling module k The linear mapping is performed and then input into the visual Transformer module. The global features are extracted through the 12-layer Transformer encoder, and further 3×3 convolution and deconvolution operations are performed to obtain the output feature f k-1 .
[0047] S5. The feature f output from step S4 is k-1 and the features extracted by the k-1th multi-hole convolutional skip connection module in step S3 The cascade is performed in the channel dimension, and the cascaded features are input into the upsampling module to obtain the segmentation prediction result of the input image.
[0048] The upsampling module is composed of k-1 attention units in cascade, each of which includes multiple attention modules and deconvolution layers connected in sequence; the input of the i-th attention unit is the feature f output by the previous module i and the features extracted by the i-th multi-hole convolutional jump connection module in step S3 The feature formed after cascading in the channel dimension, the output of the i-th attention unit is the feature f i-1 .
[0049] The specific steps of the attention unit include:
[0050] S51. Input the input features to the multi-attention module to extract global high-level semantic features and filter information, such as Figure 3 As shown, the following steps are included:
[0051] S511. Input the features into the convolutional long short-term memory module (ConvLSTM) to extract the global features F in the image.
[0052] S512. Input the feature F output from step S511 into the global context module, and perform a 1×1 convolution on it. con1×1 () and the softmax function f softmax () Calculate the global attention W c , the formula is:
[0053] W c =f softmax (f con1×1 (F)
[0054] S513. Global Attention W c Normalize it, multiply it with the feature F, and add it to the feature F after linear transformation to get the output F of the global context module GC , the specific formula is:
[0055] F GC =W*(f norm (W c )*F)+F
[0056] Where W is the linear transformation matrix, f norm () is the normalization function.
[0057] S514. The feature F output from step S513 is GC The information is input into the channel attention module, which adaptively learns the weight of each channel, giving larger weights to important channel information and smaller weights to unimportant channel information.
[0058] S52. Input the features output from step S514 into the deconvolution layer for feature upsampling, and enlarge its spatial resolution by two times.
[0059] Example:
[0060] The steps of this embodiment are the same as those of the specific implementation method, and will not be described in detail here. The following is a demonstration of some implementation processes and implementation results.
[0061] The technology of the present invention is implemented in the data set ATLAS-V2.0, which contains 401 MRI case images, including a large number of images of non-lesion areas. In order to reduce the number of images of non-lesion areas, MRI slices in the middle part are selected to obtain 40,000 images as a training set and 15,000 images as a test set. For the training process of the model, the image batch size for each iteration is set to 8, and the Adam optimizer is used to train the model of the present invention, and the learning rate is set to 0.0001.
[0062] In order to verify the effectiveness of the technology of the present invention, Figure 4The segmentation effect diagram of the technology of the present invention after training on the data set ATLAS-V2.0 is shown, and compared with methods such as U-Net, CA-Net, and CLCI-Net. Compared with methods such as U-Net and CLCI-Net, the technology of the present invention can achieve more accurate predictions in small lesion areas, which proves the effectiveness of the designed multi-hole convolution jump connection module. When predicting large lesion areas, the prediction results of the technology of the present invention are more detailed, and the predicted shapes are very similar to the true labels, which shows that the global context and channel attention modules can enrich the details of the prediction results. The above results prove the effectiveness of the technology of the present invention.
[0063] In order to verify the effectiveness of the technology of the present invention, Table 1 uses the indicators Dice (measure the overlap between two sets), Precision (accuracy), Recall (recall rate, measuring the proportion of samples correctly predicted by the model as positive examples to all actual positive examples), VOE (volume overlap error, measuring the error of the volume overlap between the segmentation result and the true segmentation), and IOU (Intersection over Union, measuring the overlap between the predicted result and the true annotation) to compare the technology of the present invention with the existing technology. As shown in Table 1, the technology of the present invention has achieved better results in many indicators, thus proving the effectiveness of the technology of the present invention.
[0064] Table 1 Quantitative comparison results between the invention technology and the prior art
[0065]
Claims
1. A stroke image segmentation method based on multi-attention and dilated convolution, characterized in that: The following steps are involved: S1. Divide the dataset into a training set and a test set, and use the interpolation method to scale the images to be segmented in the dataset to a uniform spatial resolution; S2. Input the image in the training set into the downsampling module, which consists of k convolution units, and each convolution unit i outputs feature F i ,i∈[1,k]; S3. The first k-1 different level features F extracted in step S2 i Input into the multi-hole convolution jump connection module respectively to extract features The multi-hole convolution jump connection module is composed of multi-branch hole convolution, specifically including: S31. Feature F i Input to the first branch, which consists of a convolution module with a convolution kernel size of 1×1 and a void rate of 1, and obtains the output feature S32. Feature F i Input to the second branch, which consists of a convolution module with a convolution kernel size of 1×1 and a void rate of 1 and a convolution module with a convolution kernel size of 3×3 and a void rate of 3, and obtain the output feature S33. Feature F i Input to the third branch, which consists of a convolution module with a convolution kernel size of 1×1 and a void rate of 1 and a convolution module with a convolution kernel size of 3×3 and a void rate of 5, and obtain the output feature S34. Feature F i Input to the fourth branch, which is the identity mapping branch, and its output is feature F i ; S35. The output features of the first three branches above are cascaded in the channel dimension f cat (), and then through a 1×1 convolution f con1×1 () Adaptively compress the channel dimension to the number of channels of the input feature, and add the obtained feature to the feature output of the fourth branch, which is the output of the i-th skip connection module S4. The feature F output by the kth convolution unit in the downsampling module k Perform linear mapping, then input it into the visual Transformer module, extract global features through n-layer Transformer encoder, and then perform convolution and deconvolution operations to obtain the output feature f k-1 ; S5. Output feature f k-1 And the features extracted by the k-1th multi-hole convolutional skip connection module Cascading is performed in the channel dimension, and the cascaded features are input into the upsampling module to obtain the segmentation result of the input image.
2. The stroke image segmentation method based on multi-attention and dilated convolution according to claim 1, characterized in that: In S2, the k convolution units, the first convolution unit is composed of a convolution layer and a maximum pooling layer cascaded; From the 2nd to the kth convolutional units, each convolutional unit is composed of several convolutional blocks, each convolutional block is obtained by sequentially combining three convolutional layers, and each convolutional block adds a residual connection to connect the input residual of the convolutional block to the output of the convolutional block.
3. The stroke image segmentation method based on multi-attention and dilated convolution according to claim 2, characterized in that: The residual connection consists of a 1×1 convolutional layer.
4. The stroke image segmentation method based on multi-attention and dilated convolution according to any one of claims 1 to 3, characterized in that: S5 The upsampling module is composed of k-1 attention units in cascade, each of which includes multiple attention modules and deconvolution layers connected in sequence; the input of the i-th attention unit is the feature f output by the previous module i and the features extracted by the i-th multi-hole convolutional jump connection module in step S3 The feature formed after the channel dimension cascade, the output of the i-th attention unit is the feature f i-1 ; The attention unit specifically includes: S51. Input the input features to the multi-attention module to extract global high-level semantic features and filter information; S52. Input the features output from step S51 into the deconvolution layer for feature upsampling, and enlarge its spatial resolution by two times.
5. The stroke image segmentation method based on multi-attention and dilated convolution according to claim 4, characterized in that: The specific process of S51 is as follows: S511. Input the feature into the convolutional long short-term memory module to extract the feature F in the feature image; S512. Input the feature F into the global context module and calculate the global attention W through 1×1 convolution and softmax function. c ; S513. Global Attention W c Normalize it, multiply it with the feature F, and add it to the feature F after linear transformation to get the output F of the global context module GC ; S514. Set feature F GC Input to the channel attention module, and learn the weight of each channel through adaptive learning.