PCMA-UNet, a stroke segmentation network based on a hybrid attention mechanism of UNet3+.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-25
- Publication Date
- 2026-03-10
Smart Images
Figure CN116452611B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of image and disease diagnosis, and particularly relates to a mixed attention mechanism stroke segmentation network PCMA-UNet based on UNet3+ BACKGROUND
[0002] Stroke is the most common manifestation of cerebrovascular disease with high incidence, high disability rate, high mortality, high recurrence rate and high economic burden, and is one of the three major causes of adult death and disability. According to the statistical report of the American Heart Association on stroke in 2020, the prevalence of stroke in the United States is 2.5%, which means that 7 million Americans over the age of 20 have suffered from stroke, and nearly 150,000 people have died. Stroke can be divided into hemorrhagic stroke and ischemic stroke, and the incidence of ischemic stroke accounts for about 87% of the total. About 700,000 people in the United States suffer from ischemic stroke every year, and more than half of the patients have moderate to severe neurological dysfunction, and the mortality rate within 30 days is between 10% and 17%.
[0003] The current early treatment strategy for stroke has a high time dependence, and thrombolytic therapy is the main treatment method for rescuing patients with ischemic brain tissue. Intravenous thrombolysis within 3-4.5 hours of ischemic stroke can effectively improve the long-term prognosis. Therefore, for patients with ischemic stroke, shortening the time from onset to vascular recanalization is crucial for the outcome of stroke, and rapid and accurate diagnosis of stroke can effectively reduce the mortality and morbidity. Clinically, doctors usually use head magnetic resonance imaging (MRI) to observe the stroke lesion. Quantifying stroke lesions helps doctors assess patient conditions and develop treatment plans. However, the location and shape of stroke lesions vary significantly, and the lesion volume can range from hundreds of cubic millimeters to tens of thousands of cubic millimeters depending on the severity of the disease, and the lesion area can occur in the cerebrum, cerebellum and other areas of the cerebrum, and the lesion edge is not clear. In MRI, the gold standard for describing lesions is manual segmentation, which is a very time-consuming and subjective process. Therefore, accurate automatic segmentation is a challenging problem.
[0004] Computer-based automatic medical image processing is increasingly entering clinical routine. Using computer-aided diagnosis technology can accurately segment ischemic lesions in images, save time and effort for manual delineation by doctors, and reduce subjective influences from different doctors. It has great potential value in rapid screening and early clinical decision-making in hyperacute ischemic stroke. In the field of deep learning, convolutional neural networks have shown great potential in medical image segmentation due to their unique self-learning ability. In 2015, Ronneberger et al. proposed the UNet model for medical image segmentation tasks, which uses skip connections to fuse feature maps of different stages to improve image segmentation accuracy. Gu et al. proposed a context encoder network CE-Net to address the problem of spatial information loss in UNet, which uses dense atrous convolution blocks to encode high-level semantic feature maps to capture higher-level abstract features and retain more spatial information. Zhou et al. proposed UNet++, which indirectly fuses features of multiple different levels using nested dense skip connections, further narrowing the semantic gap between the encoder and the decoder. Huang et al. proposed UNet3+ to perform skip connections between different scales, and each decoder layer fuses feature maps from the encoder at small and same scales, as well as from the decoder at large scales. These feature maps capture fine-grained and coarse-grained semantics at full scale, which makes the segmentation of different parts and lesions in medical images good.
[0005] However, UNet3+ only uses shallow features for prediction, which limits the use of deep semantic features and affects the network's ability to accurately locate targets. In addition, UNet3+ can obtain feature maps of different scales through upsampling, but it lacks awareness of the most suitable scale for image segmentation, resulting in suboptimal segmentation results and some missed or missegmented phenomena. SUMMARY
[0006] The purpose of the present application is to solve the above problems and provide a segmentation network PCMA-UNet based on a hybrid attention mechanism of UNet3+ and its application in stroke image processing
[0007] The segmentation network PCMA-UNet based on the hybrid attention mechanism of UNet3+ uses UNet3+ as the framework, and the encoder performs downsampling to obtain feature maps of different scales. Different scale spatial information is obtained through pyramid squeeze attention PSA, and then coordinate attention CA is connected to retain feature map position information. Full-scale skip connections are used to concatenate different scale feature maps output by each layer of the decoder, input multi-scale attention MSA, and obtain network segmentation results.
[0008] The pyramid squeeze attention PSA comprises: firstly, a split and splice SPC module is used to obtain multi-scale feature maps in the channel direction; secondly, an SEWeight module is used to extract the attention of the feature maps of different scales to obtain channel feature weights; thirdly, a Softmax is used to recalibrate the channel-level feature weights to obtain the weights of the multi-scale channels; and fourthly, the recalibrated weights and the corresponding feature maps are multiplied by pixel products to obtain a refined feature map with richer multi-scale feature information as output.
[0009] The split and splice SPC module comprises:
[0010] The input is split into four parts, and four different scale features are extracted and spliced, which can be represented as:
[0011] (1)
[0012] , wherein represents the feature maps of different scales, and Cat represents a splicing operation is the obtained multi-scale feature map.
[0013] The channel attention weight information is extracted from the multi-scale preprocessed feature map to obtain an attention weight vector of different scales; the attention weight vector can be represented as:
[0014] (2)
[0015] The SEWeight module is used to obtain the attention weight from the input feature maps of different scales , so that the PSA module can fuse the context information of different scales, enhance useful features, and suppress useless features; a Softmax is used to obtain the recalibrated weights of the multi-scale channels, which contain all the position information in the space and the attention weight in the channel; the recalibrated channel attention of the features is concatenated and fused to obtain the entire channel attention vector as:
[0016] (3)
[0017] (4)
[0018] , wherein represents the multi-scale channel weight after attention interaction; the recalibrated multi-scale channel attention weight is multiplied by the feature map of the corresponding scale to obtain:
[0019] (5)
[0020] The feature map represents the attention weights with multi-scale channel orientations; finally, concatenating all scale features yields the final output:
[0021] (6).
[0022] The coordinate attention CA mentioned above includes two parts: coordinate information embedding and coordinate attention generation.
[0023] 1) Coordinate information embedding
[0024] The CA mechanism transforms global pooling decomposition into two parallel one-dimensional feature codes; given the input feature map... The global pooling operation is performed along the horizontal direction of the input feature map. and vertical direction By performing pooling operations, relevant location information of the input feature map can be obtained. These are the height and width of the feature map corresponding to the current attention module, respectively. These values change with downsampling, with the height being... The Each channel output is:
[0025] (7)
[0026] Width is The Each channel output is:
[0027] (8);
[0028] 2) Coordinate Attention Generation
[0029] The two transformations described above are concatenated in the spatial dimension, and a 1×1 convolution is used to compress the channels. Then, batch normalization and non-linearity are used to encode the spatial information in the vertical and horizontal directions. Next, a 1×1 convolution is used to segment the encoded information, resulting in two feature tensors along the horizontal and vertical directions. The number of channels in the attention map is adjusted to be equal to the number of channels in the input feature map. The sigmoid function is used for normalization and weighted fusion. The final output can be represented as follows:
[0030] (9)
[0031] For the input feature map, and These represent the attention weights in the height and width spatial directions, respectively.
[0032] The aforementioned multi-scale attention MSA,
[0033] The feature maps obtained from the decoder at different scales are upsampled to the same size as the original image using bilinear interpolation. These feature maps are then compressed into four channels using convolution and concatenated to obtain a hybrid feature map. This is used as the input to the multi-scale attention module; the input is then subjected to average pooling. Max pooling and multilayer perceptron Combined modules to obtain the coordination coefficient for each channel. It can adaptively adjust the scale information of the segmented object; in order to distribute multi-scale soft attention weights on each pixel, a spatial attention module is also referenced. To obtain pixel-level multi-scale attention coefficients It is with As input, including a and The first layer of the convolutional layer goes through... The activation function connects to the second layer, and finally passes through... The output channels are 4; its final output is:
[0034] (10)
[0035] Using residual connections to facilitate information transfer during training, multi-scale attention modules can obtain the most suitable scale for the network to acquire segmentation targets.
[0036] The application of the segmentation network PCMA-UNet, based on the hybrid attention mechanism of UNet3+, in stroke image processing.
[0037] This invention provides PCMA-UNet, an attention mechanism network based on UNet3+, for medical image segmentation:
[0038] 1) During the encoding stage, UNet3+ introduces Pyramid Squeeze Attention (PSA) after downsampling to obtain feature maps, extracting spatial information at different scales of the feature maps to obtain rich image context information.
[0039] 2) Introduce Coordinate Attention (CA) to emphasize useful features, suppress irrelevant features, improve the network's ability to acquire location and channel information, and reduce the impact of downsampling.
[0040] 3) In the decoding stage, multi-scale attention (MSA) is introduced at the network output to fuse features at different scales and adaptively acquire the importance of features at different scales, thereby further enhancing the expressive power of features.
[0041] Introducing PSA during the encoding stage effectively extracts multi-scale spatial information and establishes long-term channel dependencies. Next, CA is used to highlight more relevant features from the extracted feature maps, and MSA is introduced into the network to aggregate semantic information at different levels. Experimental results show that CAMA-Net outperforms current mainstream network models in segmentation performance and improves segmentation accuracy to a certain extent. Attached Figure Description
[0042] Figure 1 shows the PCMA-UNet network architecture.
[0043] Figure 2. PSA module structure diagram;
[0044] Figure 3. CA module structure diagram;
[0045] Figure 4 MSA module structure diagram;
[0046] Figure 5. Heatmap of attention weights for different attention methods on the AIS dataset;
[0047] Figure 6. Heatmap of attention weights for different attention methods on the ISLES2022 dataset;
[0048] Figure 7. Visual comparison of different methods for stroke segmentation on the AIS dataset;
[0049] Figure 8. Visual comparison of stroke segmentation using different methods on the ISLES2022 dataset. Detailed Implementation
[0050] Example 1: PCMA-UNet, a stroke segmentation network based on a hybrid attention mechanism of UNet3+
[0051] UNet, a widely used medical image segmentation network, has a U-shaped symmetrical structure with upsampling and downsampling. It effectively fuses deep global features and shallow detailed features using skip connections, but simple feature concatenation still leaves a semantic gap. UNet3+ redesigns skip connections using multi-scale features, stacking features from lower-level encoders (including those at the same level) and higher-level decoders into new features. This global skip connection reduces overly redundant feature fusion and lowers the possibility of losing crucial information.
[0052] A hybrid attention network, PCMA-UNet, based on UNet3+, introduces three attention mechanisms: PSA, CA, and MSA. The basic network structure is as follows: Figure 1As shown, the left part of the network is the encoder, which consists of convolutional and pooling layers. During the encoding stage, the image is downsampled to extract image features and generate feature maps at different scales. After PSA (Power Sampling Algorithm), spatial information at different scales can be extracted from the generated feature maps, enabling more accurate integration of contextual feature information and establishing long-term dependencies between multi-scale channel attention. Then, the feature maps are aggregated using CA (Carrier Algorithm) to aggregate features in both horizontal and vertical directions. This captures long-term dependencies in one spatial direction while retaining positional information in the other, allowing the network to have a more accurate grasp of the overall structural information of the target. The decoder consists of convolutional, pooling, and deconvolutional layers. It uses full-scale skip connections to fuse the input feature maps at different scales, gradually restoring the feature maps to their original size to achieve pixel classification. MSA (Multi-Scale Algorithm) is introduced in the decoding stage to enhance the model's ability to learn features of lesion regions at different scales, adapting to lesions of different sizes and shapes, thereby improving segmentation accuracy.
[0053] PCMA-Net employs a U-shaped encoding / decoding structure. During the encoding phase, the encoder downsamples to obtain feature maps at different scales. These feature maps then undergo Pyramid Squeeze Attention (PSA) to extract spatial information at different scales. Next, Coordinate Attention (CA) is applied to preserve the feature map's positional information. In the decoding phase, full-scale skip connections are used to pass the feature maps to the decoder, ensuring that each decoder layer incorporates small-scale, same-scale, and large-scale feature maps from the encoder. The network terminates with Multi-Scale Attention (MSA) to enhance its ability to learn features from lesion regions at different scales. MSA takes the concatenated feature maps from the decoder as input and outputs the network's segmentation result.
[0054] In practice, the input image size is 256×256. After being input into the network, the encoder downsamples the image four times to obtain five feature maps at different scales: 256×256, 128×128, 64×64, 32×32, and 16×16. These feature maps are then processed through PSA and CA, with the image size remaining unchanged. Next, each feature map is upsampled or downsampled to change its size before being passed to the corresponding decoder. Taking the third-layer decoder as an example, the received feature map size is 64×64. The first layer feature map is downsampled by a factor of 4, the second layer by a factor of 2, the third layer remains unchanged, the fourth layer is upsampled by a factor of 2, and the fifth layer is upsampled by a factor of 4. These unified feature maps are then fused to capture feature information across all scales. Finally, the feature maps output from the first four decoder layers are processed through MSA to obtain the network segmentation result.
[0055] 1. Pyramid Squeeze Attention (PSA)
[0056] PSA is a more efficient channel attention mechanism, implemented in four main steps. First, a Squeeze and Concat (SPC) module is used to obtain multi-scale feature maps along the channel directions. Second, the SEWeight module extracts attention from the feature maps at different scales, obtaining the feature weights for each channel. Third, Softmax is used to recalibrate the channel-level feature weights, resulting in multi-scale channel weights. Fourth, the recalibrated weights and corresponding feature maps are obtained through pixel multiplication. Finally, a refined feature map with richer multi-scale feature information is output.
[0057] Using multi-scale convolutional kernels in a pyramid structure can produce different spatial resolutions and depths. By compressing the channel dimension of the input tensor, spatial information at different scales can be effectively extracted from each channel-level feature map. SPC is key to achieving multi-scale feature extraction. This module uses a multi-branch approach to extract spatial information from the input feature map, thereby obtaining richer location information. It splits the input into four parts, extracts features at different scales from the four parts, and concatenates them, which can be represented as:
[0058] (1)
[0059] These represent feature maps at different scales, with Cat representing the concatenation operation. This is the obtained multi-scale feature map. By extracting channel attention weight information from the multi-scale preprocessed feature map, attention weight vectors at different scales are obtained. The attention weight vector can be represented as:
[0060] (2)
[0061] The SEWeight module obtains attention weights from input feature maps at different scales. This allows the PSA module to fuse contextual information at different scales, enhancing useful features and suppressing useless ones. Softmax is used to obtain the recalibrated weights for multi-scale channels, which include all spatial location information and channel attention weights. The recalibrated channel attentions are then concatenated and fused to obtain the entire channel attention vector:
[0062] (3)
[0063] (4)
[0064] This represents the multi-scale channel weights after the attention interaction. The recalibrated multi-scale channel attention weights are then compared with the corresponding scales. The feature maps are multiplied as follows:
[0065] (5)
[0066] This represents a feature map with attention weights across multiple scale channels. Finally, concatenating all scale features yields the final output:
[0067] (6)
[0068] The PSA module integrates multi-scale spatial information and channel attention into each set of features. Its basic structure is as follows: Figure 2 As shown. Therefore, the PSA module enables better information exchange between channel attention and global spatial attention, preventing information loss.
[0069] 2. Coordinate Attention (CA)
[0070] Coordinate attention acquires positional information by calculating the average vector of two spatial directions, thereby aggregating features from different coordinates. It embeds positional information into channel attention, capturing both channel and directional information, allowing the network to focus on larger, more important regions with relatively low computational cost. Its structure... Figure 3 As shown in the figure, it consists of two parts: coordinate information embedding and coordinate attention generation.
[0071] 1) Coordinate information embedding
[0072] Global pooling helps the network capture global information, but it struggles to preserve positional information. Therefore, the CA (Charge-Action) mechanism decomposes global pooling into two parallel one-dimensional feature encodings. Given an input feature map... The global pooling operation is performed along the horizontal direction of the input feature map. and vertical direction By performing pooling operations, relevant location information of the input feature map can be obtained. These are the height and width of the feature map corresponding to the current attention module, respectively. These values change with downsampling, with the height being... The Each channel output is:
[0073] (7)
[0074] Width is The Each channel output is:
[0075] (8)
[0076] The two transformations described above aggregate features along two spatial directions. They generate a pair of direction-aware feature maps, enabling CA to capture dependencies along one spatial path and retain precise location information along the other.
[0077] 2) Coordinate Attention Generation
[0078] To fully utilize the captured location information, the two transformations mentioned above are concatenated in the spatial dimension, and a 1×1 convolution is used to compress the channels. Then, batch normalization and non-linearity are used to encode the spatial information in the vertical and horizontal directions. Next, a 1×1 convolution is used to segment the encoded information, resulting in two feature tensors along the horizontal and vertical directions, and the number of channels in the attention map is adjusted to equal the number of channels in the input feature map. Normalization and weighted fusion are then performed using the Sigmoid function. The final output can be represented as follows:
[0079] (9)
[0080] For the input feature map, and These represent the attention weights in the height and width spatial directions, respectively.
[0081] 3. Multi-Scale Attention
[0082] Common medical image segmentation networks extract features at different scales to handle lesions of varying sizes, fusing these features before prediction, such as UNet, UNet++, and UNet3+. However, for a specific object, feature maps at different scales may have different correlations. To enable the network to automatically determine the proportional weight of each pixel and adaptively provide the input with the appropriate proportion, a multi-scale attention layer is introduced at the end of the network to learn specific weights for feature images at each scale, thus calibrating features at different scales.
[0083] The feature maps obtained from the decoder at different scales are upsampled to the same size as the original image using bilinear interpolation. To improve computational efficiency, convolution is used to compress these feature maps into four channels and then concatenate them to obtain a hybrid feature map. This is used as the input to the multi-scale attention module. The input is then subjected to average pooling. Max pooling and multilayer perceptron Combined modules to obtain the coordination coefficient for each channel. It can adaptively adjust the scale information of the segmented object. To distribute multi-scale soft attention weights on each pixel, a spatial attention module is also referenced. To obtain pixel-level multi-scale attention coefficients It is with As input, including a and The first layer of the convolutional layer goes through... The activation function connects to the second layer, and finally passes through... The output channel is 4. Its final output is:
[0084] (10)
[0085] Using residual connections to facilitate information transfer during training, multi-scale attention modules can obtain the most suitable scale for the network to acquire segmentation targets.
[0086] Example 2: Performance Evaluation of PCMA-UNet, a Stroke Segmentation Network Based on a Hybrid Attention Mechanism of UNet3+
[0087] To evaluate the performance of the proposed network, segmentation task experiments were conducted on two stroke datasets from collaborating hospitals: AIS and ISLES2022. An ablation study was then performed to validate the effectiveness of our proposed network and compare it with state-of-the-art networks.
[0088] 1. Data Source
[0089] 1.1 AIS Dataset
[0090] The stroke dataset AIS comprises 132 patients who underwent diffusion-weighted imaging (DWI) of magnetic resonance imaging (MRI) within 4.5 hours of onset at collaborating hospitals between January and July 2021. A neurologist with over 3 years of experience in head MRI diagnosis used ITK-SNAP software to draw contour lines and labels along the lesion edges on the DWI images, and the data was proofread by two neurologists with over 20 years of experience in head MRI diagnosis. The image size is 256×256. 80% of the samples were used for training, and 20% were used for testing and validation.
[0091] 1.2 ISLES2022 Dataset
[0092] The ISLES2022 dataset (Ischemic Stroke Lesion Segmentation, ISLES) comes from the Ischemic Stroke Lesion Segmentation Challenge and is used to segment acute to subacute stroke lesions. This dataset includes MRI cases of patients at different stages of ischemic stroke. Only the training set of 250 cases and their corresponding labels are publicly available. Each case consists of three different MR imaging modalities: Apparent Diffusion Coefficient (ADC), Diffusion Weighted Imaging (DWI), and Fluid Attenuated Inversion Recovery (FLAIR). Each case contains 22 to 70 2D images. This study uses the DWI modal images from the cases, dividing the training and test sets in an 8:2 ratio.
[0093] 1.3 Parameter Settings
[0094] The experiment used two NVIDIA GeForce RTX 2080Ti GPUs, each with 11 GB of memory. The deep learning framework used was PyTorch 1.4, with 200 batches trained on the model. The learning rate was set to 0.0001, and Adam was selected as the optimization algorithm. The batch size for both training and testing was set to 4.
[0095] 1.4 Loss Function
[0096] The loss function is used to measure the quality of the network output; the smaller the loss function, the better the network fit. The binary cross-entropy (BCE) loss function ensures accurate segmentation of the foreground and background, improving segmentation accuracy and accelerating network convergence. Since stroke lesions are binary, with only lesion regions and background regions, the binary cross-entropy loss function is chosen. Its calculation formula is:
[0097] (11)
[0098] For real labels, Labels for predicted values, The number of training images in a batch. This is the Sigmoid function.
[0099] 1.5 Evaluation Indicators
[0100] Medical image segmentation is a crucial foundation for medical image analysis and processing. However, judging the feasibility of an algorithm subjectively is inherent, necessitating the use of evaluation metrics to quantify the prediction results. To assess network performance, the evaluation metrics used are Intersection over Union (IoU), Dice coefficient (DSC), Sensitivity, and Precision. The calculation formulas are as follows:
[0101] (12)
[0102] (13)
[0103] (14)
[0104] (15)
[0105] Among them, True Positive (TP) is the part where both the actual value and the predicted value are true, True Negative (TN) is the part where both the actual value and the predicted value are false, False Positive (FP) is the part where the actual value is false and the predicted value is true, and False Negative (FN) is the part where the actual value is true and the predicted value is false.
[0106] Both DSC and IoU are used to calculate the similarity between two sets, ranging from [0,1]. The closer the value is to 1, the higher the similarity. Sensitivity calculates the proportion of actual lesion pixels that are judged as lesion pixels, and precision calculates the probability of correctly predicting actual lesions. Both values range from [0,1], and the closer they are to 1, the more accurate the model segmentation.
[0107] 2 Ablation Experiment
[0108] 2.1 Quantitative Analysis
[0109] Under the same experimental environment, UNet3+ was set as the base reference network, and ablation experiments were conducted on two datasets by progressively adding PSA, CA, and MSA modules. As shown in Table 1, the ablation experiment results on the AIS dataset show that introducing only the PSA module improves the network's ability to extract multi-scale features. Further introducing the CA module allows for simultaneous encoding of positional and inter-channel information, enabling the network to focus more on effective regions. The PSA and CA combination module improved IoU, DSC, and SEN by 1.89%, 1.82%, and 4.42%, respectively. The introduction of MSA allows the network to better integrate the original features in the decoder; after adding the MSA module, PRE increased significantly, and combined with the PSA module, PRE improved by 2.99%. The network that successively introduced the PSA, CA, and MSA modules outperformed other networks in all four metrics. IoU and DSC improved by 3.28% and 3.47%, respectively, while Sen and Pre improved by 4.78% and 5.65%, respectively.
[0110] surface Ablation experiments of PCMA-UNet on the AIS dataset
[0111] Method IOU DSC SEN PRE Base (UNet3+) 0.6975±0.25 0.7848±0.24 0.7856±0.23 0.8410±0.18 Base+PSA 0.7085±0.23 0.7956±0.23 0.8213±0.25 0.8500±0.19 Base+PSA+CA 0.7164±0.24 0.8030±0.23 0.8298±0.24 0.8464±0.19 Base+PSA+MSA 0.7106±0.25 0.7965±0.24 0.8006±0.26 0.8709±0.18 Base+PSA+CA+MSA 0.7303±0.23 0.8195±0.25 0.8334±0.25 0.8975±0.17
[0112] The ablation experiments on the ISLES2022 dataset are shown in Table 2. The PCMA-Net segmentation network outperforms other networks across all four metrics. IoU and DSC are improved by 2.85% and 2.48%, respectively, while SEN and PRE are improved by 6.19% and 2.18%, respectively. Notably, the CA module makes a significant contribution to the improvement in SEN; the PSA module combined with the CA module achieves a 5.02% improvement in SEN compared to the baseline network. This indicates that each attention module contributes to improving the network's segmentation performance.
[0113] surface Ablation experiments of PCMA-UNet on the ISLES2022 dataset
[0114] Method IOU DSC SEN PRE Base (UNet3+) 0.6601±0.25 0.7575±0.24 0.7348±0.26 0.8357±0.22 Base+PSA 0.6680±0.24 0.7679±0.23 0.7443±0.24 0.8533±0.21 Base+PSA+CA 0.6850±0.26 0.7785±0.23 0.7850±0.25 0.8406±0.23 Base+PSA+MSA 0.6824±0.24 0.7784±0.23 0.7515±0.23 0.8462±0.21 Base+PSA+CA+MSA 0.6886±0.24 0.7823±0.23 0.7967±0.24 0.8575±0.19
[0115] 2.2 Qualitative Analysis
[0116] Figure 5 and Figure 6The images show attention weight maps on the AIS and ISLES2022 datasets. Red indicates areas where the network focuses more on convolutional weights during feature extraction. The heatmaps show that each attention module contributes to focusing on lesion regions and is better at distinguishing between lesions and non-lesion regions than the baseline network. However, some networks still focus on non-lesion regions, leading to missegmentation. In contrast, the segmentation network using a hybrid of the three attention modules focuses more closely on the target than the methods described above, indicating that PCMA-Net is better at distinguishing lesions and thus achieves better segmentation results.
[0117] 3. Comparative Experiment
[0118] 3.1 Quantitative Analysis
[0119] To verify the superiority of the proposed network, it was compared with commonly used medical image segmentation networks, including UNet, UNet++, Attention UNet, and UNet3+. As shown in Table 3, on the AIS dataset, PCMA-Net achieved the best performance across all four metrics. Specifically, IoU, DSC, and SEN were improved by 2%, 1.97%, and 1.72% respectively compared to the second-best performing Attention UNet. PRE was improved by 2.65% compared to the second-best performing UNet3+. The proposed network has approximately 4M more parameters than UNet3+, but the improvements across the four metrics are 1.28%, 3.47%, 4.78%, and 2.65%, respectively, indicating that the proposed network sacrifices a slight increase in complexity for higher segmentation performance.
[0120] surface Comparison with mainstream methods on the AIS dataset
[0121] Method IOU DSC SEN PRE Params (M) UNet 0.6767±0.23 0.7678±0.24 0.6871±0.25 0.8323±0.18 31.0435 UNet++ 0.7043±0.24 0.7918±0.23 0.7832±0.23 0.8823±0.20 9.1634 Attention UNet 0.7103±0.26 0.7998±0.25 0.8162±0.24 0.8462±0.18 34.8786 UNet3+ 0.6975±0.25 0.7848±0.24 0.7856±0.23 0.8710±0.18 26.9718 PCMA-Net 0.7303±0.23 0.8195±0.25 0.8334±0.25 0.8975±0.17 30.1604
[0122] Comparative experiments were conducted on the ISLES2022 dataset, and the results are shown in the table. The proposed network achieved the best performance across all four metrics. Specifically, IoU, DSC, SEN, and PRE were improved by 2.69%, 2.37%, 4.98%, and 0.42% respectively compared to the second-best performing Attention UNet. The proposed network also performed excellently on two stroke datasets, demonstrating its superior performance and generalization ability.
[0123] surface Comparison with mainstream methods on the ISLES2022 dataset
[0124] Method IOU DSC SEN PRE Params (M) UNet 0.6352±0.29 0.7306±0.28 0.7198±0.28 0.8184±0.22 31.0435 UNet++ 0.6594±0.27 0.7678±0.27 0.7684±0.25 0.8278±0.23 9.1634 Attention UNet 0.6617±0.26 0.7586±0.24 0.7469±0.24 0.8533±0.21 34.8786 UNet3+ 0.6601±0.25 0.7575±0.24 0.7348±0.26 0.8357±0.22 26.9718 PCMA-Net 0.6886±0.24 0.7823±0.23 0.7967±0.24 0.8575±0.19 30.1604
[0125] 3.2 Qualitative Analysis
[0126] Visualization of the splitting results of the two datasets as followsFigure 7 and Figure 8 As shown, Examples 1-5 are images from the AIS dataset test set, and Examples 6-10 are images from the ISLES2022 dataset test set. The green lines mark the actual lesion areas, and the yellow lines mark the lesion areas predicted by each network. The areas within the red boxes represent the parts where other methods performed poorly in segmentation. The proposed network has a stronger ability to encode features related to the main and edge regions of the image, and a stronger ability to transfer subtle and useful features. For example, in Example 2, the proposed network can obtain segmentation results that are closer to the labeled image, while other methods all exhibit oversegmentation. In Example 5, the proposed network can accurately segment two adjacent lesions, while other methods show phenomena such as lesion adhesion or unclear boundary segmentation. This indicates that the proposed network can capture more image features and learn more global information, thereby ensuring the integrity and continuity of medical image segmentation. For large lesions with complex boundaries, such as in Example 10, the proposed network can clearly segment the lesion boundaries, avoiding problems such as segmentation breakage and missegmentation. For extremely small lesions, such as those in Examples 6 and 7, the proposed network can accurately locate the lesions without any missegmentation.
[0127] This invention presents a hybrid attention PCMA-Net network based on an improved version of UNet3+ for stroke image segmentation. PSA (Power-Side Allocation) is introduced during the encoding stage to effectively extract multi-scale spatial information and establish long-term channel dependencies. Then, the extracted feature maps are highlighted using CA (Carrier-Allocation) to emphasize more relevant features, and MSA (Multi-Side Allocation) is introduced into the network to aggregate semantic information at different levels. Experimental results show that CAMA-Net outperforms current mainstream network models in segmentation performance, improving segmentation accuracy to a certain extent.
Claims
1. A segmentation network PCMA-UNet based on a hybrid attention mechanism of UNet3+ applied to stroke image processing, characterized in that: The UNet3+ is used as a framework, the encoder is used for downsampling to obtain feature maps of different scales, the pyramid squeeze attention PSA is used to obtain spatial information of different scales, and the coordinate attention CA is connected to retain the position information of the feature maps; Different scale feature maps output by different scale decoders of each layer are spliced using full-scale skip connection, input into the multi-scale attention MSA, and the network segmentation result is obtained; The input of the segmentation network is the DWI modal image of the stroke data set; The multi-scale attention MSA is to use bilinear interpolation to upsample different scale feature maps obtained by the decoder to the same size as the original image, use convolution to compress these feature maps to 4 channels and splice to obtain a mixed feature map , which is input into the multi-scale attention module; the input is subjected to average pooling , maximum pooling and multi-layer perception combined module to obtain the cooperative coefficient of each channel , which is used to adaptively adjust the scale information of the segmented object; in order to distribute the multi-scale soft attention weight on each pixel, a spatial attention module is additionally cited to obtain the pixel-level multi-scale attention coefficient ; it takes as input, including a and a convolution layer, the first layer is connected to the second layer through activation function, and finally through output channel is 4; The final output is: （10） ； Residual connection is used to facilitate information transmission in training.
2. The segmentation network PCMA-UNet based on the hybrid attention mechanism of UNet3+ applied to stroke image processing according to claim 1, characterized in that: The pyramid squeeze attention PSA comprises: first, a split and splice SPC module is used to obtain multi-scale feature maps in the channel direction; second, an SEWeight module is used to extract attention of the feature maps of different scales to obtain channel feature weights; third, a Softmax is used to recalibrate the channel-level feature weights to obtain multi-scale channel weights; and fourth, a refined feature map with more rich multi-scale feature information is obtained as output by pixel multiplication of the recalibrated weights and the corresponding feature maps.
3. The segmentation network PCMA-UNet based on a hybrid attention mechanism of UNet3+ applied to stroke image processing according to claim 2, characterized in that: The split and splice SPC module comprises: The input is split into four parts, and four different scale features are extracted and spliced, denoted as: （1）； denote feature maps of different scales, respectively, and Cat is a concatenation operation is the obtained multi-scale feature map; The attention weight vector of different scales is obtained by extracting the channel attention weight information from the multi-scale preprocessed feature maps, and the attention weight vector is denoted as: （2）； The SEWeight module: it is to obtain attention weights from input feature maps of different scales The PSA module fuses context information of different scales, enhances useful features, and suppresses useless features; Softmax is used to obtain recalibrated weights of multi-scale channels, which contain all position information in space and attention weights in channels; and the channel attention of feature recalibration is serially fused, so that the whole channel attention vector is obtained: （3）； （4）； representing the multi-scale channel attention weights after attention interaction; multiplying the re-calibrated multi-scale channel attention weights with the feature maps of the corresponding scales are multiplied as: （5）； feature maps representing attention weights with multi-scale channel directions; and finally concatenating all scale features to obtain a final output: （6）。 4. The segmentation network PCMA-UNet based on the hybrid attention mechanism of UNet3+ applied to stroke image processing according to claim 1, 2 or 3, characterized in that: The coordinate attention CA comprises two parts of coordinate information embedding and coordinate attention generation: 1) Coordinate information embedding The CA mechanism decomposes the global pooling into two parallel one-dimensional feature encodings; given the input feature map , the global pooling operation is performed along the horizontal direction and the vertical direction of the input feature map, respectively, to obtain the relevant position information of the input feature map, the height and width of the feature map corresponding to the current attention module, respectively, the values of which change with downsampling, the height being , the output of the th channel is: （7）； width is the first channel output is: （8）； 2) Coordinate attention generation The horizontal direction pooling result and the vertical direction pooling result of the above input feature map are spliced in the spatial dimension, and a 1x1 convolution is used to compress the channel; then batch normalization and nonlinearity are used to encode the spatial information in the vertical and horizontal directions; then a 1x1 convolution is used to split the encoded information to obtain two feature tensors along the horizontal and vertical directions, and the channel of the attention map is adjusted to be equal to the channel number of the input feature map; a Sigmoid function is used for normalization and weighted fusion; and the final output is denoted as follows: （9）； for input feature maps, and denote attention weights in height and width spatial directions, respectively.