Context aware hybrid attention network for medical image segmentation

Abstract

The invention belongs to the technical field of image processing, and particularly relates to a context-aware hybrid attention network for medical image segmentation, which is characterized in that a multi-scale feature extraction module and a multi-scale context cross attention module are arranged between an encoder and a decoder based on a U-Net network architecture; a self-adaptive context feature fusion module is arranged on each jump connection of the U-Net network architecture, a multi-scale context cross attention module is used for fusing features of different scales extracted from the output of the encoder by the multi-scale feature extraction module, and global pooling and grouping convolution are combined in the process; dynamically adjusting attention weights of different scales of information; and the adaptive context feature fusion module introduces a gating mechanism to fuse low-level features and high-level features in the jump connection, and adjusts the attention weight based on the contribution of the low-level features and the high-level features. According to the method, information exchange is optimized through a cross attention mechanism among different scales, and the cross-scale feature consistency is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a context-aware hybrid attention network for medical image segmentation. Background Art

[0002] Medical image segmentation technology is crucial for disease diagnosis and treatment planning, but it remains challenging due to factors such as anatomical variability, complex tissue boundaries, and multi-scale structures. However, existing deep learning methods still face the following key problems in medical image segmentation:

[0003] The local receptive fields of existing CNN architectures limit their ability to capture global information. The anatomical structures in medical images often have complex spatial relationships, and relying solely on local feature extraction may lead to missegmentation, especially in regions with low contrast or blurred structures.

[0004] The self-attention mechanism of existing Transformer architectures can capture long-range dependencies, but its computational complexity is O(N 2 )(where "N" is the number of image pixels), which is too computationally expensive for high-resolution medical images and not suitable for practical clinical applications.

[0005] Existing multi-scale feature fusion methods, such as Atrous Spatial Pyramid Pooling (ASPP) and Feature Pyramid Network (FPN), rely on fixed dilation rates or hierarchical features. This static feature fusion strategy lacks adaptability when faced with different anatomical structures and is difficult to effectively integrate information at different scales.

[0006] Therefore, the existing technology lacks the ability to dynamically adjust different-scale features during the multi-scale fusion process, is difficult to perform adaptive optimization under different image structures, and has a large amount of redundant computation and high computational complexity. Summary of the Invention

[0007] The purpose of the present invention is to provide a context-aware hybrid attention network for medical image segmentation, which is used to solve the technical problems in the existing technology, such as the lack of the ability to dynamically adjust different-scale features, the difficulty in performing adaptive optimization under different image structures, and the existence of a large amount of redundant computation and high computational complexity.

[0008] The context-aware hybrid attention network for medical image segmentation is based on the U-Net network architecture, and a multi-scale feature extraction module and a multi-scale context cross-attention module are set between the encoder and the decoder, while an adaptive context feature fusion module is set on each skip connection of the U-Net network architecture. The multi-scale context cross-attention module fuses the features of different scales extracted from the output of the encoder by the multi-scale feature extraction module, combines global pooling and grouped convolution in the process, and dynamically adjusts the attention weights of different scale information; the adaptive context feature fusion module introduces a gating mechanism to fuse the low-level features and high-level features in the skip connection, and adjusts the attention weights based on the contributions of the low-level features and high-level features.

[0009] Preferably, the processing flow of the multi-scale context cross-attention module for the input features is as follows:

[0010] First, the multi-scale feature extraction module performs multi-scale decomposition on the input features to extract features of different scales;

[0011] Then, the cross-attention weights between the features of different scales are calculated;

[0012] Subsequently, the fused multi-scale features are obtained by multiplying the cross-attention weights with the input multi-scale feature points.

[0013] Preferably, the multi-scale feature extraction module includes atrous spatial pyramid pooling, and convolutions with different dilation rates are used to extract the feature F d The expression of which is: F d = Conv k，d (X), Conv k，d represents the atrous convolution with a dilation rate of d, and the features of different scales are connected to obtain the output multi-scale feature F.

[0014] Preferably, the calculation formula for calculating the cross-attention weight A scale and the fused multi-scale feature F refined is:

[0015]

[0016] where F is the input multi-scale feature, GlobalPool is the global pooling operation, C global is the output of the global pooling, the global feature embedding obtained through adaptive global pooling; is the grouped convolution operation, k is the number of groups, F local is the output of the grouped convolution to ensure that the local features are retained; Φ represents the concatenation operation, and W1 and b1 respectively represent the weight and bias term used to calculate the cross-attention weight A scale and σ represents the sigmoid function.

[0017] Preferably, the fusion process of the adaptive context feature fusion module is as follows:

[0018] First, calculate the attention gating value for the input features of the skip connection;

[0019] Then, apply the attention weight to the encoder features and combine them with the decoder features to obtain the optimized features.

[0020] Preferably, the calculation formula for the attention gating value is as follows:

[0021] G = σ(W g ·F encoder + b g )

[0022] where W g and b g respectively represent the weight coefficient and bias term used to calculate the attention gating value, F encode is the encoder feature, σ represents the sigmoid function, and G represents the attention gating value.

[0023] Preferably, a modulation function ψ(F encoder ) is also introduced to calculate the attention gating value, and the corresponding calculation formula is as follows:

[0024] G = σ(W g ·F encoder + b g )·ψ(F encoder )

[0025] Preferably, apply the attention gating value to the encoder feature F encoder and combine it with the decoder feature F decoder to obtain the optimized feature F skip , and the calculation formula is as follows:

[0026] F skip = G·F encoder + (1 - G)·F decoder

[0027] Through the above adaptive feature fusion method, the ACFF module selectively amplifies significant features and suppresses irrelevant features.

[0028] Preferably, the computational complexity of the multi-scale context cross-attention module is: O(N) + O(k 2 C in C out HW), where N = H × W, H and W represent the height and width of the feature map, k represents the number of groups, C represents the number of channels of the feature map, C in and C outrespectively represent the number of input channels and the number of output channels.

[0029] The present invention has the following advantages:

[0030] 1. Different from the existing technology that uses dilated convolutions with a fixed dilation rate, the present invention uses the MS-CA module to dynamically adjust the fusion method of features at different scales with learnable attention weights, enabling it to adaptively optimize under different image structures. This method optimizes information exchange through a cross-attention mechanism between different scales, enhancing the cross-scale feature consistency.

[0031] 2. The MS-CA module uses global pooling and grouped convolutions to reduce the computational complexity. When fusing, it only calculates the cross-correlation between multi-scale information, avoiding storing and updating the global attention matrix during the calculation process, thereby reducing the consumption of computing resources. This improvement also avoids the limitation of the local receptive field of traditional CNNs, making the calculation more efficient and more suitable for the processing of high-resolution medical images.

[0032] 3. Traditional skip connection methods directly transfer all low-level features to the decoder, resulting in a large amount of redundant calculations. The present invention introduces a gating mechanism through the ACFF module to only retain the most contributing features, reducing the computational and storage overheads, further reducing the computational complexity, and at the same time ensuring the segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a schematic diagram of the context-aware hybrid attention network for medical image segmentation of the present invention.

[0034] Figure 2 is a schematic diagram of the MS-CA module in the present invention.

[0035] Figure 3 is a schematic diagram of the ACFF module in the present invention.

[0036] Figure 4 is a schematic diagram of the grouped convolution mechanism in the present invention.

[0037] Figure 5 is a comparison diagram of the segmentation results of the present invention and other existing technologies on the RETOUCH dataset (3D OCT). DETAILED DESCRIPTION OF THE INVENTION

[0038] The following is a more detailed description of the specific implementation manner of the present invention with reference to the accompanying drawings and through the description of embodiments, in order to help those skilled in the art have a more complete, accurate, and in-depth understanding of the inventive concept and technical solution of the present invention.

[0039] As Figures 1 - 5As shown in the figure, the present invention provides a context-aware hybrid attention network for medical image segmentation. The context-aware hybrid attention network is based on the U-Net network architecture, and a multi-scale feature extraction module and a multi-scale context-aware cross-attention (MS-CA) module are set between the encoder and the decoder. An adaptive contextual feature fusion (ACFF) module is set on each skip connection of the U-Net network architecture. The multi-scale context-aware cross-attention module fuses the features of different scales extracted from the output of the encoder by the multi-scale feature extraction module, combines global pooling and grouped convolution in the process, and dynamically adjusts the attention weights of different scale information; the adaptive contextual feature fusion module introduces a gating mechanism to fuse the low-level features and high-level features in the skip connection, and adjusts the attention weights based on the contributions of the low-level features and high-level features.

[0040] Multi-scale context-aware cross-attention module: This module uses a multi-scale feature extraction strategy, combines global pooling and grouped convolution to dynamically adjust the attention weights of different scale information. The specific processing flow of the input features of this module is as follows.

[0041] First, the multi-scale feature extraction module performs multi-scale decomposition on the input features to extract features of different scales. The multi-scale feature extraction module can use atrous spatial pyramid pooling (ASPP), and captures multi-scale features by applying convolutions with multiple dilation rates. However, this module lacks an adaptive mechanism to dynamically weight the most relevant features, and the obtained features of different scales are concatenated and output. Therefore, it lacks adaptability when facing different anatomical structures and is difficult to effectively integrate information of different scales.

[0042] Atrous spatial pyramid pooling extracts features F using convolutions with different dilation rates d The expression of is: F d = Conv k,d (X), ConV k，d represents the atrous convolution with a dilation rate of d. In this embodiment, d = 1, 3, 5. The features of different scales are concatenated to obtain the output multi-scale feature F.

[0043] Then, calculate the cross-attention weight A scale between the features of different scales, and then calculate the fused multi-scale feature F refined . The formula is:

[0044]

[0045] Among them, F is the input multi-scale feature, GlobalPool is the global pooling operation, and C global is the output of global pooling, the global feature embedding obtained through adaptive global pooling; is the grouped convolution operation (GroupedConv), k is the number of groups, and F local is the output of grouped convolution to ensure that local features are retained; Φ represents the concatenation operation, and W1 and b1 respectively represent the weights and bias terms used to calculate the cross-attention weight A scale , and σ represents the sigmoid function. After obtaining the cross-attention weight A scale , it is multiplied by the input multi-scale feature points to obtain the fused multi-scale feature F refined .

[0046] In this way, the MS-CA module can adaptively adjust the attention distribution according to the target scale characteristics in medical images, while avoiding the limitation of the local receptive field of traditional CNN networks, ensuring that important anatomical structures can be effectively captured at different scales.

[0047] Adaptive Context Feature Fusion Module: Traditional skip connection methods usually directly fuse low-level features and high-level features without considering their correlation, which may introduce redundant information or noise. The ACFF module dynamically adjusts the feature fusion method through lightweight gating units. The fusion process of the adaptive context feature fusion module is as follows.

[0048] First, calculate the attention gating value for the input features of the skip connection. The calculation formula is as follows:

[0049] G = σ(W g ·F encoder + b g )

[0050] where W g and b g respectively represent the weight coefficient and bias term used to calculate the attention gating value, F encoder is the encoder feature, σ represents the sigmoid function, and G represents the attention gating value. Based on the above calculation formula, the attention gating value can be further optimized by introducing the modulation function ψ(F encoder ). The calculation formula is as follows:

[0051] G = σ(W g ·F encoder + b g )·ψ(F encoder ).

[0052] Then, apply the attention gating value to the encoder feature Fencoder is combined with the decoder feature F decode to obtain the optimized feature F skip , and the calculation formula is as follows:

[0053] F skip = G·F encoder +(1 - G)·F decoder

[0054] Through the above adaptive feature fusion method, the ACFF module selectively amplifies significant features and suppresses irrelevant features, and this mechanism significantly improves the segmentation accuracy, especially for small or low-contrast structures.

[0055] Computational complexity analysis: The computational complexity of the traditional Transformer module is O(N 2 C), where N = H×W, H and W represent the height and width of the feature map, and C represents the number of channels of the feature map. This is mainly because the self-attention mechanism needs to calculate the global interaction matrix, that is, each pixel is pairwise matched with all pixels, so the computational complexity is relatively large. The computational complexity of the MS-CA module mainly includes two parts to extract the information of multi-scale features. Among them, the computational complexity of global pooling is O(N), and the computational complexity of grouped convolution is O(k 2 C in C out HW), where k represents the number of groups, C in and C out represent the number of input channels and the number of output channels respectively. After improvement, since this solution avoids the global interaction calculation at the pixel level, the computational complexity is significantly reduced.

[0056] On the other hand, the existing technology uses self-attention calculation based on the global correlation matrix, which needs to store and process the attention map of the O(N 2 ) scale, and consumes a lot of computing resources. The MS-CA module provided by the present invention calculates only the cross-correlation between multi-scale information through the local feature fusion method, and avoids storing and updating the global attention matrix during the calculation process, thereby reducing the consumption of computing resources.

[0057] Compared with the conventional skip connection that transmits all low-level features to the decoder (there is redundant calculation), the ACFF module in the present invention only retains the most contributing features, reduces the computational and storage overheads, and further reduces the computational complexity.

[0058] Therefore, the improvement of the present invention makes this calculation more efficient and more suitable for the processing of high-resolution medical images. At the same time, it ensures that important anatomical structures can be enhanced at different scales, thereby effectively enhancing the medical image segmentation effect. Especially in the lesion area with blurred boundaries, better segmentation accuracy can be obtained. In terms of applicability, the present invention is not only applicable to CT and MRI images, but also can be used for ultrasound, X-ray, pathological images, etc. For example, in ultrasound images, due to the large amount of noise, the attention mechanism of MS-CAM can more effectively enhance the key areas and improve the accuracy of lesion recognition.

[0059] The tests of the present invention on multiple publicly available medical image datasets show that the segmentation network incorporating MS-CAM has a 4-6% improvement in the Dice score compared with traditional methods, and at the same time, the computational cost is reduced by about 30%. Among them, the comparison graph of the segmentation results of the present invention and other existing technologies applied to the RETOUCH dataset (3D OCT) can be seen in the appendix Figure 5 . This result proves the feasibility and efficiency of MS-CAM in medical segmentation.

[0060] The present invention has been described exemplarily above in conjunction with the accompanying drawings. Obviously, the specific implementation of the present invention is not limited by the above-mentioned manner. As long as various non-substantive improvements are made by adopting the inventive concept and technical solution of the present invention, or the inventive concept and technical solution of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. Context-Aware Hybrid Attention Network for Medical Image Segmentation, characterized in that: Based on the U-Net network architecture, a multi-scale feature extraction module and a multi-scale context cross-attention module are set between the encoder and the decoder, and an adaptive context feature fusion module is set on each skip connection of the U-Net network architecture. The multi-scale context cross-attention module fuses the features of different scales extracted from the output of the encoder by the multi-scale feature extraction module, combines global pooling and grouped convolution in the process, and dynamically adjusts the attention weights of different scale information; The adaptive context feature fusion module introduces a gating mechanism to fuse the low-level features and high-level features in the skip connection, and adjusts the attention weights based on the contributions of the low-level features and high-level features.

2. The context-aware hybrid attention network for medical image segmentation according to claim 1, wherein: The processing flow of the multi-scale context cross-attention module for the input features is as follows: First, the multi-scale feature extraction module performs multi-scale decomposition on the input features to extract features of different scales; Then, calculate the cross-attention weights between the features of different scales; Subsequently, the fused multi-scale features are obtained by multiplying the cross-attention weights with the input multi-scale feature points.

3. The context-aware hybrid attention network for medical image segmentation according to claim 2, characterized in that: The multi-scale feature extraction module includes atrous spatial pyramid pooling, and convolutions with different dilation rates are used to extract the feature F d The expression of F d is: F k，d = Conv k，d (X), where Conv k，d represents an atrous convolution with a dilation rate of d. After connecting features of different scales, the output multi-scale feature F is obtained.

4. The context-aware hybrid attention network for medical image segmentation according to claim 3, wherein: Calculate the cross-attention weight A scale and the fused multi-scale feature F refined The calculation formula is as follows: Among them, F is the input multi-scale feature, GlobalPool is the global pooling operation, and C global is the output of the global pooling, the global feature embedding obtained through adaptive global pooling; is the grouped convolution operation, k is the number of groups, and F local is the output of the grouped convolution to ensure that local features are retained; Φ represents the concatenation operation, and W1 and b1 respectively represent the weights and bias terms used to calculate the cross-attention weight A scale The sigmoid function is represented by σ.

5. The context-aware hybrid attention network for medical image segmentation according to claim 1, wherein: The fusion process of the adaptive context feature fusion module is as follows: First, calculate the attention gating value for the input features of the skip connection; Then, apply the attention weights to the encoder features and combine them with the decoder features to obtain the optimized features.

6. The context-aware hybrid attention network for medical image segmentation according to claim 5, characterized in that: The calculation formula of the attention gating value is as follows: G = σ(W g · F encoder + b g ) Among which W g and b g represent the weight coefficient and bias term used to calculate the attention gating value respectively, F encoder is the encoder feature, σ represents the sigmoid function, and G represents the attention gating value.

7. The context-aware hybrid attention network for medical image segmentation according to claim 6, wherein: Calculating the attention gating value also introduces a modulation function ψ(F encoder ), and the corresponding calculation formula is as follows: G = σ(W g ·F encoder + b g )·ψ(F encoder )。 8. The context-aware hybrid attention network for medical image segmentation according to claim 6 or 7, characterized in that: Apply the attention gating value to the encoder feature F encoder and combine it with the decoder feature F decoder to obtain the optimized feature F skip , and the calculation formula is as follows: F skip = G·F encoder +(1 - G)·F decoder Through the above adaptive feature fusion method, the ACFF module selectively amplifies the significant features and suppresses the irrelevant features.

9. The context-aware hybrid attention network for medical image segmentation according to claim 1, wherein: The computational complexity of the multi-scale context cross-attention module is: O(N)+O(k 2 C in C out HW), where N = H×W, H and W represent the height and width of the feature map, k represents the number of groups, C represents the number of channels of the feature map, C in and C out represent the number of input channels and the number of output channels respectively.

Images