Context-aware hybrid attention network for medical image segmentation

Abstract

The present invention belongs to the field of image processing technology, and specifically relates to a context-aware hybrid attention network for medical image segmentation. The network is characterized by being based on a U-Net network architecture, and providing a multi-scale feature extraction module and a multi-scale contextual cross-attention module between the encoder and decoder. Furthermore, an adaptive contextual feature fusion module is provided on each jump connection of the U-Net network architecture. The multi-scale contextual cross-attention module extracts features of different scales from the output of the encoder by the multi-scale feature extraction module and fuses them. In this process, global pooling and grouped convolution are combined to dynamically adjust the attention weights of information at different scales. The adaptive contextual feature fusion module introduces a gating mechanism to fuse low-level features and high-level features in the jump connection, and adjusts the attention weights based on the contributions of the low-level and high-level features. The present invention optimizes information exchange between different scales through the cross-attention mechanism, thereby improving cross-scale feature consistency.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a context-aware hybrid attention network for medical image segmentation. Background Art

[0002] Medical image segmentation technology is crucial for disease diagnosis and treatment planning, but it remains challenging due to factors such as anatomical variability, complex tissue boundaries, and multi-scale structures. However, existing deep learning methods still face the following key issues in medical image segmentation:

[0003] The local receptive field of existing CNN architectures limits their ability to capture global information. Anatomical structures in medical images often have complex spatial relationships, and relying solely on local feature extraction can lead to missegmentation, especially in areas with low contrast or ambiguous structures.

[0004] The existing Transformer architecture’s self-attention mechanism can capture long-distance dependencies, but its computational complexity is O(N 2 ) (where "N" is the number of image pixels) is computationally too expensive in high-resolution medical images to be practical for clinical applications.

[0005] Existing multi-scale feature fusion methods, such as Atrous Spatial Pyramid Pooling (ASPP) and Feature Pyramid Network (FPN), rely on fixed dilation rates or hierarchical features. This static feature fusion strategy lacks adaptability when facing different anatomical structures and is difficult to effectively integrate information at different scales.

[0006] Therefore, the existing technology lacks the ability to dynamically adjust features of different scales during the multi-scale fusion process, making it difficult to perform adaptive optimization under different image structures. In addition, there are a lot of redundant calculations and the computational complexity is relatively high. Summary of the Invention

[0007] The purpose of the present invention is to provide a context-aware hybrid attention network for medical image segmentation, which is used to solve the technical problems in the existing technology, such as the lack of dynamic adjustment capability for features of different scales, difficulty in adaptive optimization under different image structures, and the existence of a large number of redundant calculations and high computational complexity.

[0008] The context-aware hybrid attention network for medical image segmentation is based on the U-Net network architecture, and a multi-scale feature extraction module and a multi-scale context cross-attention module are set between the encoder and the decoder, and an adaptive context feature fusion module is set on each jump connection of the U-Net network architecture. The multi-scale context cross-attention module extracts features of different scales from the output of the encoder by the multi-scale feature extraction module and fuses them. In the process, global pooling and group convolution are combined to dynamically adjust the attention weights of information of different scales; the adaptive context feature fusion module introduces a gating mechanism to fuse low-level features and high-level features in the jump connection, and adjusts the attention weights based on the contributions of low-level features and high-level features.

[0009] Preferably, the multi-scale contextual cross-attention module processes the input features as follows:

[0010] First, the multi-scale feature extraction module performs multi-scale decomposition on the input features to extract features of different scales;

[0011] Then, the cross-attention weights between features of different scales are calculated;

[0012] Then, the fused multi-scale features are obtained by multiplying the cross-attention weights with the input multi-scale features.

[0013] Preferably, the multi-scale feature extraction module includes a dilated spatial pyramid pooling, which uses convolutions with different expansion rates to extract features F d The expression is: F d =Conv k，d (X), Conv k，d Represents a dilated convolution with a dilation rate of d. Features of different scales are connected to obtain the output multi-scale feature F.

[0014] Preferably, calculate the cross attention weight A scale And the fused multi-scale features F refined The calculation formula is:

[0015]

[0016] Among them, F is the multi-scale feature of the input, GlobalPool is the global pooling operation, C global is the output of global pooling, the global feature embedding obtained by adaptive global pooling; is the group convolution operation, k is the number of groups, F local is the output of the grouped convolution to ensure that local features are preserved; Φ represents the connection operation, W1 and b1 represent the calculation of the cross attention weight A respectively. scale The weights and bias terms used, σ represents the sigmoid function.

[0017] Preferably, the fusion process of the adaptive context feature fusion module is as follows:

[0018] First, the attention gate value is calculated for the input features of the skip connection;

[0019] Then, the attention weights are applied to the encoder features and combined with the decoder features to obtain the optimized features.

[0020] Preferably, the calculation formula of the attention gate value is as follows:

[0021] G=σ(W g ·F encoder +b g )

[0022] Where W g and b g Represent the weight coefficient and bias term used to calculate the attention gate value, F encode is the encoder feature, σ represents the sigmoid function, and G represents the attention gate value.

[0023] Preferably, the calculation of the attention gate value also introduces a modulation function ψ(F encoder ), the corresponding calculation formula is as follows:

[0024] G=σ(W g ·F encoder +b g )·ψ(F encoder ).

[0025] Preferably, the attention gate value is applied to the encoder feature F encoder Above, with the decoder feature F decoder Combined, we get the optimized feature F skip , the calculation formula is as follows:

[0026] F skip =G·F encoder +(1-G)·F decoder

[0027] Through the above adaptive feature fusion method, the ACFF module selectively amplifies salient features and suppresses irrelevant features.

[0028] Preferably, the computational complexity of the multi-scale contextual crisscross attention module is: O(N)+O(k 2 C in C out HW), where N = H × W, H and W represent the height and width of the feature map, k represents the number of groups, C represents the number of channels of the feature map, and C in and C outRepresent the number of input channels and output channels respectively.

[0029] The present invention has the following advantages:

[0030] 1. Unlike existing techniques that use dilated convolutions with fixed dilation rates, this paper uses a MS-CA module with learnable attention weights to dynamically adjust the fusion of features at different scales, enabling adaptive optimization under different image structures. This method optimizes information exchange between scales through a cross-attention mechanism, improving cross-scale feature consistency.

[0031] 2. The MS-CA module utilizes global pooling and grouped convolution to reduce computational complexity. During fusion, only the cross-correlations between multi-scale information are calculated, avoiding the need to store and update the global attention matrix during the computation process, thereby reducing computational resource consumption. This improvement also avoids the limitations of the local receptive field of traditional CNNs, making the computation more efficient and more suitable for processing high-resolution medical images.

[0032] 3. Traditional skip connection methods directly pass all low-level features to the decoder, resulting in a large amount of redundant computation. This invention introduces a gating mechanism through the ACFF module to retain only the most contributing features, reducing computational and storage overhead, further reducing computational complexity while ensuring segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of the context-aware hybrid attention network for medical image segmentation in the present invention.

[0034] Figure 2 Schematic diagram of the MS-CA module in the present invention.

[0035] Figure 3 Schematic diagram of the ACFF module in the present invention.

[0036] Figure 4 Schematic diagram of the group convolution mechanism in the present invention.

[0037] Figure 5 The figure compares the segmentation results of the present invention and other existing technologies on the RETOUCH dataset (3D OCT). DETAILED DESCRIPTION

[0038] The following is a further detailed description of the specific implementation methods of the present invention through the description of the embodiments with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.

[0039] like Figure 1-Figure 5As shown, the present invention provides a context-aware hybrid attention network for medical image segmentation, which is based on the U-Net network architecture, and sets a multi-scale feature extraction module and a multi-scale context-aware cross-attention (MS-CA) module between the encoder and the decoder, and sets an adaptive contextual feature fusion (ACFF) module on each jump connection of the U-Net network architecture. The multi-scale contextual cross-attention module extracts features of different scales from the output of the encoder by the multi-scale feature extraction module and fuses them. In the process, global pooling and group convolution are combined to dynamically adjust the attention weights of information of different scales; the adaptive contextual feature fusion module introduces a gating mechanism to fuse low-level features and high-level features in the jump connection, and adjusts the attention weights based on the contributions of low-level features and high-level features.

[0040] Multi-scale Contextual Cross-Attention Module: This module utilizes a multi-scale feature extraction strategy, combined with global pooling and grouped convolution, to dynamically adjust the attention weights for information at different scales. The module's input feature processing flow is as follows.

[0041] First, the multi-scale feature extraction module decomposes the input features into multiple scales to extract features at different scales. This module can employ atrous spatial pyramid pooling (ASPP) to capture multi-scale features by applying convolutions with various dilation rates. However, this module lacks an adaptive mechanism to dynamically weight the most relevant features. The resulting features at different scales are concatenated and output, making it difficult to adapt to different anatomical structures and effectively integrate information at different scales.

[0042] The void space pyramid pooling uses convolution with different expansion rates to extract features F d The expression is: F d =Conv k,d (X), ConV k，d Denotes a dilated convolution with a dilation rate of d. In this embodiment, d = 1, 3, or 5. Features of different scales are concatenated to obtain the output multi-scale feature F.

[0043] Then, the cross attention weight A between features of different scales is calculated scale , and then calculate the fused multi-scale feature F refined , the formula is:

[0044]

[0045] Among them, F is the multi-scale feature of the input, GlobalPool is the global pooling operation, C global is the output of global pooling, the global feature embedding obtained by adaptive global pooling; is the grouped convolution operation (GroupedConv), k is the number of groups, F local is the output of the grouped convolution to ensure that local features are preserved; Φ represents the connection operation, W1 and b1 represent the calculation of the cross attention weight A respectively. scale The weights and bias terms used, σ represents the sigmoid function. Get the cross attention weight A scale After that, the multi-scale feature F is obtained by multiplying the input multi-scale feature points refined .

[0046] In this way, the MS-CA module can adaptively adjust attention allocation according to the target scale characteristics in medical images, while avoiding the limitations of the local receptive field of traditional CNN networks, ensuring that important anatomical structures can be effectively captured at different scales.

[0047] Adaptive Contextual Feature Fusion Module: Traditional skip connection methods typically directly fuse low-level and high-level features without considering their correlation, potentially introducing redundant information or noise. The ACFF module dynamically adjusts the feature fusion method using lightweight gating units. The fusion process of the adaptive contextual feature fusion module is as follows.

[0048] First, the attention gate value is calculated for the input features of the jump connection, and the calculation formula is as follows:

[0049] G=σ(W g ·F encoder +b g )

[0050] Where W g and b g Represent the weight coefficient and bias term used to calculate the attention gate value, F encoder is the encoder feature, σ represents the sigmoid function, and G represents the attention gate value. Based on the above calculation formula, we can also introduce the modulation function ψ(F encoder ) Further optimize the calculation of attention gate value, the calculation formula is as follows:

[0051] G=σ(W g ·F encoder +b g )·ψ(F encoder ).

[0052] Then, the attention gate value is applied to the encoder feature Fencoder Above, with the decoder feature F decode Combined, we get the optimized feature F skip , the calculation formula is as follows:

[0053] F skip =G·F encoder +(1-G)·F decoder

[0054] Through the above adaptive feature fusion method, the ACFF module selectively amplifies salient features and suppresses irrelevant features. This mechanism significantly improves the segmentation accuracy, especially for small or low-contrast structures.

[0055] Computational complexity analysis: The computational complexity of the traditional Transformer module is O(N 2 C), where N = H × W, H and W represent the height and width of the feature map, and C represents the number of channels of the feature map. This is mainly because the self-attention mechanism needs to calculate the global interaction matrix, that is, each pixel is matched with all pixels, so the computational complexity is relatively high. The computational complexity of the MS-CA module mainly includes two parts to extract multi-scale feature information, of which the computational complexity of global pooling is O(N), and the computational complexity of group convolution is O(k 2 C in C out HW), where k represents the number of groups, C in and C out Denote the number of input channels and output channels respectively. After improvement, this scheme significantly reduces the computational complexity by avoiding global interactive calculations at the pixel level.

[0056] On the other hand, the existing technology uses self-attention calculation based on the global correlation matrix, which requires storage and processing of O(N 2 )-scale attention mapping requires more computing resources. The MS-CA module provided by the present invention only calculates the cross-correlation between multi-scale information through local feature fusion, avoiding the storage and updating of the global attention matrix during the calculation process, thereby reducing computing resource consumption.

[0057] Compared with the conventional skip connection method of passing all low-level features to the decoder (which involves redundant calculations), the ACFF module in the present invention only retains the most contributing features, reduces calculation and storage overhead, and further reduces computational complexity.

[0058] Therefore, the improvements of the present invention make the computation more efficient and more suitable for processing high-resolution medical images. At the same time, they ensure that important anatomical structures are enhanced at different scales, thereby effectively enhancing the medical image segmentation effect, especially in lesion areas with blurred boundaries, achieving better segmentation accuracy. In terms of applicability, the present invention is not only applicable to CT and MRI images, but also to ultrasound, X-ray, and pathological images. For example, in ultrasound images, due to the high noise, the attention mechanism of MS-CAM can more effectively enhance key areas and improve the accuracy of lesion identification.

[0059] The test results of the present invention on multiple public medical image datasets show that the segmentation network introduced by MS-CAM improves the Dice score by 4-6% compared with the traditional method, while reducing the computational overhead by about 30%. The comparison of the segmentation results of the present invention and other existing technologies on the RETOUCH dataset (3D OCT) can be seen in the attached figure. Figure 5 This result demonstrates the feasibility and efficiency of MS-CAM in medical segmentation.

[0060] The present invention is described above by way of example in conjunction with the accompanying drawings. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned method. As long as various non-substantial improvements are made using the inventive concept and technical solution of the present invention, or the inventive concept and technical solution are directly applied to other occasions without improvement, they are all within the scope of protection of the present invention.

Claims

1. Context-aware hybrid attention network for medical image segmentation, characterized by: Based on the U-Net network architecture, a multi-scale feature extraction module and a multi-scale contextual cross-attention module are set between the encoder and decoder. An adaptive contextual feature fusion module is set on each jump connection of the U-Net network architecture. The multi-scale contextual cross-attention module extracts features of different scales from the output of the encoder by the multi-scale feature extraction module and fuses them. In the process, global pooling and group convolution are combined to dynamically adjust the attention weights of information at different scales. The adaptive context feature fusion module introduces a gating mechanism to fuse low-level features and high-level features in the skip connection, and adjusts the attention weight based on the contribution of low-level features and high-level features; The multi-scale contextual cross-attention module processes input features as follows: First, the multi-scale feature extraction module performs multi-scale decomposition on the input features to extract features of different scales; Then, the cross-attention weights between features of different scales are calculated; Then, the fused multi-scale features are obtained by multiplying the cross attention weights with the input multi-scale features. The multi-scale feature extraction module includes void space pyramid pooling, which uses convolution with different expansion rates to extract features F. d The expression is: F d =Conv k，d (X), Conv k，d Represents a dilated convolution with a dilation rate of d. Features of different scales are concatenated to obtain the output multi-scale feature F. Calculate the cross attention weight A scale And the fused multi-scale features F refined The calculation formula is: Among them, F is the multi-scale feature of the input, GlobalPool is the global pooling operation, C global is the output of global pooling, the global feature embedding obtained by adaptive global pooling; is the group convolution operation, k is the number of groups, F local is the output of the grouped convolution to ensure that local features are preserved; Φ represents the connection operation, W1 and b1 represent the calculation of the cross attention weight A respectively. scale The weights and bias terms used, σ represents the sigmoid function; The fusion process of the adaptive context feature fusion module is as follows: First, the attention gate value is calculated for the input features of the skip connection; Then, the attention weights are applied to the encoder features and combined with the decoder features to obtain the optimized features.

2. The context-aware hybrid attention network for medical image segmentation according to claim 1, characterized in that: The calculation formula of the attention gate value is as follows: G=σ(W g ·F encoder +b g ) Where W g and b g Represent the weight coefficient and bias term used to calculate the attention gate value, F encoder is the encoder feature, σ represents the sigmoid function, and G represents the attention gate value.

3. The context-aware hybrid attention network for medical image segmentation according to claim 2, characterized in that: The modulation function ψ(F encoder ), the corresponding calculation formula is as follows: G=σ(W g ·F encoder +b g )·ψ(F encoder )。 4. The context-aware hybrid attention network for medical image segmentation according to claim 2 or 3, characterized in that: Apply the attention gate value to the encoder feature F encoder Above, with the decoder feature F decoder Combined, we get the optimized feature F skip , the calculation formula is as follows: F skip =G·F encoder +(1-G)·F decoder 。 5. The context-aware hybrid attention network for medical image segmentation according to claim 1, characterized in that: The computational complexity of the multi-scale contextual crisscross attention module is: O(N)+O(k 2 C in C out HW), where N = H × W, H and W represent the height and width of the feature map, k represents the number of groups, C represents the number of channels of the feature map, and C in and C out Represent the number of input channels and output channels respectively.

Images