Heart medical image segmentation method based on improved U-Net

Abstract

The invention discloses a heart medical image segmentation method based on improved U-Net, relates to a medical image segmentation method, and provides a multi-dimensional context attention module (MDCA) and a reverse feature modulation module (RFM) through an improved U-Net model for technical challenges existing in heart MRI image segmentation. The modules not only enhance the segmentation precision of the model on a complex heart anatomical structure, but also effectively solve the problem of class imbalance in multi-class segmentation. By means of the MDCA module, the model can better fuse global and local information, so that the detail capturing capability of the heart image is improved. And the RFM module ensures the high-precision performance when the boundary blur and small-area region is processed through dynamic weight adjustment. Through the improvement, effective technical support is provided for automatic diagnosis and evaluation of cardiovascular diseases, and meanwhile, a solid foundation is laid for further development of the field of medical image analysis in the future.

Description

Technical Field

[0001] The present invention relates to a medical image segmentation method, and in particular to a cardiac medical image segmentation method based on an improved U-Net. Background Art

[0002] Cardiac image segmentation plays a key role in the diagnosis, treatment, and monitoring of cardiovascular diseases. Accurate segmentation of cardiac anatomical structures, such as the left ventricle, right ventricle, and myocardium, is crucial for quantitative analysis of cardiac function and identification of lesions. Cardiac magnetic resonance imaging (MRI) has become a commonly used imaging technology in clinical practice due to its high resolution and non-invasive characteristics. However, cardiac MRI image segmentation faces multiple challenges, including complex cardiac anatomical structures, individual differences, the influence of pathological conditions, and image noise and artifacts. These problems make the manual segmentation process time-consuming and laborious, further promoting the development of automated segmentation methods.

[0003] As a classic convolutional neural network structure, U-Net has been widely used in medical image segmentation. Its unique encoder-decoder architecture and skip connection mechanism enable it to effectively capture global information and restore high-resolution features. In the task of cardiac image segmentation, although U-Net performs well, it still has limitations in dealing with complex structures and multi-scale features, especially in the decoding stage, which may lead to the loss of detailed information and affect the accuracy of cardiac boundary segmentation. Existing models do not pay enough attention to category imbalance when dealing with multi-category segmentation tasks, resulting in unsatisfactory segmentation results in small areas or cardiac structures with blurred boundaries (such as the myocardium). Summary of the invention

[0004] The purpose of the present invention is to provide a cardiac medical image segmentation method based on an improved U-Net. The present invention improves the segmentation accuracy of complex cardiac structures and enhances the performance of the model in multi-category segmentation tasks by proposing a multidimensional context attention module (MDCA) and a reverse feature modulation module (RFM). This not only provides new technical support for automated cardiovascular disease diagnosis, but also lays a foundation for further promoting research in the field of medical image analysis.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A cardiac medical image segmentation method based on an improved U-Net, the method comprising a multidimensional context awareness module (MDCA), a reverse feature modulation module (RFM), and a model architecture;

[0007] Multidimensional Context-Aware Module (MDCA) is proposed in the high-level skip connection of the U-Net model. This module can focus on both global and local context information. Capturing this information helps to understand and segment the cardiac structure more accurately. The MDCA module measures the correlation between features in the horizontal and vertical directions by generating a spatial attention map. Query, Key, and Value represent Figure 1 In the query vector, key vector, and value vector, Query, Key, and Value are generated by generating query, key, and value feature maps from the input feature map through a 1×1 convolution operation, which is expressed as follows:

[0008] Query = W q ×X,Key=W k ×X,Value=W v ×X (1)

[0009] Where: X is the input feature map, W q , W k , W v They are 1×1 convolution weight matrices used to generate query, key, and value feature maps, respectively; enabling the model to provide higher segmentation accuracy and robustness when processing complex cardiac structures;

[0010] The reverse feature modulation module (RFM) generates a reverse mask and adaptively adjusts the feature weights, so that the model can maintain high accuracy in difficult-to-segment areas such as the myocardium. It effectively improves the accuracy of multi-category segmentation tasks, especially when dealing with areas with unbalanced categories or blurred boundaries.

[0011] The model architecture retains the symmetric encoder-decoder structure of U-Net, which enhances the segmentation capability of complex cardiac anatomical structures. In the encoder, multiple 3x3 convolutional layers are used alternately with ReLU activation functions to extract multi-layer features of the input image, and batch normalization (BatchNorm) is used to accelerate training and improve stability. The specific convolution and pooling operations are expressed by the following formula:

[0012] X l+1 =MaxPool(ReLU(BatchNorm(Conv(X l )))) (9)

[0013] Among them, X l The input feature map of the lth layer is extracted through 3x3 convolution, and then nonlinearly transformed through batch normalization and ReLU activation function. Finally, the spatial resolution of the feature map is gradually reduced through maximum pooling (MaxPool) to capture more global information while reducing the computational burden;

[0014] By combining the MDCA and RFM modules, the improved U-Net model not only has better performance in multi-scale feature capture and global information fusion, but also solves the problems of detail loss and category imbalance.

[0015] In the cardiac medical image segmentation method based on improved U-Net, the calculation of the correlation matrix of the multidimensional context-aware module (MDCA) generates a correlation matrix A between features by calculating the dot product of the query and the key:

[0016]

[0017] Here, the softmax function is used to normalize the correlation matrix A so that it reflects the relative importance of features, d k is the dimension scaling factor of Key, which is used to prevent the dot product result from being too large. The attention matrix A is then applied to Value to generate the weighted output:

[0018] O=A×Value (3)

[0019] This operation can help the model capture global and local contextual information, especially for accurate segmentation of complex heart boundaries.

[0020] The cardiac medical image segmentation method based on the improved U-Net, the reverse feature modulation module (RFM) model generates a rough segmentation mask M for each category. i , where i represents the category index; the coarse mask is generated by Sigmoid activation:

[0021] M i =Sigmoid(f coarse (X)) (5)

[0022] Among them, f coarse Represents the rough segmentation result generated by the network; then the reverse mask R of each category is generated by the reverse operation i :

[0023] R i =1-M i (6)

[0024] This reverse mask R i It allows the model to focus more accurately on areas outside the current category, further improving segmentation accuracy.

[0025] The cardiac medical image segmentation method based on improved U-Net, the model architecture convolution and pooling operations are expressed by the following formula:

[0026] X l+1 =MaxPool(ReLU(BatchNorm(Conv(X l )))) (9)

[0027] Among them, X l The input feature map of the lth layer is extracted through 3x3 convolution, and then nonlinearly transformed through batch normalization and ReLU activation function. Finally, the spatial resolution of the feature map is gradually reduced through maximum pooling (MaxPool) to capture more global information while reducing the computational burden.

[0028] The advantages and effects of the present invention are:

[0029] In view of the technical challenges in cardiac MRI image segmentation, the present invention proposes a multi-dimensional context attention module (MDCA) and a reverse feature modulation module (RFM) by improving the U-Net model. These modules not only enhance the segmentation accuracy of the model for complex cardiac anatomical structures, but also effectively solve the problem of category imbalance in multi-category segmentation. With the help of the MDCA module, the model can better integrate global and local information, thereby improving the ability to capture details of cardiac images. The RFM module ensures high-precision performance when processing blurred boundaries and small areas through dynamic weight adjustment. Through these improvements, the present invention provides effective technical support for the automated diagnosis and evaluation of cardiovascular diseases, and lays a solid foundation for the further development of the field of medical image analysis in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a structural diagram of the multi-dimensional context perception module of the present invention;

[0031] Figure 2 This is a structural diagram of the reverse characteristic modulation module of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be described in detail below with reference to the embodiments shown in the accompanying drawings.

[0033] The present invention is constituted as follows:

[0034] 1. Multi-dimensional Context Awareness Module (MDCA)

[0035] In cardiac structure analysis, it is crucial to accurately capture complex anatomical features. In this paper, to improve this capability, we propose a multi-dimensional context-aware module (MDCA) in the high-level skip connection of the U-Net model, which can simultaneously focus on global and local context information. Capturing this information helps to more accurately understand and segment cardiac structures. The MDCA module measures the correlation between features in the horizontal and vertical directions by generating a spatial attention map. Query, Key, and Value represent Figure 1 In the query vector, key vector, and value vector, Query, Key, and Value are generated by generating query, key, and value feature maps from the input feature map through a 1×1 convolution operation, which is expressed as follows:

[0036] Query = W q ×X,Key=W k ×X,Value=W v ×X (1)

[0037] Where: X is the input feature map, W q , W k , W v They are 1×1 convolution weight matrices used to generate query, key, and value feature maps. The correlation matrix is ​​calculated by calculating the dot product of query and key to generate the correlation matrix A between features:

[0038]

[0039] Here, the softmax function is used to normalize the correlation matrix A so that it reflects the relative importance of features, d k is the dimension scaling factor of Key, which is used to prevent the dot product result from being too large. The attention matrix A is then applied to Value to generate the weighted output:

[0040] O=A×Value (3)

[0041] This operation can help the model capture global and local context information, especially for the accurate segmentation of complex heart boundaries. In order to retain the original information in the input features, the MDCA module uses a residual connection to add the input feature X directly to the output feature O to reduce feature loss:

[0042] Y=O+γ×X (4)

[0043] Among them, γ is a learnable weight parameter used to adjust the strength of the residual connection. This helps to retain the key details of the input feature map while reducing the loss of important information. The MDCA module can better capture complex cardiac anatomical structures by enhancing the spatial context awareness of the model. Unlike the skip connection of traditional U-Net, MDCA retains more fine-grained features in the decoding stage, significantly improving image resolution and detail processing capabilities. This enables the model to provide higher segmentation accuracy and robustness when processing complex cardiac structures.

[0044] 2. Reverse Feature Modulation Module (RFM)

[0045] In this paper, we also propose a reverse feature modulation module (RFM) to focus on solving the class imbalance problem in multi-class segmentation tasks. RFM generates a reverse mask and adaptively adjusts the feature weights to ensure that the model maintains high accuracy in difficult-to-segment areas such as the myocardium. First, the model generates a rough segmentation mask M for each class. i , where i represents the category index. The rough mask is generated by Sigmoid activation:

[0046] M i =Sigmoid(f coarse (X)) (5)

[0047] Among them, f coarse Represents the rough segmentation result generated by the network. Then the reverse mask R of each category is generated by the reverse operation i :

[0048] R i =1-M i (6)

[0049] This reverse mask R i This allows the model to focus more accurately on areas outside the current category, further improving segmentation accuracy. In order to adaptively adjust features of different categories, the RFM module combines a dynamic weight generator that generates a dynamic weight w for each category from the input features through a global pooling operation. i The dynamic weights adaptively weight the features of each category according to the category, ensuring that the model is more sensitive to key areas in complex anatomical structures. For each category, the features are element-wise multiplied with the inverse mask and the dynamic weights:

[0050] O i =w i ×R i ×Conv(X) (7)

[0051] In this way, the model enhances these features to make it perform better in difficult areas. i are combined (by averaging or summing) to generate the final output feature O final To prevent the loss of important original information, the RFM module also introduces a residual connection to add the input feature X back to the final output:

[0052] Y=O final +X (8)

[0053] This residual connection ensures that the key details of the input are preserved. By combining dynamic weights and inverse masks, the RFM module effectively improves the accuracy in multi-category segmentation tasks, especially when dealing with areas with unbalanced categories or blurred boundaries.

[0054] 3. Model Architecture

[0055] In the present invention, the improved model retains the symmetrical encoder-decoder structure of U-Net and enhances the segmentation capability of complex cardiac anatomical structures. In the encoder, multiple 3x3 convolutional layers are used to alternate with ReLU activation functions to extract multi-layer features of the input image, and batch normalization (BatchNorm) is used to accelerate training and improve stability. The specific convolution and pooling operations can be expressed by the following formula:

[0056] X l+1 =MaxPool(ReLU(BatchNorm(Conv(X l )))) (9)

[0057] Among them, X l The input feature map of the lth layer is extracted through 3x3 convolution, and then nonlinearly transformed through batch normalization and ReLU activation function. Finally, the spatial resolution of the feature map is gradually reduced through maximum pooling (MaxPool) to capture more global information while reducing the computational burden. In the decoder part, upsampling is performed through 3x3 deconvolution to restore the spatial resolution of the image. The deconvolution operation formula is as follows:

[0058] X up =ConvTranspose(X low ) (10)

[0059] After each upsampling, the feature map of the upper decoder is concatenated with the corresponding encoder feature map. This operation can be expressed by the following formula:

[0060] X concat =Concat(X encoder ,X decoder) (11)

[0061] The features of the encoder and decoder are combined to ensure that important local information is not lost when recovering high-resolution features. To further enhance the segmentation performance, this paper adds a multi-dimensional contextual attention module (MDCA) to the skip connection. We choose to add the MDCA module to the higher-level skip connections because the feature maps of these levels contain richer global and local information. By enhancing the fusion of this information through MDCA, the complex structures in cardiac images can be better captured. The MDCA module generates a contextual attention map in the spatial dimension to ensure that important spatial details are not lost when fusing the features of the high-level encoder and the low-level decoder, while enhancing the guiding role of global information on local features. This design improves the ability to capture multi-scale features, especially in the processing of boundary details and complex structures of the heart. The introduction of the MDCA module enables the model to not only recover high-resolution features during decoding, but also significantly improve the segmentation accuracy of the blurred boundary areas in cardiac images.

[0062] At the end of the model, we proposed a reverse feature modulation module (RFM) to specifically deal with the class imbalance problem in multi-class segmentation tasks. RFM adaptively assigns weights to each class by generating a reverse mask combined with a dynamic weight generator. The advantage of this design is that it can dynamically adjust the feature weights of different classes to ensure that the model performs more accurately when dealing with small and difficult-to-segment areas. RFM not only enhances the focus on difficult-to-segment areas through the reverse mask mechanism, but also enhances the adaptability and robustness of the model in multi-class segmentation tasks by dynamically adjusting the features of each class, especially maintaining a high segmentation accuracy under complex pathological conditions.

[0063] By combining the MDCA and RFM modules, the improved U-Net model not only has better performance in multi-scale feature capture and global information fusion, but also effectively solves the problems of detail loss and category imbalance.

Claims

1. A cardiac medical image segmentation method based on improved U-Net, characterized in that: The method includes a multi-dimensional context awareness module (MDCA), a reverse feature modulation module (RFM), and a model architecture; A multi-dimensional context-aware module (MDCA) is proposed in the high-level skip connection of the U-Net model. This module can focus on both global and local context information. Capturing this information helps to understand and segment cardiac structures more accurately. The MDCA module measures the correlation between features in the horizontal and vertical directions by generating a spatial attention map. Query, Key, and Value represent the query vector, key vector, and value vector in Figure 1, respectively. Query, Key, and Value are generated by generating query, key, and value feature maps from the input feature map through a 1×1 convolution operation, as shown below: Query=W q ×X,Key=W k ×X,Value=W v ×X (1) Where: X is the input feature map, W q , W k , W v They are 1×1 convolution weight matrices used to generate query, key, and value feature maps, respectively; enabling the model to provide higher segmentation accuracy and robustness when processing complex cardiac structures; The reverse feature modulation module (RFM) generates a reverse mask and adaptively adjusts the feature weights, so that the model can maintain high accuracy in difficult-to-segment areas such as the myocardium. It effectively improves the accuracy of multi-category segmentation tasks, especially when dealing with areas with unbalanced categories or blurred boundaries. The model architecture retains the symmetric encoder-decoder structure of U-Net, which enhances the segmentation capability of complex cardiac anatomical structures. In the encoder, multiple 3x3 convolutional layers are used alternately with ReLU activation functions to extract multi-layer features of the input image, and batch normalization (BatchNorm) is used to accelerate training and improve stability. The specific convolution and pooling operations are expressed by the following formula: X l+1 =MaxPool(ReLU(BatchNorm(Conv(X l )))) (9) Among them, X l The input feature map of the lth layer is extracted through 3x3 convolution, and then nonlinearly transformed through batch normalization and ReLU activation function. Finally, the spatial resolution of the feature map is gradually reduced through maximum pooling (MaxPool) to capture more global information while reducing the computational burden; By combining the MDCA and RFM modules, the improved U-Net model not only has better performance in multi-scale feature capture and global information fusion, but also solves the problems of detail loss and category imbalance.

2. The cardiac medical image segmentation method based on improved U-Net according to claim 1, characterized in that: The calculation of the correlation matrix of the multi-dimensional context-aware module (MDCA) generates a correlation matrix A between features by calculating the dot product of the query and the key: Here, the soft max function is used to normalize the correlation matrix A so that it reflects the relative importance of features, d k is the dimension scaling factor of Key, which is used to prevent the dot product result from being too large; then the attention matrix A is applied to Value to generate the weighted output: O=A×Value (3) This operation can help the model capture global and local contextual information, especially for accurate segmentation of complex heart boundaries.

3. The cardiac medical image segmentation method based on improved U-Net according to claim 1, characterized in that: The reverse feature modulation module (RFM) model generates a rough segmentation mask M for each category i , where i represents the category index; the coarse mask is generated by Sigmoid activation: M i =Sigmoid(f coarse (X)) (5) Among them, f coarse Represents the rough segmentation result generated by the network; then the reverse mask R of each category is generated by the reverse operation i : R i =1-M i (6) This reverse mask R i It allows the model to focus more accurately on areas outside the current category, further improving segmentation accuracy.

4. The cardiac medical image segmentation method based on improved U-Net according to claim 1, characterized in that: The model architecture convolution and pooling operations are expressed by the following formula: X l+1 =MaxPool(ReLU(BatchNorm(Conv(X l )))) (9) Among them, X l The input feature map of the lth layer is extracted through 3x3 convolution, and then nonlinearly transformed through batch normalization and ReLU activation function. Finally, the spatial resolution of the feature map is gradually reduced through maximum pooling (MaxPool) to capture more global information while reducing the computational burden.

Images