Attention and edge perception enhanced magnetic resonance placenta automatic segmentation method

Abstract

The invention belongs to the field of medical image processing, and provides an improved deep learning model based on U-Net for a placenta automatic segmentation task in a magnetic resonance imaging (MRI) image. Technical improvement key points are as follows: a convolutional block attention mechanism module (CBAM) is embedded in a down-sampling path, and the key channel and spatial feature extraction capability is enhanced; meanwhile, an edge perception enhancement module (BPM) is embedded, so that the edge distinction degree of the placenta and the uterus tissue is enhanced; in addition, an attention gating module (AG) is additionally arranged on an up-sampling path, the semantic distinguishing capability of a placenta target area and a background is improved, and background noise interference is reduced. Experimental verification shows that the segmentation precision of the model in a placenta MRI automatic segmentation task is obviously superior to that of an existing method.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of medical image processing, and particularly relates to an automatic placenta image segmentation method with attention and edge perception enhancement. BACKGROUND

[0002] The rapid development of medical imaging technology has greatly improved the key role of medical imaging in disease diagnosis, treatment plan development and preventive healthcare. Among them, magnetic resonance imaging (MRI) can present the human soft tissue anatomy and pathological changes with high resolution, providing an indispensable tool for clinicians to accurately assess disease progression. However, manual interpretation of complex anatomical features in medical images not only consumes a large amount of professional human resources, but also has strong subjectivity and is prone to diagnostic differences. In recent years, automatic segmentation technology based on deep learning has become an efficient solution, which has shown advantages in both efficiency and accuracy in medical image organ recognition tasks, and is expected to optimize clinical workflow and improve the objectivity and reliability of diagnosis.

[0003] In obstetric and gynecological clinical practice, accurate segmentation of the placenta has important diagnostic value. The spatial position of the placenta (such as the anterior or posterior wall attachment) is closely related to various adverse pregnancy outcomes, such as late gestational bleeding and preterm birth. Accurate placental positioning is crucial for predicting and alleviating pregnancy complications and developing individualized delivery plans, especially when assessing the feasibility of vaginal delivery and cesarean section. In addition, placental volume is an important biomarker for predicting placenta-related complications (such as placental abruption and placenta implantation spectrum diseases), and also provides a key clinical reference for assessing fetal health (such as fetal growth restriction and macrosomia). Placental segmentation results are also a basic preprocessing step for subsequent placental lesion automatic detection, imageomics analysis and other advanced algorithms. Currently, manual segmentation methods that rely on physician experience not only consume time and effort, but also have significant variability in reproducibility depending on the operator's professional level, making it difficult to be widely used in clinical practice. Therefore, there is an urgent clinical need and important application value to develop fast and accurate fully automatic placental segmentation technology.

[0004] Early placental segmentation methods cover a variety of traditional image processing techniques, and these methods have their own advantages in specific scenarios. However, under the condition of having sufficient labeled data, deep learning models usually show superior performance in handling complex segmentation tasks. Its core advantage lies in its ability to autonomously learn hierarchical feature representations from raw image data, and its accuracy and robustness in placental analysis tasks have far exceeded traditional methods.

[0005] Although deep learning has shown great potential in medical image segmentation, there are still severe challenges in directly applying it to placenta segmentation. First, there is a huge inter-individual difference in the location, shape and size of the placenta. Second, there are a large number of blood vessels between the placenta and the uterine wall, resulting in a blurred boundary. Third, the inherent low contrast characteristics of the placental tissue and the surrounding background further increase the difficulty of accurately outlining its contour. The receptive field range, feature extraction capability and perception ability of the existing general segmentation model to subtle boundaries are still insufficient when dealing with these specific challenges, making it difficult for the segmentation accuracy to fully match the manual segmentation results of experienced clinicians.

[0006] In summary, in view of the limitations of existing automatic placenta segmentation techniques in dealing with complex scenarios with large morphological variations, blurred boundaries and low contrast, there is an urgent need for a novel and more robust deep learning segmentation method to improve the accuracy and reliability of placenta segmentation and meet the growing demand for clinical auxiliary diagnosis. SUMMARY

[0007] The present application proposes an attention and edge perception enhanced automatic magnetic resonance placenta segmentation method, aiming to improve the segmentation accuracy and efficiency of MRI image placenta segmentation, thereby assisting clinicians to achieve accurate screening of placental abnormalities.

[0008] To achieve the above-mentioned purpose, the present application provides the following technical scheme: an attention and edge perception enhanced automatic magnetic resonance placenta segmentation method, comprising the following steps:

[0009] Step 1: Preprocess the original placenta image data to obtain a preprocessed placenta image dataset;

[0010] Step 2: Construct an attention and edge perception enhanced U-Net image segmentation network;

[0011] In step 2, the proposed segmentation network is divided into an input layer, an encoder, a decoder, and an output layer.

[0012] In the input layer, the first layer of convolution converts the picture into a 64-channel feature map of the same size, and the second layer of convolution performs size reduction and channel doubling downsampling on the feature map, and outputs the result to the encoder.

[0013] The encoder stage contains three encoders, each containing a downsampling convolution module, a convolution block attention mechanism module (CBAM) and an edge perception enhancement module (BPM), and through an attention gating module (AG), the output of the same level decoder and the output of the previous layer encoder are channel spliced as input, and after two layers of convolution to extract features, not only output to the same level decoder, but also through downsampling into the next level encoder.

[0014] The decoding stage includes three decoder modules, each of which uses the output feature map of the attention gate module (AG) of the encoder and the output map of the previous decoder module as input, extracts features through two layers of convolution, and outputs not only to the same level encoder but also to the next level decoder through up-sampling. After feature extraction through three decoders, it enters the output layer.

[0015] The output layer splices the output of the decoding module and the output of the input layer as input, extracts features through two layers of convolution, and obtains the final output through the Sigmoid function.

[0016] The encoder includes a down-sampling convolution module, a convolution block attention mechanism module (CBAM), an edge perception enhancement module (BPM), and an attention gate module (AG).

[0017] The down-sampling convolution module includes convolution, batch normalization, ReLU activation function, and down-sampling operation.

[0018] The convolution block attention mechanism module (CBAM) is integrated in each down-sampling convolution module of the encoder path in turn, and is used for adaptive refinement of the primary and advanced feature maps extracted by the encoder. The CBAM module infers the attention weight maps in the channel dimension and the spatial dimension in turn through the concatenated channel attention submodule and spatial attention submodule, and multiplies them with the input feature map to realize the recalibration of the features. The channel attention submodule uses global average pooling and global maximum pooling operations to aggregate the spatial information of the feature map to generate two different spatial context descriptors F avg and F max , which are sent to a shared multi-layer perceptron (MLP). The output of the MLP is passed through a Sigmoid activation function to generate the final channel attention map.

[0019] The calculation process of channel attention can be expressed as:

[0020]

[0021] where σ represents the Sigmoid function, W0 and W1 are the shared weights of the MLP, and the input feature map F is multiplied by the channel attention map M c (F) to obtain the channel-refined feature map F':

[0022]

[0023] The spatial attention sub-module respectively applies average pooling and maximum pooling along the channel dimension to generate two two-dimensional feature maps, splices the two feature maps in the channel dimension, uses a standard 7x7 convolution layer to perform convolution operation on the spliced feature map, reduces the dimension of the feature map to a single-channel feature map, and generates a spatial attention map after Sigmoid function activation;

[0024] The calculation process of spatial attention can be expressed as:

[0025]

[0026] wherein, ƒ 7X7 represents a 7x7 convolution operation, [·] represents a splicing operation, and the channel-refined feature map F' is multiplied by the spatial attention map M s (F') position by position to obtain the final refined output feature map F'':

[0027]

[0028] The edge perception enhancement module (BPM) assumes that the input feature map is F in The module realizes the function through the following three consecutive operations:

[0029] The boundary extraction branch processes the input feature to obtain a boundary-sensitive feature representation:

[0030]

[0031] wherein, DepthwiseConv 3X3 represents a depthwise separable convolution with a kernel size of 3x3 and a padding value of 1, Conv 1X1 represents a point-by-point convolution, BN represents batch normalization, and δ is a ReLU activation function.

[0032] The extracted boundary feature is spliced with the original input feature along the channel dimension:

[0033]

[0034] The feature fusion convolution integrates the complementary information flow:

[0035]

[0036] wherein, Conv 3X3 represents a standard convolution with a kernel size of 3x3 and a padding value of 1.

[0037] The attention gate module (AG) applies a 1x1 convolution to the up-sampling feature map g of a certain layer in the decoder path and compresses the channel number to a lower dimension, and then performs batch normalization to align and preliminarily process the features. A 1x1 convolution is also applied to the encoder feature x to adjust the channel number to the same as that of the processed g, so that the two can be added element by element. The processed gate signal g and the encoder feature x are added element by element to fuse high-level semantics and low-level details. A ReLU activation function is applied to the fused feature map to introduce nonlinearity. Another 1x1 convolution is used to compress the channel number of the feature map to 1, and then a Sigmoid activation function is used to generate a spatial dimension attention weight map.

[0038] The process can be expressed as:

[0039]

[0040] where W x ,W g ,W σ is the weight of the 1x1 convolution, b1 and b2 are bias terms, and σ is the Sigmoid function. The generated attention weight map α is multiplied element by element with the original encoder feature map x. The weighted encoder feature α⊙x is passed to the decoder and concatenated with the up-sampled feature map as the input of the next layer of the decoder.

[0041] The operation of each decoder includes convolution, batch normalization, ReLU activation function, and up-sampling operation.

[0042] Step 3: input the preprocessed placenta image into the proposed image segmentation network for training to obtain a trained placenta image segmentation network model.

[0043] Step 4: use the trained network model to segment the preprocessed placenta image to obtain a placenta segmentation result. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and serve to explain the principles of the application. In the drawings:

[0045] Figure 1 is a flowchart of the implementation of the present application;

[0046] Figure 2 is a segmentation network structure diagram of the present application;

[0047] Figure 3 is an encoder module structure diagram of the present application;

[0048] Figure 4 is a down-sampling convolution module structure diagram of the present application;

[0049] Figure 5 is a convolution block attention enhancement module (CBAM) structure diagram of the present application;

[0050] Figure 6 is a boundary perception module (BPM) structure diagram of the present application;

[0051] Figure 7 is an attention gate module (AG) structure diagram of the present application;

[0052] Figure 8 is a decoder structure diagram of the present application;

[0053] Figure 9 is an experimental segmentation result of the present application. DETAILED DESCRIPTION

[0054] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0055] Embodiment: A magnetic resonance placenta automatic segmentation method enhanced by attention and boundary perception, comprising the following steps:

[0056] Reference Figure 1 , the steps for implementing the present application include: (1) preprocessing the original pregnant woman magnetic resonance image data to obtain a preprocessed pregnant woman magnetic resonance image data set; (2) constructing an attention and boundary perception enhanced U-Net image segmentation network; (3) inputting the preprocessed pregnant woman magnetic resonance image into the proposed image segmentation network for training to obtain a trained placenta segmentation model; (4) using the trained model to segment the preprocessed placenta image to obtain a placenta segmentation result.

[0057] Reference Figure 2 is a network structure diagram proposed, and the network is divided into an input layer, an encoder, a decoder, and an output layer.

[0058] In the input layer, the first layer of convolution converts the picture into a 64-channel feature map of the same size, and the second layer of convolution performs a size halving and channel doubling down-sampling on the feature map, and outputs the result to the encoder.

[0059] The encoder stage includes three encoders, each of which includes a down-sampling convolution module, a convolution block attention mechanism module (CBAM), and an edge-aware enhancement module (BPM), and the output of the same level decoder and the output of the previous layer encoder are spliced as input through an attention gating module (AG), and after two layers of convolution feature extraction, the output is not only output to the same level decoder, but also enters the next level encoder through down-sampling.

[0060] The decoding stage includes three decoder modules, each of which uses the output feature map of the attention gating module (AG) of the encoder and the output map of the previous decoder module as input, and after two layers of convolution feature extraction, the output is not only output to the same level encoder, but also enters the next level decoder through up-sampling, and after three decoder feature extractions, it enters the output layer.

[0061] The output layer splices the output of the decoding module and the output of the input layer as input, and after two layers of convolution feature extraction, the final output is obtained through the Sigmoid function.

[0062] During model training, 101 T2WI scan images were included, and the inclusion and exclusion criteria were included. Inclusion criteria: singleton pregnancy, and the pregnant woman received MRI examination before natural delivery or cesarean section. Exclusion criteria: mainly including poor image quality, obvious artifacts, or cases with fatal congenital malformations. This study conforms to the ethical standards and has obtained the approval of the relevant institutional review board. The research data is randomly divided into training set and test set in the ratio of about 9:1.

[0063] The model training uses a hybrid loss function combining BCE loss and DICE loss, and the weights of the two are 0.4 and 0.6 respectively. The specific formula is as follows (where yi represents the predicted pixel value, pi represents the corresponding true pixel value, and N is the total number of pixels):

[0064]

[0065]

[0066]

[0067] The BCE loss is used to measure the difference between the predicted value and the true value in the binary classification task; the DICE loss optimizes the segmentation boundary by calculating the overlap of the predicted region and the true region; the hybrid loss balances the class imbalance and boundary accuracy problem. The optimizer uses the Adam optimizer with adaptive moment estimation, and the initial learning rate is set to 1e-4. To prevent overfitting, the performance of the model on the validation set is continuously monitored for 100 training cycles, and the best-performing model is selected for subsequent evaluation. The model training uses 5-fold cross-validation method. The experiment is carried out on a Windows workstation equipped with NVIDIA A100 GPU, and the programming language is Python (3.9.15).

[0068] Reference Figure 3 , is the encoder structure diagram, which includes the down-sampling convolution module, the attention mechanism module (CBAM), the edge perception enhancement module (BPM) and the attention gating module (AG).

[0069] Reference Figure 4 , is the structure diagram of the down-sampling convolution module, which includes the convolution layer, the batch normalization layer, the ReLu activation function layer and the down-sampling.

[0070] Reference Figure 5 , is the structure diagram of the convolution block attention module (CBAM), which is integrated in the encoder path after each down-sampling convolution module. The CBAM module is used to adaptively refine the primary and high-level feature maps extracted by the encoder. The CBAM module uses a series of channel attention sub-modules and spatial attention sub-modules to infer the attention weight maps in the channel dimension and spatial dimension, respectively, and multiplies them with the input feature maps to realize the recalibration of the features. The channel attention sub-module uses global average pooling and global maximum pooling operations to aggregate the spatial information of the feature maps, generating two different spatial context descriptors F avg and F max , which are sent to a shared multi-layer perceptron (MLP). The output of the MLP is passed through a Sigmoid activation function to generate the final channel attention map.

[0071] The calculation process of channel attention can be expressed as:

[0072]

[0073] Where σ represents the Sigmoid function, W0 and W1 are the shared weights of the MLP, and the input feature map F is multiplied by the channel attention map M c (F) to obtain the channel-refined feature map F':

[0074]

[0075] The spatial attention sub-module respectively applies average pooling and maximum pooling along the channel dimension to generate two two-dimensional feature maps, splices the two feature maps in the channel dimension, uses a standard 7x7 convolution layer to perform convolution operation on the spliced feature map, reduces the dimension of the feature map to a single-channel feature map, and generates a spatial attention map after Sigmoid function activation;

[0076] The calculation process of spatial attention can be expressed as:

[0077]

[0078] Wherein, ƒ 7X7 represents a 7x7 convolution operation, [·] represents a splicing operation, and the channel-refined feature map F' is multiplied by the spatial attention map M s (F') position by position to obtain the final refined output feature map F'':

[0079]

[0080] Reference Figure 6 is a structure diagram of a boundary perception enhancement module (BPM), and it is assumed that the input feature map is F in The module realizes the function through the following three consecutive operations:

[0081] The boundary extraction branch processes the input feature to obtain a boundary-sensitive feature representation:

[0082]

[0083] Wherein, DepthwiseConv 3X3 represents a depthwise separable convolution with a kernel size of 3x3 and a padding value of 1, Conv 1X1 represents a pointwise convolution, BN represents batch normalization, and δ is a ReLU activation function.

[0084] The extracted boundary feature is spliced with the original input feature along the channel dimension:

[0085]

[0086] The feature fusion convolution integrates the complementary information flow:

[0087]

[0088] Wherein, Conv 3X3 represents a standard convolution with a kernel size of 3x3 and a padding value of 1.

[0089] Reference Figure 7Here is the structure diagram of the attention gating module (AG). A 1×1 convolution is applied to the upsampled feature map g of a certain layer in the decoder path, and its channel number is compressed to a lower dimension. Then, a batch normalization is applied to align and preliminarily process the features. A 1×1 convolution is also applied to the encoder feature x, and its channel number is adjusted to be the same as that of the processed g, ensuring that the two can be added element-wise. The processed gating signal g and the encoder feature x are added element-wise to fuse high-level semantics and low-level details. A ReLU activation function is applied to the fused feature map to introduce non-linearity. Another 1×1 convolution is used to compress the channel number of the feature map to 1. Then, a Sigmoid activation function is applied to generate a spatial dimension attention weight map.

[0090] This process can be described as follows:

[0091]

[0092] Among them, W x W g W σ b1 and b2 are the weights of the 1×1 convolution, b1 and b2 are the bias terms, and σ is the Sigmoid function. The generated attention weight map α is multiplied element-wise with the original encoder feature map x. The weighted encoder feature α⊙x is passed to the decoder and concatenated with the upsampled feature map as the input to the next layer of the decoder.

[0093] refer to Figure 8 This is a diagram of the decoder structure. The operations of each decoder include convolutional layers, ReLU activation function, batch normalization, convolutional layers, and upsampling.

[0094] refer to Figure 9 The images show examples of placental segmentation, with a total of three test segmentation examples. From left to right, they are the original MRI image, the result of manual placental segmentation, the result of nn-UNet placental segmentation, and the result of the method of this invention.

[0095] The method of this invention achieved a mean DSC of 0.810 ± 0.181 on the test set, which was 0.052 higher than nn-UNet (p<0.05, paired t-test); a mean IoU of 0.694 ± 0.185, which was 0.078 higher than nn-UNet (p<0.05, paired t-test); and a mean ACC of 0.986 ± 0.010, which was 0.003 higher than nn-UNet (p<0.05, paired t-test).

Claims

1. A method for automatic placental segmentation using magnetic resonance imaging with enhanced attention and edge perception, characterized in that, It includes the following steps: Step 1: Preprocess the original MRI images of pregnant women to obtain a preprocessed MRI image dataset of pregnant women; Step 2: Construct a U-Net image segmentation network with enhanced attention and edge awareness; Step 3: Input the preprocessed MRI image of the pregnant woman into the proposed image segmentation network for training to obtain a trained placental segmentation model; Step 4: Use the trained model to segment the preprocessed placental image to obtain the placental segmentation result.

2. The attention- and edge-aware enhanced magnetic resonance placenta automatic segmentation method according to claim 1, characterized in that: In step 2, the proposed segmentation network is divided into an input layer, an encoder, a decoder, and an output layer; In the input layer, the first convolutional layer converts the image into a 64-channel feature map of the same size, and the second convolutional layer downsamples the feature map by halving its size and doubling its channels, and outputs the result to the encoder; The encoder stage contains three encoders. Each encoder contains a downsampling convolution module, a block convolutional attention mechanism (CBAM) module, and an edge awareness enhancement module (BPM). The output of the same-level decoder and the output of the previous-level encoder are concatenated through the attention gating module (AG) as input. After two layers of convolution to extract features, the output is not only sent to the same-level decoder, but also downsampled and sent to the next level encoder. The decoding stage contains three decoder modules. Each decoder module uses the output feature map of the encoder's attention gating module (AG) and the output map of the previous level decoder module as input. After two layers of convolution to extract features, it is not only output to the encoder at the same level, but also enters the next level decoder through upsampling. After feature extraction by the three decoders, it enters the output layer. The output layer concatenates the output of the decoding module with the output of the input layer as input. After two layers of convolution to extract features, the final output is obtained by passing through the Sigmoid function.

3. The attention- and edge-aware enhanced magnetic resonance placenta automatic segmentation method according to claim 2, characterized in that: The encoder includes a downsampling convolution module, a block convolutional attention mechanism (CBAM) module, an edge awareness enhancement module (BPM) module, and an attention gating module (AG).

4. The attention- and edge-aware enhanced magnetic resonance placenta automatic segmentation method according to claim 3, characterized in that: The downsampling convolution module includes convolution, batch normalization, ReLU activation function, and downsampling operation.

5. The attention- and edge-aware enhanced magnetic resonance placenta automatic segmentation method according to claim 3, characterized in that: The Convolutional Block Attention (CBAM) module is sequentially integrated after each downsampling convolutional module in the encoder path. It is used to adaptively refine the primary and high-level feature maps extracted by the encoder. The CBAM module infers attention weight maps in the channel dimension and spatial dimension in sequence through the cascaded channel attention sub-module and spatial attention sub-module, and multiplies them with the input feature map to achieve feature recalibration.

6. The attention- and edge-aware enhanced magnetic resonance placenta automatic segmentation method according to claim 5, characterized in that: The channel attention submodule uses both global average pooling and global max pooling to aggregate the spatial information of the feature maps, generating two different spatial context descriptors F. avg and F max The two descriptors are fed into a shared multilayer perceptron (MLP), and the output of the MLP is passed through a sigmoid activation function to generate the final channel attention map. The calculation process for channel attention can be described as follows: Where σ represents the Sigmoid function, W0 and W1 are the shared weights of the MLP, and the input feature map F and the channel attention map M are... c (F) Multiply each channel sequentially to obtain the channel-refined feature map F':

7. The attention- and edge-aware enhanced magnetic resonance placenta automatic segmentation method according to claim 5, characterized in that: The spatial attention submodule applies average pooling and max pooling along the channel dimension to generate two two-dimensional feature maps. These two feature maps are then concatenated along the channel dimension. A standard 7×7 convolutional layer is used to convolve the concatenated feature map to reduce its dimensionality to a single-channel feature map. After activation by the Sigmoid function, a spatial attention map is generated. The computational process of spatial attention can be described as follows: Among them, ƒ 7X7 This represents a 7×7 convolution operation, and [·] represents a concatenation operation. The feature map F' after channel thinning and the spatial attention map M are shown. s (F') is multiplied position by position to obtain the final refined output feature map F'':

8. The attention- and edge-aware enhanced magnetic resonance placenta automatic segmentation method according to claim 3, characterized in that: Edge-aware enhancement module (BPM), assuming the input feature map is F in This module achieves its function through the following three consecutive operations: The boundary extraction branch processes the input features to obtain a boundary-sensitive feature representation: Among them, DepthwiseConv 3X3 This represents a depthwise separable convolution with a kernel size of 3×3 and padding of 1. 1X1 δ represents pointwise convolution, BN represents batch normalization, and δ is the ReLU activation function; The extracted boundary features are concatenated with the original input features along the channel dimension: Integrate complementary information flows through feature fusion convolution: Among them, Conv 3X3 This represents a standard convolution with a kernel size of 3×3 and a padding value of 1.

9. The attention- and edge-aware enhanced magnetic resonance placenta automatic segmentation method according to claim 3, characterized in that: The Attention Gating (AG) module applies a 1×1 convolution to the upsampled feature map g of a certain layer in the decoder path and compresses its channel count to a lower dimension. Then, it performs batch normalization to align and preliminarily process the features. Similarly, a 1×1 convolution is applied to the encoder feature x to adjust its channel count to be the same as the processed g, ensuring that the two can be added element-wise. The processed gated signal g and the encoder feature x are added element-wise to fuse high-level semantics and low-level details. A ReLU activation function is applied to the fused feature map to introduce non-linearity. Another 1×1 convolution is used to compress the channel count of the feature map to 1. Finally, a Sigmoid activation function is used to generate an attention weight map with spatial dimensions. This process can be described as follows: Among them, W x W g W σ b1 and b2 are the weights of the 1×1 convolution, b1 and b2 are the bias terms, and σ is the Sigmoid function. The generated attention weight map α is multiplied element-wise with the original encoder feature map x. The weighted encoder feature α⊙x is passed to the decoder and concatenated with the upsampled feature map as the input to the next layer of the decoder.

10. The attention- and edge-aware enhanced magnetic resonance placenta automatic segmentation method according to claim 2, characterized in that: Each decoder operation includes convolution, batch normalization, ReLU activation function, and upsampling.

Images