A fetal cerebellum ultrasound image segmentation method of deep attention network

Abstract

The application discloses a fetal cerebellum ultrasound image segmentation method based on a deep attention network, and comprises the following steps: firstly, fetal ultrasound data sets are randomly divided into a training set and a test set according to a 4:1 ratio, and the fetal ultrasound data sets are preprocessed; then, a DA-Net network model is built, a six-layer U-shaped network is constructed, a residual cascade block and an improved triple attention module are encapsulated into a composite module embedded into an encoder and a decoder of the network, and the composite module is used for effective feature extraction; next, a pixel position perception loss is used to train the network model on the training set; finally, the model is tested on the test set, and according to the test result, the model with the optimal performance is selected as a final model; and the model is used for fetal cerebellum ultrasound image segmentation. The fetal cerebellum ultrasound image segmentation method based on the deep attention network DA-Net can accurately realize positioning and segmentation of fetal cerebellum ultrasound images, and good segmentation performance and robustness are obtained.

Description

Technical Field

[0001] This invention relates to the fields of medical image segmentation and artificial intelligence technology, specifically to a method for segmenting fetal cerebellar ultrasound images using a deep attention network. Background Technology

[0002] Ultrasound (US), as one of the most commonly used imaging modalities, has become an indispensable diagnostic tool in clinical practice. Due to its advantages such as low cost, non-invasiveness, and real-time imaging, ultrasound imaging is widely used in prenatal screening worldwide. The detection of fetal cerebellar development is an essential part of prenatal examinations, as cerebellar hypoplasia (reduced cerebellar volume) seriously affects fetal intellectual development. Low image quality caused by noise and artifacts means that ultrasound examinations are highly dependent on the experience of the operator or diagnostic physician. Ultrasound image measurements are generally performed manually by physicians, resulting in a heavy workload and difficulty in ensuring examination quality. Furthermore, factors such as maternal obesity or amniotic fluid can easily generate artifacts in the images that resemble key structures, leading to misdiagnosis. Digital image processing technology can help doctors make diagnoses, reduce their workload, locate lesions in organs, and accurately segment them, providing important evidence for disease diagnosis and biomedical research. However, ultrasound images are often characterized by low contrast, complex structures, and unclear boundaries. Traditional biomedical image segmentation methods cannot accurately segment target regions and are gradually being replaced by deep learning-based methods. Deep convolutional neural networks (DCNNs) can extract abstract features that are difficult for humans to capture by using a large number of convolutional operations, but they inevitably suffer from the problem of feature information loss. Therefore, obtaining sufficiently rich and effective features is the key to improving network performance and segmentation accuracy. Summary of the Invention

[0003] The purpose of this invention is to address the problem of lost network feature information by proposing a deep attention network method for fetal cerebellar ultrasound image segmentation, namely DA-Net. This method uses an improved triple attention module (ITA) to extract more comprehensive and effective image features, thereby improving the segmentation accuracy, performance, and robustness of the network.

[0004] To achieve the above objectives, the present invention employs the following technical solution.

[0005] A method for segmenting fetal cerebellar ultrasound images using a deep attention network includes the following steps:

[0006] Step 1: Randomly divide the fetal ultrasound dataset into training and test sets in a 4:1 ratio. Preprocess the fetal ultrasound dataset by randomly cropping, horizontally flipping, vertically flipping, and adding Gaussian noise to expand the training set.

[0007] Step 2: Build the DA-Net network model;

[0008] Step 3: On the training set, train the DA-Net network model using pixel location-aware loss, perform the segmentation task, and generate a segmentation model;

[0009] Step 4: Test the segmentation model on the test set, and select the segmentation model with the best performance as the final segmentation model based on the test results;

[0010] Step 5: Use the final segmentation model to segment the fetal cerebellum ultrasound image and obtain the segmentation results.

[0011] Furthermore, the structure of the DA-Net network model described in step two is as follows:

[0012] The DA-Net network model adds a new layer to the original U-Net, forming a six-layer U-shaped network; the original U-Net's double convolutional modules are replaced with composite modules (CB).

[0013] Furthermore, in step two, the structure of the composite module CB in the DA-Net network model is as follows:

[0014] The composite module CB consists of the residual cascade block RCB and the improved triple attention module ITA.

[0015] Furthermore, the structure and design method of the residual cascade block RCB in the composite module CB are as follows:

[0016] The Residual Concatenated Block (RCB) integrates multiple convolutions of different sizes through addition and concatenation. In the RCB, the input features are filtered by a 1×1 convolution and two 3×3 convolutions to obtain feature maps f_1, f_2, and f_3, respectively. After each convolution, batch normalization and ReLU activation are performed. Then, f_1 and f_2 are added to obtain feature map f_4, and f_3 and f_4 are concatenated to obtain the final output feature map.

[0017] Furthermore, the structure and design method of the improved triple attention module ITA in the composite module CB are as follows:

[0018] An improved triple attention module (ITA) is constructed using three parallel branches, labeled as path (a), path (b), and path (c). Path (c) is improved using a cross-channel interaction strategy that does not require dimensionality reduction, as employed in efficient channel attention (ECA), thereby processing the input features. The output features of the three paths are averaged to obtain the final effective feature map.

[0019] In path (a), the input feature map X is rotated 90° counterclockwise along the H-axis to obtain a feature map X_a with shape W×H×C. Then, X_a is processed by Z-pooling, 7×7 convolution, batch normalization and sigmoid function to obtain attention weights w_a. Next, X_a is weighted by dot product to generate feature map X_a′. Then, it is rotated 90° clockwise along the H-axis to restore the original input shape and generate feature map M_a.

[0020] The structure of path (b) is similar to that of path (a); the input feature map x is rotated 90° counterclockwise along the w-axis to obtain feature map x_b. Next, it is processed by z-pooling, 7×7 convolution, batch normalization and sigmoid function to obtain attention weights w_b. Then, dot product weighting is performed on X_b to obtain feature map x_b'. Finally, X_b' is rotated 90° clockwise along the W-axis to restore the original input shape, thereby generating feature map M_b.

[0021] In path (c), the input feature map X is processed by global average pooling, 3×3 convolution, batch normalization and sigmoid function to obtain attention weights w_c. Finally, the weights are summed by dot product to obtain feature map M_c.

[0022] The Z-pool layer in paths (a) and (b) is responsible for connecting the features obtained by average pooling and max pooling, and can be represented by formula (1):

[0023] Z-pool(x) = [MaxPool 0d (x), AvgPool 0d (x)], (1)

[0024] Where 0d represents the 0th dimension where max pooling and average pooling operations occur, MaxPool(x) represents max pooling, and AvgPool(x) represents average pooling.

[0025] Furthermore, the pixel position-aware loss described in step three is defined as:

[0026] The pixel position aware loss (PPA loss) consists of a weighted IoU loss and a BCE loss; the PPA loss is defined by formula (2):

[0027]

[0028] In formula (2), L represents the PPA loss. Indicates weighted IoU loss. This represents the weighted BCE loss.

[0029] The advantages and beneficial effects of this invention are:

[0030] (1) Using residual concatenated blocks (RCB) to extract semantic features can make up for the loss of feature information in the previous convolution process and retain richer semantic information;

[0031] (2) An improved triple attention module (ITA) was designed to enhance effective feature representation, suppress background noise, tissue artifacts and irrelevant features, and improve the robustness of the network.

[0032] (3) DA-Net combines the improved triple attention module ITA and the residual concatenation block RCB into a composite module CB. This composite module CB is used to replace the double convolution operation in the original U-Net network, which increases the width of the network and helps the network obtain stronger feature extraction capabilities.

[0033] (4) DA-Net adds a layer to the original U-Net network, forming a six-layer U-shaped network, which increases the depth of the network and helps the network obtain a larger receptive field;

[0034] (5) Use pixel position-aware loss to train the network model, so that performance indices such as IoU and Dice can grow synchronously and steadily.

[0035] (6) The designed DA-Net network model achieved higher accuracy and network performance in the fetal cerebellar ultrasound image segmentation task, with mIoU and mDice scores reaching 87.45% and 92.98%, respectively. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the fetal cerebellum ultrasound image segmentation method using deep attention networks.

[0037] Figure 2 This is a schematic diagram of the DA-Net structure.

[0038] Figure 3 This is a structural schematic diagram of the composite module CB.

[0039] Figure 4 This is a schematic diagram of the residual cascade block RCB.

[0040] Figure 5 A schematic diagram of the structure of the improved triple attention module ITA.

[0041] Figure 6 The result of instance segmentation for DA-Net. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] A method for segmenting fetal cerebellar ultrasound images using deep attention networks, such as Figure 1 As shown, it includes the following steps:

[0044] Step 1: Randomly divide the fetal ultrasound dataset into training and test sets in a 4:1 ratio. Preprocess the fetal ultrasound dataset by randomly cropping, horizontally flipping, vertically flipping, and adding Gaussian noise to expand the training set.

[0045] The preprocessing methods and steps described herein are as follows:

[0046] (a) To avoid information loss due to image size changes, random cropping is used to directly crop out a fixed-size image for inputting into the network. Here, the fixed size is 512×512.

[0047] (b) The target structure in the image may be in any position, so the training data is randomly sampled with a probability of p = 0.5 and then horizontally and vertically flipped.

[0048] (c) The dataset is expanded by adding Gaussian noise;

[0049] Step 2: Build the DA-Net network model;

[0050] like Figure 2 As shown, the structure and method of the DA-Net network model are as follows:

[0051] A new layer was added to the original U-Net, forming a six-layer U-shaped network; the original U-Net's dual convolutional modules were replaced with composite modules (CB).

[0052] like Figure 3 As shown, the structure of the composite module CB in the DA-Net network model is as follows:

[0053] It consists of a residual cascade block (RCB) and an improved triple attention module (ITA).

[0054] like Figure 4 As shown, the structure and design method of the residual cascade block RCB in the composite module CB are as follows:

[0055] The Residual Concatenated Block (RCB) integrates multiple convolutions of different sizes through addition and concatenation. In the RCB, the input features are filtered by a 1×1 convolution and two 3×3 convolutions to obtain feature maps f_1, f_2, and f_3, respectively. After each convolution, batch normalization and ReLU activation are performed. Then, f_1 and f_2 are added to obtain feature map f_4, and f_3 and f_4 are concatenated to obtain the final output feature map.

[0056] like Figure 5 As shown, the structure and design method of the improved triple attention module ITA in the composite module CB are as follows:

[0057] An improved triple attention module (ITA) is constructed using three parallel branches, labeled as path (a), path (b), and path (c). Path (c) is improved using a cross-channel interaction strategy that does not require dimensionality reduction, as employed in Efficient Channel Attention (ECA), thereby processing the input features. The output features of the three paths are averaged to obtain the final effective feature map.

[0058] In path (a), the input feature map X is rotated 90° counterclockwise along the H-axis to obtain a feature map X_a with shape W×H×C. Then, X_a is processed by Z-pooling, 7×7 convolution, batch normalization and sigmoid function to obtain attention weights w_a. Next, X_a is weighted by dot product to generate feature map X_a′. Then, it is rotated 90° clockwise along the H-axis to restore the original input shape and generate feature map M_a.

[0059] The structure of path (b) is similar to that of path (a); the input feature map X is rotated 90° counterclockwise along the W axis to obtain feature map X_b. Next, it is processed by Z-pooling, 7×7 convolution, batch normalization and sigmoid function to obtain attention weights w_b. Then, X_b is multiplied and weighted to obtain feature map X_b′. Finally, X_b′ is rotated 90° clockwise along the W axis to restore the original input shape, thereby generating feature map M_b.

[0060] In path (c), the input feature map X is processed by global average pooling, 3×3 convolution, batch normalization and sigmoid function to obtain attention weights w_c. Finally, the weights are summed by dot product to obtain feature map M_c.

[0061] The Z-pool layer in paths (a) and (b) is responsible for connecting the features obtained by average pooling and max pooling, and can be represented by formula (1):

[0062] Z-pool(x) = [MaxPool0d (x), AvgPool 0d (x)], (1)

[0063] Where 0d represents the 0th dimension where max pooling and average pooling operations occur, MaxPool(x) represents max pooling, and AvgPool(x) represents average pooling.

[0064] Step 3: On the training set, train the DA-Net network model using pixel location-aware loss, perform the segmentation task, and generate a segmentation model;

[0065] The pixel position awareness loss is defined as follows:

[0066] The pixel position aware loss (PPA loss) consists of a weighted IoU loss and a BCE loss; the PPA loss is defined by formula (2):

[0067]

[0068] In formula (2), L represents the PPA loss. Indicates weighted IoU loss. Indicates the weighted BCE loss;

[0069] Step 4: Test the segmentation model on the test set, and select the segmentation model with the best performance as the final segmentation model based on the test results;

[0070] Step 5: Use the final segmentation model to segment the fetal cerebellum ultrasound image and obtain the segmentation results.

[0071] Example

[0072] The specific implementation of this invention is illustrated below through experimental examples using fetal ultrasound datasets:

[0073] (1) Preprocessing the dataset: The dataset is randomly divided into training and test sets in a 4:1 ratio, and then data augmentation is performed to expand the training set: (a) To avoid information loss due to image size changes, random cropping is used to directly crop images of a fixed size into the network. Here, the fixed size is 512×512; (b) The target structure in the image may be in any position, so the probability p is taken as 0.5 to randomly sample training data for horizontal and vertical flipping; (c) Gaussian noise is added to expand the dataset to alleviate the interference of artifacts and signal amplitude on the accurate segmentation of target structures and enhance the robustness of the network.

[0074] (2) Constructing the DA-Net model; The DA-Net network model adds a new layer to the original U-Net, forming a six-layer U-shaped network; The dual convolutional modules of the original U-Net are replaced with composite modules CB; The composite module CB consists of residual concatenated blocks RCB and improved triple attention modules ITA; The structure and design method of the residual concatenated blocks RCB are as follows:

[0075] The Residual Concatenated Block (RCB) integrates multiple convolutions of different sizes through addition and concatenation. In the RCB, the input features are filtered by a 1×1 convolution and two 3×3 convolutions to obtain feature maps f_1, f_2, and f_3, respectively. After each convolution, batch normalization and ReLU activation are performed. Then, f_1 and f_2 are added to obtain feature map f_4, and f_3 and f_4 are concatenated to obtain the final output feature map.

[0076] The improved structure and design method of the triple attention module ITA are as follows:

[0077] An improved triple attention module (ITA) is constructed using three parallel branches, labeled as path (a), path (b), and path (c). Path (c) is improved using a cross-channel interaction strategy that does not require dimensionality reduction, as employed in Efficient Channel Attention (ECA), thereby processing the input features. The output features of the three paths are averaged to obtain the final effective feature map.

[0078] In path (a), the input feature map X is rotated 90° counterclockwise along the H-axis to obtain a feature map X_a with shape W×H×C. Then, X_a is processed by Z-pooling, 7×7 convolution, batch normalization and sigmoid function to obtain attention weights w_a. Next, X_a is weighted by dot product to generate feature map X_a′. Then, it is rotated 90° clockwise along the H-axis to restore the original input shape and generate feature map M_a.

[0079] The structure of path (b) is similar to that of path (a); the input feature map X is rotated 90° counterclockwise along the W axis to obtain feature map X_b. Next, it is processed by Z-pooling, 7×7 convolution, batch normalization and sigmoid function to obtain attention weights w_b. Then, x_b is multiplied and weighted to obtain feature map X_b′. Finally, X_b′ is rotated 90° clockwise along the W axis to restore the original input shape, thereby generating feature map M_b.

[0080] In path (c), the input feature map x is processed by global average pooling, 3×3 convolution, batch normalization and sigmoid function to obtain attention weights w_c. Finally, the feature map M_c is obtained by weighting through dot product.

[0081] The Z-pool layer in paths (a) and (b) is responsible for connecting the features obtained by average pooling and max pooling, and can be represented by formula (1):

[0082] Z-pool(x) = [MaxPool 0d (x), avgPool 0d (x_], (1)

[0083] Where 0d represents the 0th dimension where max pooling and average pooling operations occur, MaxPool(x) represents max pooling, and AvgPool(x) represents average pooling.

[0084] (3) The DA-Net network model is trained on the training set using pixel position-aware loss; the pixel position-aware loss is defined as:

[0085] The pixel position aware loss (PPA loss) consists of a weighted IoU loss and a BCE loss; the PPA loss is defined by formula (2):

[0086]

[0087] In formula (2), L represents the PPA loss. Indicates weighted IoU loss. Indicates the weighted BCE loss;

[0088] (4) Test the segmentation model on the test set, and select the segmentation model with the best performance as the final segmentation model based on the test results;

[0089] (5) The final segmentation model was used to segment the fetal cerebellum ultrasound image, and the segmentation results were obtained. The results of this method and the classic image segmentation algorithm are compared and are shown in Table 1:

[0090] By comparing with other methods, the performance indicators of this method are significantly higher than those of other classic algorithms. Compared with the original U-Net, DA-Net improves mIoU, mDice, mPrecision, mRecall, and mF1 by 3.8%, 7.78%, 0.51%, 3.84%, and 2.52%, respectively. In mIoU and mDice, DA-Net outperforms ECAU-Net by 1.44% and 1.63%, respectively.

[0091] Table 1. Comparison of experimental results for semantic segmentation models (%)

[0092]

[0093]

[0094] like Figure 6 As shown, Unet+cSE and CE-Net have undersegmentation problems when segmenting fetal cerebellar image samples in group (A); compared with other methods, DA-Net can accurately determine the location of the fetal cerebellum, and the area and contour of the segmented target region are closest to the ground truth map, which can verify that the method proposed in this invention has obvious advantages and prospects.

[0095] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fetal cerebellum ultrasound image segmentation method of a deep attention network, characterized in that, The method comprises the following steps: Step 1: randomly divide the fetal ultrasound data set into a training set and a test set according to a ratio of 4:1, pretreat the fetal ultrasound data set, and expand the training set by means of random cropping, horizontal flipping, vertical flipping and adding Gaussian noise; Step 2: build a DA-Net network model; The structure of the DA-Net network model is that one layer is newly added on the basis of an original U-Net to form a six-layer U-shaped network; a double convolution module of the original U-Net is replaced by a composite module CB; The structure of the composite module CB is that the composite module CB is composed of a residual cascade block RCB and an improved triple attention module ITA; The structure and design method of the residual cascade block RCB are as follows: multiple convolutions of different sizes are integrated through addition and cascade; in the residual cascade block RCB, input features are filtered through one convolution and two convolutions, respectively to obtain feature maps , and ; batch normalization processing and Relu function activation are required after each convolution; then, is added to to obtain a feature map , and is cascaded with , thereby obtaining the final output feature map; The structure and design method of the improved triple attention module ITA are that: Step 1: use three parallel branches to compose the improved triple attention module ITA, and mark the three parallel branches as path (a), path (b) and path (c) respectively; Step 2: the cross-channel interaction strategy without dimension reduction in the effective channel attention ECA is adopted to improve the path (c) to process the input features; Step 3: the output features of the three paths are averaged to obtain the final effective feature map; Among them, path (a) will input the feature map. along Rotate the axis counterclockwise To obtain the shape as Feature map ,Then pass , Attention weights are obtained through convolution, batch normalization, and sigmoid function processing. Next, we will use dot product to... Weighting is performed to generate feature maps , and then, along Rotate the axis clockwise To restore the original input shape, generate a feature map. ; Path (b) will input the feature map along Rotate the axis counterclockwise To obtain feature maps Next, after , Convolution, batch normalization, and sigmoid function processing are used to obtain attention weights. Then, for To obtain the feature map, perform dot product weighting. ,at last, along Rotate the axis clockwise To restore the original input shape, thereby generating a feature map. ; In path (c), the input feature map is globally average-pooled, convolved, batch-normalized, and sigmoid-processed to obtain attention weights , and finally, weighted by point multiplication to obtain the feature map ; In path (a) and path (b) The layer is responsible for connecting the features obtained by average pooling and max pooling, which is expressed by formula (1): , (1) wherein, denotes the 0th dimension where max pooling and average pooling operations occur, denotes max pooling, denotes average pooling; Step 3: train the DA-Net network model using a pixel position perception loss on the training set, perform a segmentation task and generate a segmentation model; Step 4: test the segmentation model on the test set, and select the segmentation model with the optimal performance as the final segmentation model according to the test result; Step 5: use the final segmentation model to perform fetal cerebellum ultrasound image segmentation to obtain a segmentation result. 2.The fetal cerebellum ultrasound image segmentation method of deep attention network according to claim 1, characterized in that, The pixel position perception loss in step 3 is: The pixel position perception loss is composed of a weighted IoU loss and a BCE loss, and is defined as formula (2): (2) In equation (2), denotes the PPA loss, denotes the weighted IoU loss, denotes the weighted BCE loss.

Images