Breast ultrasonic image segmentation method based on local-global feature complementation and boundary perception

Abstract

The invention belongs to the technical field of ultrasonic image segmentation, and particularly relates to a breast ultrasonic image segmentation method based on local-global feature complementation and boundary perception, and the method comprises the following steps: S1, selecting a proper data set, and then carrying out the ROI region selection and mask image processing of the data set; s2, a data set is prepared, preprocessing, contrast increasing, normalization and data enhancement are carried out on the mammary gland data, and the data set comprises a training set and a test set; and S3, establishing a 2D convolutional neural network for breast segmentation. According to the invention, the method achieves the recognition and segmentation of the breast tumor tissue in the breast tumor image, improves the generalization segmentation performance of a network model for the breast tumor through the design of a network named as LGCBNet, improves the extraction of the boundary features of the tumor, improves the enhancement of the boundary features, accelerates the training speed and recognition speed of the network, and achieves the recognition and segmentation of the breast tumor tissue. And the segmentation precision and the generalization of the model are improved on the premise of ensuring the size of the model.

Description

Technical Field

[0001] The present invention relates to the field of ultrasound image segmentation technology, and in particular to a breast ultrasound image segmentation method based on local-global feature complementarity and boundary perception. Background Art

[0002] The breast tumor treatment guidelines point out that early detection and intervention treatment can improve the patient's survival rate. Clinical studies have shown that dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) can simultaneously display the physiological tissue characteristics and anatomical structure of breast cancer, and is an effective tool for breast cancer diagnosis. Currently, ultrasound imaging is widely used in the clinical diagnosis of breast diseases due to its advantages such as non-invasiveness, low cost, and rapidity. Unfortunately, due to the complexity of ultrasound images, even experienced radiologists find it difficult to accurately and quickly mark the lesion area.

[0003] With the gradual development of deep learning technology, the field of automatic segmentation of breast tumors has been increasingly valued. It has broad development prospects, and there are many specific research and application methods. Breast tumor segmentation research often needs to face complex images and variable tumor shapes and sizes, as well as the problem that the intensity distribution of tumors and dense tissues is similar and the boundaries are blurred. In order to solve these problems, Roneberger et al. proposed a U-Net network based on Fully Convolutional Networks (FCN) in 2015. The network can adapt to very small data sets and obtain more accurate segmentation results. In order to improve the segmentation accuracy, researchers consider adding various modules to supplement semantic information to improve U-Net. However, the design of adding dense connections in complex modules will cause invalid features to be transmitted together, and the improvement of the jump connection part is only a simple filtering of invalid features in the deep and shallow feature connection part. This simple processing of invalid features cannot cope with the segmentation task of complex medical images. The segmentation performance cannot be improved due to too many invalid features, which has a great impact on the segmentation performance of small mass areas. Summary of the invention

[0004] By designing a network called LGCBNet, the generalization of the network model for breast tumor segmentation is improved, and the model's ability to resist complex background interference is enhanced. At the same time, the network is lightweight and does not use a large number of parameters for training, thus improving the segmentation accuracy while ensuring the model size. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 is a flow chart of the present invention;

[0006] Figure 2 A specific structural diagram of the Transformer used in the global branch of the convolution module of the present invention;

[0007] Figure 3 is a structural diagram of the global context optimization module in the present invention;

[0008] Figure 4 is a structural diagram of the boundary perception module of the present invention;

[0009] Figure 5 This is a schematic diagram of the LGCBNet network structure of the present invention;

[0010] Figure 6 A schematic diagram showing a comparison between the method proposed in the present invention and the corresponding evaluation indicators of the prior art;

[0011] Figure 7 A schematic diagram showing the comparison of evaluation indicators for testing the effects of different modules using the method proposed in the present invention; DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] Example 1

[0014] like Figure 1-6 As shown, a breast ultrasound image segmentation method based on local-global feature complementation and boundary perception is proposed in one embodiment of the present invention, and the method includes the following steps:

[0015] S1. Select a suitable data set, then select the ROI area and process the mask map of the data set;

[0016] The dataset uses BUSC_Dataset and BUSI_Dataset breast tumor ultrasound image datasets. The dataset provides a region of interest roi file. After running, the region of interest in the image is obtained, as well as a binary mask map for network training and testing. The BUSC_Dataset dataset contains 250 cases. The data collected by BUSI_Dataset includes breast ultrasound images of women aged between 25 and 75 years old. This data was collected in 2018. The number of patients is 600 female patients. The dataset consists of 780 images with an average image size of 500*500 pixels. These images are in PNG format. The ground truth images are presented together with the original images. The images are divided into three categories, namely normal, benign, and malignant.

[0017] S2. Prepare the data set, preprocess the breast data, increase contrast, normalize and enhance the data. The data set includes a training set and a test set.

[0018] Prepare the data set and preprocess the data. First, divide the data set into a training set and a test set at a ratio of 8:2. However, the amount of available data is small, and deep learning requires a huge amount of training data as a basis. Therefore, image enhancement is required after normalization:

[0019] 1) Increase contrast. Perform nonlinear stretching on the image and redistribute the image pixel values ​​so that the number of pixels in a certain grayscale range is roughly the same, and change the grayscale histogram of the original image from a relatively concentrated grayscale interval to a uniform distribution in the entire grayscale range.

[0020] 2) Normalization. When processing and analyzing data, it is often necessary to normalize the data to eliminate the dimensional differences between different features so that they can be compared on the same scale. One common normalization method is to centralize the data by removing the mean, that is, normalization with 0 mean and 1 variance. In the field of medical imaging, the grayscale information of the same tissue may be inconsistent due to the acquisition and imaging factors of different devices. Image normalization is an image conversion method that reduces or even eliminates the grayscale inconsistency in the image while retaining the grayscale differences with diagnostic value, which is more conducive to computer automatic analysis and processing.

[0021] Finally, data enhancement is performed: 1). Flip. 2). Translation. (The translation operation is to translate the image up and down while keeping the integrity of the breast area unchanged.) 3). Mirror. At this point, the image data preprocessing is completed.

[0022] S3. Build a 2D convolutional neural network for breast segmentation;

[0023] Use the training set to train the convolutional neural network. First, read the training set, and then input the image into the improved convolutional network model (such as Figure 5 ), the input of the convolutional neural network is a 512*512 grayscale image, and the output is a 512*512 binary image. The network structure consists of three parts: the upper part, the lower part, and the right part. The upper part of the network is the encoder part of the input image, the lower part is the decoder part of the input image, and the right part is used to fuse the regional features and global features obtained by the two convolution branches;

[0024] The complementary feature extraction module of the network uses a parallel two-way convolution of the CNN and Transformer models (see Appendix Figure 5), and the dual-path convolution module consists of two branches. The first branch is composed of a CNN convolution model, and the second branch is composed of a Transformer convolution model (see Appendix Figure 2 ). Then the global features and multi-layer local features obtained by the two branches are input into the global context optimization module, which can effectively extract complementary global-local features and enhance the model's ability to identify the segmented target and its background. The second module is the global context optimization module (such as Figure 3 ). The main function is that this module can adaptively generate a set of global features to make up for the diluted semantic information, thereby enhancing the global feature representation of the model. Among them, the window self-attention module can perform self-attention calculations on each position in the input sequence to capture local dependencies. Specifically, the window self-attention mechanism performs self-attention calculations within a window of a certain size around each position to integrate local and global dependency information. Finally, the integrated features are subjected to convolution, activation, pooling and other operations to adaptively generate a set of global features specific to each decoding layer, and all local features are transmitted to the corresponding decoding layer to make up for the diluted semantic information. The global context optimization module controls and balances the global information flow in an adaptive manner, effectively overcomes the problem of global semantic information dilution, and enhances the global representation ability of the model. In the decoding stage, a mutual attention module (such as Figure 4 ), the module consists of five cascaded mutual attention blocks BA i , where i represents the corresponding sequence number in the mutual attention module. i It can decompose the input features into foreground and background information, increase the weights of the foreground and background areas, and enhance the representation ability of the foreground-background contrast features. Then the weights of the two are fused in a top-down manner to obtain the edge contour map, and finally the edge contour feature map is mapped to the input feature map to achieve edge enhancement and finally output the result. The mutual attention module can not only integrate the multi-scale features in the foreground and background views from top to bottom, but also provide mutual contrast guidance, improve the model's ability to distinguish the segmented target and its background, and enable the algorithm to effectively overcome the interference caused by complex backgrounds.

[0025] S4. Train the neural network using the training set to obtain a fully convolutional neural network model;

[0026] The processed 2D breast tumor ultrasound image data was input into the built LGCBNet network framework model. In the experiment, five-fold cross validation was used, the learning rate was set to 0.0001, the number of epochs was set to 200, L2 regularization was used to prevent overfitting, and the Adam optimizer was used to optimize the model. The model was trained.

[0027] After the training is completed, the obtained weight parameter file is stored in a folder.

[0028] S5. Use the obtained network model to segment the breast tumor ultrasound data in the test set and obtain the segmentation results.

[0029] Input the weight file into the network, then input the data in the test set into the network model, and finally output the segmented result. Use four commonly used performance indicators, Accuracy (ACC), Recall, Precision, and Dice, as direct expressions of the segmentation results. ACC is the most intuitive performance indicator for classification tasks. Accuracy and recall have a good effect on the performance judgment of small data networks. The specific function form of the ACC value is:

[0030]

[0031] Among them, TN, TP, FN, and FP represent the number of true negative, true positive, false negative, and false positive pixels, respectively.

[0032] Example 2

[0033] The present invention conducted an ablation experiment to test the role of different modules; by setting the U-Net network as the baseline of the ablation experiment and the complementary feature fusion module as the backbone, the importance of the complementary feature fusion module, the global context optimization module, and the boundary perception module for the segmentation task were tested respectively. The data obtained by different module combinations are used to test the role of each module.

[0034] Among them, the implementation of methods such as Transformer, CNN and maximum pooling are algorithms well known to technical personnel in this field, and the specific processes and implementation procedures can be found in corresponding textbooks or technical literature.

[0035] A schematic diagram comparing the corresponding evaluation indicators of the prior art and the method proposed in the present invention is shown in FIG. Figure 6 As shown in the figure, it can be seen that the accuracy and precision of the method proposed in the present invention are better than those of existing methods such as U-Net and U2Net; these indicators show that the method proposed in the present invention is better than the existing methods, has better generalization for breast tumor segmentation tasks, and has higher ability to resist interference from complex backgrounds. The corresponding evaluation index comparison diagram of the ablation experiment of the method proposed in the present invention is shown in the figure. Figure 7 As shown in the figure, it can be seen that each module of the present invention plays a certain role in improving the performance of the network.

[0036] Summary: In order to solve the problems of low generalization and susceptibility to complex background interference in the breast tumor segmentation task in ultrasound images faced by the field of breast tumor segmentation, the present invention designs an improved network based on CNN and Transformer parallel convolution, thereby improving the generalization of the network for breast tumor segmentation tasks and enhancing the network's ability to resist complex background interference.

[0037] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A breast ultrasound image segmentation method based on local-global feature complementation and boundary perception, characterized by: The method comprises the following steps: S1. Select a suitable data set, then select the ROI area and process the mask map of the data set; S2. Prepare the data set, preprocess the breast data, increase contrast, normalize and enhance the data. The data set includes a training set and a test set. S3. Build a 2D convolutional neural network for breast segmentation; S4. Train the neural network using the training set to obtain a fully convolutional neural network model; S5. Use the obtained network model to segment the breast tumor image data in the test set to obtain a segmentation result; S6. Conduct ablation experiments on the network to test the impact of different modules on the results.

2. The method for breast ultrasound image segmentation based on local-global feature complementation and boundary perception according to claim 1, characterized in that: In S2, the image is pre-processed by inverting the image left to right and flipping it upside down, in order to amplify the data.

3. The method for breast ultrasound image segmentation based on local-global feature complementation and boundary perception according to claim 1, characterized in that: In S3, the construction of the convolutional neural network is as follows: The input of the convolutional neural network is a 512*512 grayscale image, and the output is a 512*512 binary image. The network structure consists of three parts: upper, lower, and right. The upper side of the network is the encoder part of the input image, the lower side is the decoder part of the input image, and the right side is the part used to fuse the regional features and global features obtained by the two convolution branches. The convolution stage of the network adopts a dual-path convolution structure that uses CNN and Transformer in parallel. The global features and multi-layer local features obtained by the two branches are then input into the global context optimization module. Through the window self-attention module, self-attention calculation is performed on each position in the input sequence to capture local dependencies and integrate local and global dependency information. Finally, the integrated features are subjected to convolution, activation, and pooling operations to adaptively generate a set of global features specific to each decoding layer. All local features are transmitted to the corresponding decoding layer to make up for the diluted semantic information. In the decoding stage, a mutual attention module is designed. This module contains five cascaded mutual attention blocks BA i , B.A. i It can decompose the input features into foreground and background information, increase the weights of the foreground and background areas, and enhance the representation ability of the foreground-background contrast features. Then the weights of the two are fused in a top-down manner to obtain the edge contour map, and finally the edge contour feature map is mapped to the input feature map to achieve edge enhancement and output the final result.

4. The method for breast ultrasound image segmentation based on local-global feature complementation and boundary perception according to claim 1, characterized in that: In S4, a fully convolutional neural network based on the constructed LGCBNet framework is used to train the fully convolutional neural network with 2D breast tumor image data as input, and the image specification is a resolution of 1.5*1.5*1.5 mm 3.

5. The method for breast ultrasound image segmentation based on local-global feature complementation and boundary perception according to claim 1, characterized in that: In S5, four commonly used performance indicators, namely, accuracy (ACC), recall (Recall), precision (Precision), and dice coefficient (Dice), are used as direct expressions of the segmentation results. Among them: Among them, TN, TP, FN, and FP represent the number of true negative, true positive, false negative, and false positive pixels, respectively.

Images