Breast tumor molecular marker prediction method based on automatic breast full volume imaging

Abstract

The present application belongs to the technical field of artificial intelligence, and particularly relates to a breast tumor molecular marker prediction method based on automatic breast full-volume imaging. The method comprises the following steps: ABVS image preprocessing, and obtaining a region of interest containing a tumor; taking a cross section and two adjacent cross sections on both sides of the cross section passing through the center of the region of interest in the transverse plane, coronal plane and sagittal plane of the three-dimensional image respectively, and using a TransU-Net model to segment the tumor regions in the transverse plane, coronal plane and sagittal plane respectively; a deep neural network model is built, which is used for predicting the molecular marker of the breast tumor; the model has two branches, one of which is a three-dimensional residual convolutional network with a channel attention mechanism, which is used for processing three-dimensional images; the other branch is a convolutional network for processing the segmentation results of the three cross sections, and finally the features obtained by the two branches are integrated through a full connection layer; a labeled data set is used for supervised learning of the model, and a prediction model of the molecular marker is obtained. The present application can improve the prediction accuracy.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to an automatic prediction method for molecular markers of breast tumors. Background Technology

[0002] Molecular markers for breast tumors, including Her2, PR, KI-67, and ER, are fundamental for differentiating different subtypes of breast cancer. However, the identification of these molecular markers usually requires a biopsy, which is invasive. Therefore, non-destructive testing methods based on image analysis are of great significance.

[0003] Breast ultrasound imaging is a non-invasive imaging method widely used for the early detection and diagnosis of breast tumors. Traditional B-mode ultrasound examinations typically involve handheld probes scanning the breast step-by-step to acquire two-dimensional images, which suffers from problems such as operator dependence, incomplete scan coverage, and poor diagnostic consistency. Automated breast volume imaging (ABVS) technology, through automated whole-breast ultrasound scanning and data processing algorithms, can acquire high-quality three-dimensional breast ultrasound images.

[0004] In recent years, deep neural networks have made significant progress in the field of medical image analysis, enabling preliminary diagnosis of breast tumors based on images. For example, U-Net networks [1-3], FCN networks [4], and DenseNet networks [5-6] have been used for breast tumor segmentation, while ResNet networks [7-9] have been used for predicting the benign or malignant nature of breast tumors. However, there is currently no suitable model for predicting molecular markers.

[0005] Due to the diversity and complexity of breast tumors in terms of morphology, size, and location, the information provided by two-dimensional ultrasound images is often insufficient to train a suitable deep neural network. ABVS images, which provide three-dimensional information, contain richer information than two-dimensional ultrasound images, allowing for better training of deep neural networks and potentially achieving higher accuracy in molecular marker prediction. Therefore, this invention proposes a method for predicting molecular markers of breast tumors based on automated whole-volume breast imaging. Summary of the Invention

[0006] The present invention aims to propose a method for predicting molecular markers of breast tumors based on automated breast volume imaging (ABVS) to improve prediction accuracy.

[0007] The method for predicting molecular markers of breast tumors based on automated breast volume imaging (ABVS) provided by this invention consists of the following three steps: ABVS image preprocessing; obtaining tumor contours in three different directions passing through the center point of the region of interest; and establishing a deep neural network model that combines three-dimensional images and two-dimensional tumor contours to predict molecular markers of breast tumors.

[0008] The first step is ABVS image preprocessing to obtain the region of interest (VOI) containing the tumor; the specific steps are as follows:

[0009] (1) Resolution normalization of ABVS image: Obtain the resolution of the three dimensions of the original ABVS image, select the median value as the target resolution, and resample the resolution of each dimension according to the target resolution to make it consistent with the target resolution, so that the pixel resolution of the three dimensions of coronal plane, sagittal plane and transverse plane is the same.

[0010] (2) Based on the defined cube region containing the tumor, extend outward by 15 pixels in each dimension as the region of interest; and normalize its scale to 128×128×128.

[0011] The second step involves taking cross sections passing through the center of the region of interest and adjacent cross sections for the transverse, coronal, and sagittal planes in the three-dimensional image, respectively, and using the TransU-Net model

[10] (structure shown in Appendix) pre-trained on a public dataset. Figure 1 The tumor region was segmented in the transverse, coronal, and sagittal planes, respectively; the specific steps are as follows:

[0012] (1) Select three consecutive cross-sectional slices that pass through the center of the region of interest from the VOI and combine them to obtain an image with a scale of 128×128×3;

[0013] (2) Perform Z-score normalization on the image;

[0014] (3) Input the processed image into TransU-Net to obtain the segmentation results of two-dimensional slices of breast tumor; the model has been pre-trained on a public dataset of breast ultrasound image segmentation;

[0015] (4) For the coronal and sagittal planes, repeat steps (1), (2), and (3) in a similar manner to obtain the segmentation results;

[0016] (5) Perform morphological opening operation on the segmentation results to remove small regions in the segmentation results; remove connected components with an area of ​​less than 50 pixels to obtain the final segmentation result of breast tumor.

[0017] The third step is to build and train a deep neural network model to predict molecular markers for breast tumors; the model structure is shown in the attached figure. Figure 2As shown, the model consists of two main branches, one for the three-dimensional full-volume ultrasound image of the breast and the other for the segmented image of its two-dimensional cross-section. One branch is a three-dimensional residual convolutional network with an added channel attention mechanism to process the three-dimensional image; the other branch is a convolutional network to process the segmentation results of the three cross-sections. Finally, the features obtained from the two branches are integrated through a fully connected layer. The model is subjected to supervised learning using a labeled dataset to obtain a prediction model for molecular markers.

[0018] Specifically, the constructed deep neural network model includes module 1, module 2, and module 3; where:

[0019] Module 1's structure includes: a first convolutional layer, an intermediate sequential deep network, and a final channel attention layer; wherein:

[0020] The first convolutional layer has a kernel size of 7×7×7. To enhance the expressive power of the network, a single-channel attention mechanism is added afterward.

[0021] The serial deep network is divided into four sub-parts, each consisting of 3, 4, 10, and 3 Bottleneck modules connected in series, respectively. The number of output channels for each part is 32, 64, 128, and 128, respectively. The structure of the Bottleneck module is shown in the appendix. Figure 3 It consists of concatenated convolutional layers and ReLU layers with kernel sizes of 1×1×1, 3×3×3 and 1×1×1, respectively. The convolutional layer with a kernel size of 1×1×1 is responsible for processing different input and output dimensions.

[0022] The final channel attention layer is used to enhance the model's ability to focus on features of different channels;

[0023] Module 2 is an encoder consisting of four 2D convolutional layers and one ReLU layer, where each convolutional operation uses a 3×3 kernel. The input to Module 2 is the tumor segmentation results in the transverse, coronal, and sagittal planes from the previous steps, and the output is the information extracted from them.

[0024] Module 3 is a classifier consisting of two fully connected linear layers and a ReLU activation function. Module 3 combines the outputs of Module 1 and Module 2 and feeds them into the classifier to obtain the final output. ReLU is used to further enhance the non-linearity of feature representation. The model uses the cross-entropy loss function.

[0025] Finally, the model was trained using ABVS images with existing molecular marker labels to obtain a model that can be used to predict molecular markers of breast tumors.

[0026] Compared with the prior art, the present invention has the following advantages:

[0027] This invention uses ABVS to predict molecular markers. Compared with pathological examination, it does not require surgical sampling, and ultrasound detection is harmless to the human body, with low cost and high safety. Compared with two-dimensional ultrasound images, ABVS does not depend on the operator during acquisition and provides richer information, which is conducive to training an effective prediction model. Attached Figure Description

[0028] Figure 1 This is the TransU-Net model structure.

[0029] Figure 2 This is an ABVS-based molecular marker prediction model.

[0030] Figure 3 It is a Bottleneck structure. Detailed Implementation

[0031] Taking a three-dimensional ultrasound image acquired using a Siemens Acuson S2000 Automated Breast Volume Scanner as an example, the practical application process of this invention is described in detail, including the following steps:

[0032] Step 1, ABVS image preprocessing: The DICOM format ABVS images exported from the ultrasound instrument are resampled to ensure the same pixel resolution in the coronal, sagittal, and transverse dimensions. Based on the defined cubic region containing the tumor, the region of interest is extended outward by 15 pixels in each dimension.

[0033] Step 2: In the coronal, transverse, and sagittal directions of the 3D image, take three consecutive slices passing through the center of the VOI, and combine them to obtain three images with a scale of 128×128×3. Use the TransU-Net segmentation network pre-trained with the public dataset INbreast

[11] to obtain the segmentation results in the transverse, coronal, and sagittal planes. Perform morphological opening operations on each segmentation result and remove connected regions with an area of ​​less than 50 pixels to obtain the final segmentation result.

[0034] Step 3: Construct a deep neural network. Input the VOI into module 1, and send the segmentation results obtained from step 2 based on VOI into module 2. Combine the outputs of the two modules and send them to the classifier in module 3. Use ABVS images with existing molecular marker labels to train the model. Randomly divide the samples into training, validation, and test sets in an 8:1:1 ratio; random rotation is performed during training to amplify the samples.

[0035] After training the model, it was tested on a test set to obtain its predictive performance for molecular markers in breast tumors. Accuracy, sensitivity, specificity, and the area under the receiver operating characteristic (ROC) curve (AUC) were used as evaluation metrics. The results of the test on 73 cases are shown in Table 1. It can be seen that the model can predict commonly used molecular markers in breast tumors with a certain degree of accuracy.

[0036] Table 1. Molecular marker prediction results from the model.

[0037] Molecular markers accuracy Sensitivity Specificity AUC Her2 0.674 0.500 0.742 0.616 PR 0.767 0.821 0.667 0.648 KI-67 0.744 0.885 0.529 0.824 ER 0.698 0.677 0.750 0.774 .

[0038] References

[0039] [1] Ronneberger O, et al. "U-Net: Convolutional networks for biomedical image segmentation", Medical Image Computing and Computer-AssistedIntervention–MICCAI 2015:18th International Conference, 2015, 2015: 234-241.

[0040] [2] Vakanski A, et al. "Attention-enriched deep learning model for breast tumor segmentation in ultrasound images", Ultrasound in Medicine & Biology, 2020, 46(10): 2819-2833.

[0041] [3]Ning Z, et al. "SMU-Net: Saliency-guided morphology-aware U-Net forbreast lesion segmentation in ultrasound image", IEEE Transactions on MedicalImaging, 2021, 41(2): 476-490.

[0042] [4]Hu Y,et al."Automatic tumor segmentation in breast ultrasoundimages using a dilated fully convolutional network combined with an activecontour model",Medical Physics,2019,46(1):215-228.

[0043] [5]Dong Y,et al."Semantic segmentation with densenets for breasttumor detection",International Conference on Communications and BroadbandNetworking,2021,2021:54-59.

[0044] [6]Vulli A,et al."Fine-tuned DenseNet-169for breast cancer metastasisprediction using FastAI and 1-cycle policy",Sensors,2022,22(8):2988.

[0045] [7]Ferreira C A,et al."Classification of breast cancer histologyimages through transfer learning using a pre-trained Inception ResNet v2",International Conference Image Analysis and Recognition,2018,763-770.

[0046] [8]Jiang Y,et al."Breast cancer histopathological imageclassification using convolutional neural networks with small SE-ResNetmodule",PLoS One,2019,14(3):e0214587.

[0047] [9]Reenadevi R,et al."Breast cancer histopathological imageclassificationusing augmentation based on optimized deep ResNet-152structure".Annals of theRomanian Society for Cell Biology,2021,25(6):5866-5874.

[0048]

[10] Chen J,et al."TransUNet:Transformers make strong encoders formedical imagesegmentation.",ArXiv Preprint,2021,2021:2102.04306.

[0049]

[11] Moreira C,et al."INbreast:toward a full-field digitalmammographicdatabase",Academic Radiology,2012,19(2):236-48。

Claims

1. A breast tumor molecular marker prediction method based on automatic breast full volume imaging, characterized in that, The following three steps are divided: (1) ABVS image preprocessing, obtaining the VOI containing the tumor; (2) For the cross section, coronal section and sagittal section in the three-dimensional image, respectively take the section passing through the center of the region of interest and the two adjacent sections on both sides, use the TransU-Net model pre-trained on the public data set to segment the tumor area in the cross section, coronal section and sagittal section respectively; (3) Build a deep neural network model, train the model, and predict the molecular markers of breast tumors; The model has two main branches, respectively for three-dimensional breast full-volume ultrasound images and two-dimensional section images; One branch is a three-dimensional residual convolutional network with channel attention mechanism, used to process three-dimensional images; The other branch is a convolutional network for processing the segmentation results of three sections, and finally the features obtained by the two branches are integrated through a fully connected layer; Use the labeled data set to perform supervised learning on the model to obtain the prediction model of the molecular marker; In step (3), the deep neural network model includes module 1, module 2 and module 3; Among them: The structure of module 1 includes: the first layer of convolutional layer, the middle serial deep network, and the last layer of channel attention layer; Among them: The first layer of convolutional layer, the size of the convolution kernel is 7x7x7, and a single channel attention mechanism is added after it; The serial deep network is divided into 4 subparts, which are composed of 3, 4, 10 and 3 Bottleneck modules in series, respectively, and the output channel numbers of each part are 32, 64, 128 and 128 respectively; The Bottleneck module is composed of convolutional layers with convolution kernel sizes of 1x1x1, 3x3x3 and 1x1x1, and ReLU layers in series, wherein the convolutional layer with a convolution kernel size of 1x1x1 is responsible for processing different input and output dimensions; The last layer of channel attention layer is used to enhance the model's attention to different channel features; The structure of module 2 is an encoder composed of four two-dimensional convolutional layers and a ReLU layer, where each layer of convolutional operation uses a 3x3 convolution kernel; The input of module 2 is the tumor segmentation result in the cross section, coronal section and sagittal section in the previous step, and the output is the information extracted therefrom; Module 3 is a classifier composed of two fully connected linear layers and a ReLU activation function; Module 3 combines the outputs of module 1 and module 2 and sends them to the classifier to obtain the final output; ReLU is used to further enhance the nonlinear ability of feature representation; The cross-entropy loss function is used in the model; Finally, the ABVS image with existing molecular marker labels is used to train the model to obtain a model for predicting the molecular markers of breast tumors.

2. The breast tumor molecular marker prediction method according to claim 1, characterized in that, The specific process of step (1) is as follows: (1) ABVS image resolution normalization; Obtain the resolution of the three dimensions of the original ABVS image, select the middle value as the target resolution, and resample the resolution of each dimension according to the target resolution to make it consistent with the target resolution, so that the pixel resolutions of the coronal section, sagittal section and cross section in three dimensions are the same; (2) According to the cuboid region containing the tumor, expand 15 pixels on each dimension as the region of interest, and normalize its size to 128x128x128.

3. The breast tumor molecular marker prediction method according to claim 2, characterized in that, The specific process of step (two) is as follows: (1) Select three continuous cross-sectional slices passing through the center of the region of interest from the region of interest, and combine them to obtain an image with a size of 128x128x3; (2) Perform Z-score normalization processing on the image; (3) Input the processed image into TransU-Net to obtain the segmentation result of the two-dimensional slice of the breast tumor; (4) For the coronal and sagittal planes, repeat steps (1), (2), and (3) to obtain the segmentation result; (5) Perform morphological opening operation processing on the segmentation result to remove small regions in the segmentation result; remove connected domains with an area less than 50 pixel units to obtain the final segmentation result of the breast tumor.

Images