A benign and malignant classification system for ovarian tumors based on a hybrid radiomics model

Abstract

The present invention discloses a benign and malignant classification system for ovarian tumors based on a hybrid radiomics model. On the one hand, the backbone network in the Med3D network is used as a deep feature extractor to construct a target network, and the target network is fine-tuned with multi-modal 3D magnetic resonance imaging data to obtain the corresponding ovarian tumor classification network for obtaining deep features and tumor classification results. On the other hand, the omics features of the tumor region of interest are extracted, and the tumor classification results corresponding to the omics prediction model are obtained. In addition, the deep features and omics features are fused and the corresponding classification results are obtained through the KNN classification algorithm. Finally, the final benign and malignant classification results of ovarian tumors are obtained according to the weighted average of the above three classification results. The present invention comprehensively uses a variety of models and algorithms to realize more comprehensive feature mining and analysis of multi-modal MRI image data of ovarian tumor patients, thereby improving the final ovarian tumor classification effect.

Description

Technical Field

[0001] The present invention relates to the fields of radiomics and deep learning, and particularly to a benign and malignant classification system for ovarian tumors based on a hybrid radiomics model. Background Art

[0002] Borderline ovarian tumors (BOTs) and malignant ovarian tumors (MOTs) are two main types of ovarian tumors, which have significant differences in biological behavior, clinical characteristics, treatment strategies, and prognosis. BOT is an intermediate entity different from benign and malignant ovarian tumors. Most BOTs do not invade ovarian tissue, and conservative surgery with fertility preservation can be considered for BOT patients under appropriate circumstances. Malignant ovarian tumor is a highly malignant ovarian epithelial tumor, and its mortality rate is much higher than that of BOT. For MOT patients, the preferred treatment method is surgical resection combined with chemotherapy. Therefore, accurate preoperative diagnosis of borderline ovarian tumors and malignant ovarian tumors is crucial for optimal patient management. However, due to the great similarities between BOT and MOT in biological, histopathological, clinical, and imaging features (Reference: Borrelli G M, de Mattos L A, de Paula Andres M, et al. Role of imaging tools for the diagnosis of borderline ovarian tumors: a systematic review and meta-analysis[J]. Journal of minimally invasive gynecology, 2017, 24(3): 353-363.), the accurate preoperative differential diagnosis between the two remains a challenge.

[0003] Magnetic resonance imaging (MRI) is a non-invasive imaging technique commonly used in the clinical diagnosis of ovarian tumors and is currently also used in the standardized examination of ovarian tumor type diagnosis. After obtaining MRI image data, radiologists usually obtain disease-related information contained in the MRI through visual recognition. This method has significant limitations, such as high subjective dependence, low time efficiency, and insufficient exploration of image information, etc. In recent years, the rapidly developing radiomics method can achieve rapid and efficient ovarian tumor classification through the combination of MRI image data and artificial intelligence algorithms. According to different feature extraction strategies, radiomics methods can be roughly divided into traditional radiomics methods based on handcrafted features and deep learning-based methods. Traditional radiomics methods first extract tumor feature information such as shape and texture in a predefined manner, and then construct a feature screening and prediction model through traditional machine learning algorithms (Reference: Li Y, Jian J, Pickhardt P J, et al. MRI‐based machine learning for differentiating borderline from malignant epithelial ovarian tumors: A multicenter study[J]. Journal of Magnetic Resonance Imaging, 2020, 52(3):897-904.). Different from traditional radiomics methods, deep learning methods can implement the entire analysis process through a deep neural network, including automatic extraction of image features, feature screening, and output of prediction results, etc.; current studies have shown that data-driven deep learning methods can achieve high prediction accuracy in ovarian tumor classification (Reference: Wang Y, Zhang H, Wang T, et al. Deep learning for the ovarian lesion localization and discrimination between borderline and malignant ovarian tumors based on routine MR imaging[J]. Scientific Reports, 2023, 13(1):2770.).

[0004] It should be noted that although the deep learning-based method can simplify the analysis process and improve the prediction performance, it also has some disadvantages, such as a high demand for the amount of training data and poor interpretability. In contrast, for the traditional radiomics method, subjective selection is required for parts such as the feature extraction method, feature selection algorithm, and model construction, relying on the design experience of the algorithm designer. However, this type of method usually has good interpretability. In addition, the image features extracted by the deep learning method often focus on local features, while the features extracted by the traditional radiomics method focus more on the global features of tumors (Reference: Ning Z, Luo J, Li Y, et al. Pattern classification for gastrointestinal stromal tumors by integration of radiomics and deep convolutional features[J]. IEEE journal of biomedical and health informatics, 2018, 23(3): 1181-1191.). In summary, both the traditional radiomics method and the deep learning method have their own advantages and disadvantages. How to develop a hybrid radiomics model that combines the advantages of both has been a research hotspot in recent years. Summary of the Invention

[0005] The object of the present invention is to propose an ovarian tumor borderline and malignant classification system based on a hybrid radiomics model for the classification of ovarian borderline tumors and malignant tumors in view of the deficiencies of the prior art based on the above research status. The hybrid radiomics model proposed by the present invention can provide a certain degree of interpretability while improving the tumor classification performance. The present invention first constructs ovarian tumor classification models through deep learning and traditional radiomics methods respectively and obtains their respective prediction results; then fuses the deep features and omics features, and then obtains the corresponding prediction results based on the fused features and the K-nearest neighbor (KNN) algorithm; finally, comprehensively combines the above various prediction results to obtain the final tumor classification prediction result. In addition, the present invention adopts a training scheme based on three-dimensional transfer learning to solve the problem of insufficient training data volume faced during the training of the deep learning model, as well as the common data type differences and dimension differences in transfer learning.

[0006] The technical solution of the present invention is: an ovarian tumor borderline and malignant classification system based on a hybrid radiomics model, which specifically includes:

[0007] A data acquisition module for acquiring ovarian multimodal 3D magnetic resonance imaging data of different ovarian tumor patients and constructing a data set;

[0008] A tumor segmentation module for segmenting the region of interest of tumors in the dataset;

[0009] A network construction module for using the backbone network in the pre-trained Med3D model as a feature extraction network and jointly constructing a deep learning-based tumor classification network with a classifier network based on a fully connected layer;

[0010] A network training module for extracting the tumor part in the image data from the segmentation result of the region of interest of tumors, cropping it and using it as the input of the deep learning-based tumor classification network, and training based on the tumor classification label data;

[0011] A feature extraction module for extracting tumor radiomics features based on the 3D magnetic resonance image data of the patient and the segmentation result of the region of interest of tumors, and screening out key features;

[0012] An omics classification module for constructing an omics classification model based on a machine learning algorithm and training the omics classification model using the key feature data and tumor classification labels;

[0013] A feature fusion module for using the output of the feature extraction network in the trained tumor classification network as deep features, combining them with the tumor radiomics features to obtain fusion features, and constructing a KNN classification model based on the fusion features of the training data;

[0014] A tumor classification module for given the ovarian multi-modal 3D magnetic resonance image data of any ovarian tumor patient, processing it respectively with a hybrid model composed of a deep learning-based tumor classification network, an omics classification model and a KNN classification model to obtain three independent tumor classification results, and calculating the final tumor classification result of the hybrid model by weighted averaging these three classification results.

[0015] Furthermore, the ovarian multi-modal 3D magnetic resonance image data includes two modalities, T2 and DWI, and the data is pre-processed by denoising and resolution adjustment.

[0016] Furthermore, in the construction of the deep learning-based tumor classification network, in order to process multi-modal image data, a feature extraction network with the same network structure is used for each modality, and after extracting the deep features of each modality, they are concatenated to serve as the final deep features and input into the subsequent classifier network based on a fully connected layer.

[0017] Furthermore, the training of the tumor classification network based on deep learning is achieved through transfer learning. That is, first, the backbone network in the pre-trained Med3D model is used to initialize the feature extraction network part of each modality in the tumor classification network based on deep learning. Then, the 3D magnetic resonance imaging data of ovarian tumor patients is used to fine-tune and train the classifier network based on the fully connected layer and some convolutional layers of the feature extraction network.

[0018] Furthermore, when extracting the omics features of multi-modal imaging data, the omics features of each modality are extracted using the imaging data of each modality and the data of the tumor region of interest, and then they are concatenated to obtain the final omics features, and then subsequent feature screening processing is carried out.

[0019] Furthermore, in the construction of the omics classification model, the selection of the feature selection model and the prediction model based on machine learning algorithms is to select the model combination with the best classification performance on the validation set from multiple feature selection models and binary classification models through the traversal method.

[0020] Furthermore, in the feature fusion module, the process of obtaining the fused features is to first screen the deep features through a feature selection algorithm, and then concatenate the screened deep features with the key features selected from the tumor imaging omics features to obtain the fused features.

[0021] Furthermore, when inputting the imaging data of an ovarian tumor patient, the prediction process of the KNN model is to find the k cases in the training set that are most similar to the fused features of the patient's imaging data, and take the average value of the tumor classification results of these k cases as the survival prediction result of the patient. The specific value of k is determined by the method of cross-validation.

[0022] Furthermore, the final tumor classification result of the hybrid model is the weighted average result of the output values of three independent prediction models, and the weight coefficients are set as a group of fixed values based on experience, or determined by performing multiple linear regression analysis on the training data.

[0023] Furthermore, the hybrid model simultaneously outputs interpretable results, specifically, based on the patient's imaging data, several cases similar to it in the fused feature space are output for doctors' reference or comparative analysis.

[0024] Compared with the background art, the beneficial effects of the present invention are as follows. By comprehensively utilizing a deep learning model, a traditional radiomics analysis model, and the KNN algorithm, the present invention realizes more comprehensive feature mining and analysis of multi-modal MRI image data of ovarian tumor patients, thereby improving the final ovarian tumor classification effect. By adopting a three-dimensional transfer learning training scheme and a data augmentation technique, the present invention solves to a certain extent the problem of insufficient training data volume in the training process of a deep neural network, as well as the common problems of data type differences and dimension differences in transfer learning, thereby improving the prediction performance of the deep learning model and the effectiveness of deep features. In addition, by combining the deep learning model, the traditional radiomics analysis method, and the KNN algorithm, the hybrid radiomics model proposed by the present invention has a certain interpretability while achieving higher tumor classification performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1 It is a schematic diagram of the hybrid radiomics model proposed by the present invention.

[0027] Figure 2 It is a schematic diagram of the three-dimensional transfer learning scheme in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] In order to make the purpose, technical solutions, and advantages of the present invention more clear, the following further illustrates the present invention with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0029] The present invention provides an ovarian tumor benign and malignant classification system based on a hybrid radiomics model. The schematic diagram of the model adopted by the system of the present invention is as Figure 1As shown in the figure, the backbone network in the Med3D model trained with a large number of 3D medical image datasets is used as the deep feature extraction network, and several fully connected layers are used as the classifier network to jointly construct an ovarian tumor classification network. After collecting and organizing the T2 modality and DWI modality MRI data of ovarian tumor patients, preprocess them first, and perform the segmentation of the tumor region ROI. Use the multi-modal MRI image data and tumor classification label data of ovarian tumor patients as the training dataset to fine-tune and train the fully connected layers and some convolutional layers of the tumor classification network. Then, on the one hand, extract the deep features of the MRI data through the trained multi-modal deep learning model and obtain the first corresponding deep learning model classification prediction value; on the other hand, extract the omics features of the multi-modal MRI image data through traditional radiomics methods and obtain the second corresponding omics model classification prediction value through traditional machine learning algorithms. In addition, perform a fusion analysis on the deep features and omics features, and obtain the third corresponding classification prediction value based on the fusion features and the KNN model. Finally, take the weighted average result of the above three prediction values as the final ovarian tumor classification result. Next, the processing flow of each module in the ovarian tumor benign and malignant classification system described in the present invention will be introduced in detail according to a specific embodiment of the present invention, as follows:

[0030] The data acquisition module is used for data collection and preprocessing. First, collect the multi-modal (T2, DWI) ovarian 3D MRI image data of 419 ovarian tumor patients (including 107 borderline cases and 312 malignant cases) from a certain hospital, as well as the corresponding tumor benign and malignant category label data. To ensure sample balance as much as possible and improve data quality, select 150 cases with better image quality from the malignant ovarian tumor data and all the borderline ovarian tumor data for subsequent experiments. Resample these 3D MRI image data to uniformly adjust the spatial resolution in the x, y, and z directions to 1.5 × 1.5 × 7mm, and the image pixel resolution is 512 × 512 × 24. In addition, register and align the image data of each modality. Randomly select 80% of all patient data as the training set, and the remaining 20% as the test set.

[0031] The tumor segmentation module is used for tumor ROI region segmentation. Realize the segmentation of the ovarian tumor ROI region through the semi-automatic segmentation function in the medical image segmentation tool ITK-SNAP. After obtaining the semi-automatic segmentation result, fine-tune the segmentation result to achieve the accuracy and reliability of the segmentation result.

[0032] The network construction module is used for the construction of a multi-modal deep learning tumor classification model. As Figure 2As shown in the figure, the backbone network in the pre-trained Med3D model based on the 3DSeg dataset (a summary dataset composed of 23 publicly available 3D medical image datasets) (implemented based on 3D-ResNet18) is used as the feature extraction network for each modality of image data. The output of the above feature extraction network is used as the deep feature, and the deep features extracted from the T2 modality and DWI modality image data are concatenated to obtain the deep feature of the multi-modal image data, which is used as the input of the fully connected layer classifier. The fully connected layer classifier consists of three layers in total, and the number of nodes in the first, second, and third layers are 1153, 128, and 2 respectively. The outputs of the two nodes in the last layer are used as the predicted probabilities of the tumor types of borderline ovarian tumors and malignant ovarian tumors respectively.

[0033] A network training module for training the deep learning model. First, the mask data of the tumor ROI is used to extract the tumor part of the multi-modal image, that is, centered on the tumor, a tumor part with a size of 256 × 256 × 12 is cropped as the input of the deep learning model, and the true label of the tumor category is used as the training label. After the above preprocessing of the training dataset, the deep learning model is fine-tuned. The training part includes the last module of the feature extraction network (including four convolutional layers) and all fully connected layers, as Figure 2 shown in the figure. The training parameter batch size is set to 16, the optimizer is Adam, the learning rate is 0.001, and the maximum number of iterations epoch is 50. After training is completed, the feature extraction network corresponding to each modality is obtained for extracting deep features, that is, when inputting a multi-modal 3D MRI image data, the deep features of each modality can be extracted through the trained feature extraction network, and the final deep feature data can be obtained through concatenation.

[0034] A feature extraction module for extracting multi-modal omics features. Based on the 3D MRI image data in the data acquisition module and the tumor ROI segmentation result mask data in the tumor segmentation module, the pyradiomics library is used to extract the omics feature data of the tumor ROI region of each modality of image data. The same types and quantities of omics features are extracted for each modality. The categories of omics features mainly include first-order statistical features, texture features, and high-order transformation features, etc. The omics features of each modality are concatenated to obtain the final multi-modal omics feature data.

[0035] Omics classification module, used for the construction and training of traditional omics classification models. First, pre-screen the above-mentioned omics feature data, that is, delete the features with low variance (variance < 0.05) and the features with no significant difference between groups (t-test, p > 0.05). Then, through the traversal method, select the optimal model combination of "feature selection + classification prediction" with the best classification performance on the validation set (randomly select 20% from the training set as the validation set) as the final omics classification model, and the small number of omics features screened by the feature selection model are called key features. The three traversal dimensions are: feature selection models (Lasso, HSICLasso, ANOVA, MRMRe, AdaBoost), binary classification models (LogisticRegression, SVM, RandomForest, GradientBoosting), and the number of key features (10, 20, 30, 40, 50).

[0036] Feature fusion module, used for feature fusion and the construction of the KNN prediction model. As Figure 1 shown, the multi-modal depth features and multi-modal omics features are respectively screened for features and then spliced and fused to obtain fused features, where the depth feature screening is realized by the HSICLasso algorithm. Then, based on the fused features of the training data and the KNN algorithm, a KNN prediction model is constructed. After the model is constructed, given the multi-modal image data of a new patient, its fused features can be extracted and input into the KNN prediction model. The KNN model calculates the 15 training data set cases that are most similar to the patient in the fused feature space, and finally obtains the prediction result of the patient's tumor category by statistically analyzing the true labels of the tumor categories of these 15 cases.

[0037] A tumor classification module is used for the construction and performance evaluation of a hybrid radiomics model. Based on the test data set in the data acquisition module, first, the method in the tumor segmentation module is used to segment the tumor ROI in the imaging data. Then, the deep learning prediction model trained by the network training module is used to process the test set data to obtain the tumor class prediction probability (P1) of the deep learning model and the deep feature data. At the same time, the omics feature data is extracted by the method in the feature extraction module and input into the trained omics prediction model in the omics classification module to obtain the tumor class prediction probability (P2) of the omics model. In addition, the fusion feature data of the test data set is obtained by the method in the feature fusion module and input into the KNN model therein to obtain the tumor class prediction probability (P3) of the KNN model. The weighted average result of the above three prediction probabilities is used as the final prediction result of the hybrid radiomics model. In the application of the present invention, it is considered that the prediction results of the three models are equally important, so the average value (P1 + P2 + P3) / 3 is directly taken as the final prediction result. By comparing the final prediction result of the test set with its true survival category label, the classification performance of the hybrid model on the test set can be calculated (the classification performance is characterized by the commonly used AUC index in the medical field). The experimental results show that the AUC of the test set of the hybrid radiomics model proposed by the present invention can reach about 0.92 in the task of ovarian tumor benign and malignant classification. By comparing this index with some existing methods in recent years, it can be seen that the hybrid model of the present invention has a significant improvement in the performance of ovarian tumor benign and malignant classification.

[0038] The above is a specific embodiment of a system for classifying benign and malignant ovarian tumors based on a hybrid radiomics model provided by the present invention. Through the above 8 main modules, a classification model for benign and malignant ovarian tumors based on multi-modal (T2 and DWI) 3D MRI image data and the performance evaluation results of the model can be obtained. Then, given the multi-modal 3D MRI image data of any ovarian tumor patient, the corresponding prediction result of the benign and malignant tumor class can be obtained through the above prediction model.

[0039] The present invention has good scalability. The present invention provides a mode of combining a deep learning model with a traditional radiomics model. For the selection of the specific types of ovarian tumors in the above embodiments, the selection of modalities in multi-modal imaging data, the definition of the tumor ROI area, and the selection of the feature extraction network structure of the deep learning model, the tumor ROI segmentation algorithm, the feature selection algorithm, etc., can all be appropriately adjusted according to the specific situations of other embodiments.

[0040] The above embodiments are used to explain the present invention, rather than to limit the present invention. Any modification and change made to the present invention within the spirit and scope of the protection of the claims of the present invention fall within the protection scope of the present invention.

Claims

1. A benign and malignant classification system for ovarian tumors based on a hybrid radiomics model, characterized in that, The system specifically includes: A data acquisition module, which is used to acquire ovarian multi-modal 3D magnetic resonance imaging data of different ovarian tumor patients and construct a data set; A tumor segmentation module, which is used to segment the regions of interest of tumors in the data set; A network construction module, which is used to use the 3D-ResNet18 backbone network in the pre-trained Med3D model as a deep feature extraction network, and jointly construct a deep learning-based tumor classification network with a classifier network based on a fully connected layer; A network training module, which is used to extract the tumor part in the image data from the segmentation results of the regions of interest of tumors, crop it and use it as the input of the deep learning-based tumor classification network, and train it based on the tumor classification label data; A feature extraction module, which is used to extract the tumor imaging omics features of each modality based on the 3D magnetic resonance imaging data of patients and the segmentation results of the regions of interest of tumors, and screen out the key features; An omics classification module, which is used to construct an omics classification model based on a machine learning algorithm, and train the omics classification model with the key feature data and tumor classification labels; during the construction of the omics classification model, the selection of the feature selection model and the prediction model based on the machine learning algorithm is to select the model combination with the best classification performance on the validation set from multiple feature selection models and binary classification models through the traversal method; A feature fusion module, which is used to use the output of the feature extraction network in the trained tumor classification network as the deep feature. After the deep feature is screened by the HSICLasso algorithm, it is combined with the tumor imaging omics features to obtain the fusion feature; a KNN classification model is constructed based on the fusion feature of the training data, that is, the tumor classification mean of the K cases most similar to the fusion feature of the input case is used as the KNN prediction result; A tumor classification module, which is used to give the ovarian multi-modal 3D magnetic resonance imaging data of any ovarian tumor patient, and process it respectively with a hybrid model composed of a deep learning-based tumor classification network, an omics classification model and a KNN classification model to obtain three independent tumor classification results, and calculate the final tumor classification result of the hybrid model by weighted averaging these three classification results.

2. The ovarian tumor benign and malignant classification system based on a hybrid radiomics model according to claim 1, wherein The ovarian multi-modal 3D magnetic resonance imaging data includes two modalities, T2 and DWI, and the data is preprocessed by denoising and resolution adjustment.

3. The ovarian tumor benign and malignant classification system based on the hybrid radiomics model according to claim 1, wherein In the construction of the deep learning-based tumor classification network, in order to process multi-modal image data, a 3D-ResNet18 feature extraction network with the same network structure is used for each modality. After extracting the deep features of each modality, they are spliced to serve as the final deep feature and input into the subsequent classifier network based on a fully connected layer.

4. The ovarian tumor benign and malignant classification system based on a hybrid radiomics model according to claim 1, wherein The training of the deep learning-based tumor classification network is realized through transfer learning, that is, first use the 3D-ResNet18 backbone network in the pre-trained Med3D model to initialize the feature extraction network part of each modality in the deep learning-based tumor classification network, and then use the 3D magnetic resonance imaging data of ovarian tumor patients to finely tune and train the classifier network based on a fully connected layer and some convolutional layers of the feature extraction network.

5. The ovarian tumor benign and malignant classification system based on the hybrid radiomics model according to claim 1, wherein The final result of tumor classification by the hybrid model is the weighted average of the output values of three independent prediction models, where the weight coefficients are set as a fixed set of values empirically or determined by performing multiple linear regression analysis on the training data.

6. The ovarian tumor benign and malignant classification system based on the hybrid radiomics model according to claim 1, characterized in that, The hybrid model simultaneously outputs interpretable results, specifically, based on the patient's imaging data, several cases similar to it in the fused feature space are output for doctors' reference or comparative analysis.

Images