Precise preoperative prediction system and method for ovarian cancer based on multimodal imaging omics features

Abstract

The present invention discloses a system and method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features. Based on US images, CT images and MRI images, the system extracts radiomic features from these three types of modal imaging data and performs combined screening to determine the matching classifier, prediction task and corresponding optimal feature combination. In this way, the multimodal radiomics features, classifiers and prediction tasks at different levels are optimally matched, and then the optimal match is used to achieve accurate preoperative diagnosis of ovarian cancer pathology, staging and PDS effect. This prediction process is close to the clinical practice diagnosis and treatment ideas, and the most diagnostically valuable imaging detection suggestions are proposed for different diagnosis and treatment stratifications.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent medical technology, and specifically relates to a system and method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features. Background Art

[0002] Ovarian cancer can be treated with a variety of treatment options, including surgical resection combined with systemic chemotherapy, neoadjuvant chemotherapy (NACT), PARP inhibitors, and immunotherapy, marking the entry into the era of precision medicine. Accurate diagnosis of ovarian cancer, including clinical staging, pathological classification, and heterogeneity, is the foundation for precision treatment. However, this information is often obtained during or after surgery. Therefore, accurately diagnosing and predicting ovarian cancer before surgery is a major medical challenge facing clinical practice.

[0003] Imaging examinations such as ultrasound, CT (computed tomography), and MRI (magnetic resonance imaging) play an important role in the screening and diagnosis of ovarian cancer. Ultrasound is primarily used for the initial assessment of ovarian space-occupying lesions. Ultrasound includes color Doppler technology to characterize the nature of tumor neovascularization. CT is primarily used to assess peritoneal metastasis and clinical staging. MRI's high-resolution soft tissue images further clarify potential malignant changes revealed by ultrasound. In clinical practice, ultrasound, CT, and MRI play complementary and cross-validating roles. Transvaginal ultrasound and its Doppler imaging technology, in particular, play a key role in ovarian cancer diagnosis and tumor neovascularization analysis. Currently, imaging diagnosis still primarily relies on visual image information acquired by the physician, combined with relevant clinical information and expert prior knowledge. Limited by the subjective nature of the images and knowledge of the experts, clinical practice lacks a method that can accurately predict the pathological type, clinical stage, efficacy of NACT, and the outcome of PDS (primary cytoreductive surgery) before surgery.

[0004] The integration of existing imaging images primarily refers to image fusion, which involves physically registering different images to provide spatial anatomical information about the tumor from different angles and dimensions. Technically, medical image registration applies image fusion techniques by geometrically aligning two images and overlapping the two input source images to produce a final image with redundant and complementary information. Two points must be met in the image fusion step: 1) all appropriate medical information present in the input images must appear in the synthesized image, and 2) the fused image should not contain any additional information that was not present in the input images. This fusion is applied to multi-sensor images obtained from imaging modalities derived from different sources, multi-focus images typically acquired from the same modality, and multimodal images widely used in medicine. Obviously, local imaging from conventional ultrasound is difficult to fuse with other images in the manner described above.

[0005] Although artificial intelligence (AI) algorithms have driven rapid progress in medical image information mining, and intelligent assisted diagnosis methods and predictive / diagnostic models have been reported, few methods have yet been applied to clinical practice. This is because existing research is largely based on single imaging techniques, whereas in clinical practice, experts aggregate information from multiple imaging examinations before making clinical decisions. The clinical implementation of AI-based medical imaging still faces the technical challenge of objectively and accurately integrating multiple imaging examinations to establish an assisted diagnosis paradigm that aligns with clinical practice. Summary of the Invention

[0006] In view of the above, the purpose of the present invention is to provide a system and method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features. Through a new way of integrating non-spatially aligned multimodal images, the integration of preoperative ultrasound, CT, and MRI images of ovarian cancer is achieved, and based on the integrated radiomics features, accurate preoperative multi-level classification of ovarian cancer is predicted.

[0007] To achieve the above-mentioned purpose of the invention, the embodiment provides a system for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features, comprising the following steps:

[0008] A data acquisition and preprocessing module, which is used to obtain and preprocess clinical data, multimodal imaging data, and text reports related to ovarian cancer, wherein the multimodal imaging data includes US images, CT images, and MRI images;

[0009] The image data delineation module is used to automatically segment the regions of interest of the pre-processed multimodal image data based on the segmentation model constructed by the neural network, and obtain the multimodal regions of interest through manual review and refinement;

[0010] A feature extraction and screening module is used to extract and normalize the radiomic features of multimodal regions of interest, and then screen the normalized radiomic features using a feature screening scheme based on at least one of analysis of variance, Spearman correlation, statistical test, and feature selection to obtain radiomic features selected for each modality;

[0011] A feature combination and classifier matching module, configured to determine a matching classifier, a prediction task, and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics feature of at least one modality on prediction tasks at each level in the classifier;

[0012] The preoperative accurate prediction module is used to combine at least one of the clinical data and text reports with the best features, and then use the matching classifier to make accurate predictions for the corresponding prediction tasks to obtain accurate preoperative prediction results.

[0013] Preferably, the segmentation model constructed based on the neural network automatically segments the regions of interest on the pre-processed multimodal image data, including:

[0014] For US images, a segmentation model constructed by fine-tuning 2D nn-Unet is used to automatically segment the regions of interest of US images. Specifically, for the US-Color image corresponding to the color standard plane contained in the US image, each channel in its RGB is input into the segmentation model as a layer output, and the union of the three layers of automatic segmentation results corresponding to the three layers of input is taken as the automatic segmentation result of the region of interest of the US-Color image; for the US-Gray image corresponding to the gray standard plane contained in the US image, it is directly input into the segmentation model to obtain the automatic segmentation result of the region of interest.

[0015] Preferably, the segmentation model constructed based on the neural network automatically segments the regions of interest on the pre-processed multimodal image data, including:

[0016] For CT images and MRI images, a segmentation model constructed by fine-tuning the volume-based nn-Unet was used to automatically segment the regions of interest (ROIs) of CT images and MRI images, respectively. Specifically, CT images were input into the segmentation model as single-channel images for automatic ROI segmentation, while MRI images contained MRI-DWI images and MRI-T2 images, which were combined into dual-channel images, were input into the segmentation model for automatic ROI segmentation.

[0017] When fine-tuning the volume-based nn-Unet, the volume-based nn-Unet was fine-tuned using the ovarian cancer dataset, the benign data subset extracted from the ovarian cancer dataset, and the non-benign data subset to obtain three different segmentation models. When segmenting CT images or MRI images, the segmentation results of the three segmentation models were combined to determine the final automatic segmentation result of the region of interest.

[0018] Preferably, the multimodal regions of interest are obtained through manual review and refinement, including:

[0019] The automatic segmentation results of each region of interest are manually reviewed and refined, including: first, judging whether there are holes inside the mask in the automatic segmentation results of each region of interest. Holes with an area of less than or equal to 20 pixels are defaulted to being mistakenly drawn during annotation, and the holes are filled; for holes with an area greater than 20 pixels, manually confirm whether they are intentionally hollowed out. If not, fill them; second, calculate the number of connected domains in the original mask, remove small connected domains with a volume of less than 50 pixels, and manually confirm whether there are problems with the mask annotation for data that still have multiple connected domains after processing. If there are mistakes, remove them; third, perform resolution sampling on the original image and the mask in the automatic segmentation results of the region of interest to align the two completely to obtain the region of interest of each modality.

[0020] Preferably, extracting radiomic features of multimodal regions of interest includes:

[0021] For US images, radiomic features are extracted from the US-Color image and US-Gray image contained in the US image. The radiomic features include first-order statistics, shape features, and grayscale co-occurrence matrix features. When the US-Color image is missing, the US-Gray image is used as the three channels to generate the RGB US-Color image.

[0022] For CT images and MRI images, the extracted radiomic features include first-order features, grayscale co-occurrence matrix features, grayscale run length matrix features, grayscale size area matrix features, grayscale correlation matrix features, and adjacent gray tone difference matrix features. During the feature extraction process, CT images and MRI images are resampled to maintain the feature space dimension, and the extracted radiomic features are also filtered.

[0023] Preferably, the statistical test includes Fisher score, Chi2 algorithm or Relief algorithm, and the constructed feature screening scheme includes:

[0024] Option 1: ANOVA, Spearman correlation, Fisher score, and feature selection;

[0025] Option 2: ANOVA, Spearman correlation, Chi2 algorithm, and feature selection;

[0026] Option 3: ANOVA, Spearman correlation, Relief algorithm, and feature selection;

[0027] Option 4: ANOVA, Spearman correlation, and Fisher score;

[0028] Option 5: ANOVA, Spearman correlation, and Chi2 algorithm;

[0029] Solution 6: ANOVA, Spearman correlation, and Relief algorithm;

[0030] Among them, for each modality-normalized radiomics feature, variance analysis is used to remove low-variance features exceeding a preset variance threshold; Spearman correlation is used to identify and remove highly correlated features exceeding a first correlation threshold between groups; the correlation between features and target labels is evaluated using Fisher score, Chi2 or Relief algorithm, and low-correlation features below a second correlation threshold are removed; feature selection is used to reduce the coefficients of less important features to zero, and the remaining features are ranked by importance;

[0031] The target labels are the classification labels of the prediction tasks at each level.

[0032] Preferably, determining the best matching classifier, prediction task, and corresponding best feature combination based on the classification effect of the feature combination formed by at least one type of radiomics feature of at least one modality on the prediction tasks at each level in the classifier includes:

[0033] Extracting features from the radiomic features of all modalities to form a feature combination according to predetermined feature types and quantities;

[0034] Each feature combination is input into various classifiers corresponding to the prediction tasks at each layer for inference and prediction, and the feature combinations are screened and removed using the three indicators of area under the curve (AUC), sensitivity (Sens), and specificity (Spec) calculated based on the prediction results;

[0035] The feature combination with the largest AUC index is selected from the remaining feature combinations after screening and removal as the best feature combination, and the classifier and prediction task corresponding to the best feature combination are used as the best classifier and prediction task matching the best feature combination;

[0036] The prediction tasks at each level include the first-level prediction task of classifying according to benign or malignant tumors, the second-level prediction task of classifying malignant tumors into epithelial ovarian cancer and non-epithelial ovarian cancer, the third-level prediction task of classifying epithelial ovarian cancer into early (FIGO I&II) and late (FIGO III&IV), the fourth-level prediction task of classifying late-stage epithelial ovarian cancer into high-grade serous carcinoma and non-high-grade serous carcinoma according to pathological tissue type, and the fifth-level prediction task of classifying late-stage high-grade serous carcinoma into R0 and non-R0.

[0037] Preferably, the feature combination is screened and removed by using the three indicators AUC, Spec, and Sens calculated based on the prediction results, including:

[0038] First, we remove the feature combinations whose AUC, Spec, and Sens indicators on the training set are all smaller than those on the validation set;

[0039] Then, based on the set AUC threshold, Spec threshold, and Sens threshold, the remaining feature combinations are further screened and removed. Specifically, the feature combinations that are greater than each threshold in both the validation set and the test set are retained. If there is no satisfactory feature combination, the Spec threshold and the Sens threshold are first reduced and updated, and the feature combinations that are greater than the updated Spec threshold and the Sens threshold in both the validation set and the test set are retained. If there is no satisfactory feature combination, the AUC threshold is reduced and updated again, and the feature combinations that are greater than the updated AUC threshold in the validation set and the test set are retained. The threshold update and retention screening process are repeated until a feature combination that is greater than all thresholds is obtained.

[0040] Preferably, the classifier includes support vector machine, XGBoost, Logistic regression, and random forest.

[0041] To achieve the above-mentioned purpose of the invention, the embodiment further provides a method for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features, comprising the following steps:

[0042] Obtain and preprocess clinical data, multimodal imaging data, and text reports related to ovarian cancer. The multimodal imaging data includes US images, CT images, and MRI images.

[0043] The segmentation model built based on the neural network automatically segments the regions of interest of the pre-processed multimodal image data and obtains the multimodal regions of interest after manual review and refinement;

[0044] After extracting and normalizing the radiomic features of the multimodal regions of interest, a feature screening scheme based on at least one of analysis of variance, Spearman correlation, statistical test, and feature selection is used to screen the normalized radiomic features to obtain the radiomic features selected for each modality;

[0045] Determining a matching classifier, a prediction task, and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics feature of at least one modality on prediction tasks at each level in the classifier;

[0046] After combining at least one of the clinical data and text reports with the optimal features, the matching classifier is used to perform accurate prediction of the corresponding prediction task to obtain accurate preoperative prediction results.

[0047] To achieve the above-mentioned purpose of the invention, an embodiment further provides a computing device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features.

[0048] To achieve the above-mentioned purpose of the invention, the embodiment further provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the above-mentioned method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] Based on US images, CT images and MRI images, the radiomics features of these three types of modal imaging data are extracted and combined to determine the matching classifier, prediction task and corresponding optimal feature combination. In this way, the multimodal radiomics features, classifiers and prediction tasks at different levels are optimally matched. Then, the optimal match is used to achieve accurate preoperative diagnosis of ovarian cancer pathology, staging and PDS effect prediction. This prediction process is close to the clinical practice diagnosis and treatment ideas, and the most diagnostically valuable imaging detection recommendations are proposed for different diagnosis and treatment stratifications. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 is a schematic structural diagram of a system for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features provided in an embodiment;

[0053] Figure 2 is a hierarchical distribution diagram of prediction tasks at each level provided by the embodiment;

[0054] Figure 3 is a schematic diagram of the region of interest segmentation process provided by the embodiment;

[0055] Figure 4 This is a flowchart of the execution flow of the system for accurate preoperative prediction of ovarian cancer provided in the embodiment;

[0056] Figure 5 This is a flowchart of a method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features provided in an embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0058] like Figure 1 and Figure 4 As shown, the embodiment provides an accurate preoperative prediction system 100 for ovarian cancer based on multimodal imaging genomics features, including a data acquisition and preprocessing module 110, an imaging data delineation module 120, a feature extraction and screening module 130, a feature combination and classifier matching module 140, and an accurate preoperative prediction module 150.

[0059] In this embodiment, the data acquisition and preprocessing module 110 is used to acquire and preprocess clinical data, multimodal imaging data, and text reports related to ovarian cancer. The clinical data related to ovarian cancer includes age, BMI, gravidity, parity, menopausal status, and six major tumor markers: CEA, AFP, SCC, CA125, CA199, and CA153. The multimodal imaging data includes US images, CT images, and MRI images. The text reports contain descriptions of the liver / spleen / pancreas, peritoneum, lymph nodes, and moderate to large ascites. These descriptions are extracted from the text reports as metastasis information, which is then used for subsequent preoperative prediction tasks.

[0060] In the embodiment, according to the clinical diagnosis and treatment ideas of ovarian cancer masses, the ovarian cancer prediction task is divided into multi-level prediction tasks, and the corresponding hierarchical relationships are as follows: Figure 2 As shown in the figure, the first layer is stratified according to benign or malignant, the second layer divides malignant tumors into epithelial ovarian cancer and non-epithelial ovarian cancer, the third layer divides epithelial ovarian cancer into early (FIGO I&II) and late (FIGOIII&IV), the fourth layer divides late-stage epithelial ovarian cancer into high-grade serous carcinoma and non-high-grade serous carcinoma according to pathological tissue type, and the fifth layer divides late-stage high-grade serous carcinoma into R0 and non-R0. Subsequently, the obtained clinical data, text reports and multimodal data are used to perform prediction tasks at each level to achieve accurate preoperative prediction of ovarian cancer.

[0061] The acquired clinical data and text reports were preprocessed, including unified representation, dataset segmentation, missing data imputation, and normalization. First, the clinical variables in each clinical data set were converted to numeric values. For example, "yes" was converted to "1" and "no" was converted to "0." Missing data for tumor markers and metastasis information were imputed using the median of the feature in each dataset. All data were normalized using the z-score using the mean and standard deviation from the training set.

[0062] The acquired multimodal data is preprocessed, including: for CT and MRI images, the standard format is DICOM, and they are converted to Nifty format for processing. For US images, the US-Color image corresponding to the color standard plane and the US-Gray image corresponding to the gray standard plane are in DICOM format, and the corresponding label image format is NRRD. Preprocessing of US images may include: 1) Color space alignment. DICOM images can contain three different color spaces: RGB, YBR_FULL_422, and MONOCHROME2. Images in other color spaces are aligned to the RGB color space to ensure consistency of color representation. 2) Cropping of the dual planes in the US image (US-Color image or US-Gray image). To maintain consistency in the training data, a dual plane and its corresponding label should be divided into two independent parts. Specifically, the cropping line is found by calculating the minimum pixel value in each column of the dual plane, and then the dual plane is separated based on the cropping line. 3) Labeling the US image based on the cropping line. Rename the cropped US images and labels by adding suffixes “-1” and “-2” to their original file names, where “-1” represents the left image and “-2” represents the right image.

[0063] In the embodiment, the image data delineation module 120 is used to automatically segment the region of interest of the pre-processed multimodal image data based on the segmentation model constructed by the neural network and obtain the multimodal region of interest through manual review and refinement. The region of interest is abbreviated as ROI region, specifically the lesion area of interest for preoperative prediction of ovarian masses, etc. The segmentation process is as follows: Figure 3 shown.

[0064] A segmentation model constructed by fine-tuning a 2D nn-Unet was used to automatically segment regions of interest (ROIs) in US images. The segmentation model was trained prior to use. US images were loaded into the segmentation software, and ultrasound physicians with over five years of experience delineated the ROIs in the US images. The boundaries of the entire tumor mass were outlined on US-gray images, and the solid components of the mass, including solid components, papillary processes, and cystic components, were outlined on US-color images. The 2D nn-Unet was fine-tuned using US-color and US-gray images of ovarian tumors as input samples and the manually outlined ROIs as labels to generate the segmentation model.

[0065] When applying the trained segmentation model to perform US image segmentation, it specifically includes: for the US-Color image corresponding to the color standard plane contained in the US image, each channel in its RGB is input into the segmentation model as a layer output, and the three layers of automatic segmentation results corresponding to the three layers of input are combined as the automatic segmentation result of the region of interest of the US-Color image; for the US-Gray image corresponding to the gray standard plane contained in the US image, it is directly input into the segmentation model to obtain the automatic segmentation result of the region of interest.

[0066] For CT images and MRI images, a segmentation model constructed by fine-tuning the volume-based nn-Unet is used to automatically segment the regions of interest of CT images and MRI images respectively. Specifically, the CT image is input as a single-channel image into the segmentation model for automatic segmentation of the region of interest, and the MRI-DWI image and MRI-T2 contained in the MRI image are merged into a dual-channel image and input into the segmentation model for automatic segmentation of the region of interest.

[0067] When fine-tuning the volume-based nn-Unet, manual ROIs are defined by manually outlining. Specifically, enhanced CT and MRI images are loaded into the segmentation software, and the ROIs are delineated by a radiologist with 10 years of experience. When outlining ovarian masses, ensure that the masses are on the same side. If there are multiple masses, only the one with the largest diameter and the highest malignancy is retained. The volume-based nn-Unet is fine-tuned using CT and MRI images as input samples and the ROIs as labels. In the embodiment, the volume-based nn-Unet is fine-tuned using an ovarian cancer dataset, a benign data subset extracted from the ovarian cancer dataset, and a non-benign data subset to obtain three different segmentation models. When segmenting CT or MRI images, the segmentation results of the three segmentation models are combined to determine the final automatic segmentation result of the ROI region. Specifically, the three segmentation results can be averaged and then binarized to determine the final segmented ROI region. The threshold for binarization can be selected as 0.5.

[0068] After automatic segmentation, if there are inconsistencies in shape and spacing, they need to be re-outlined by professional doctors, and they are also manually reviewed and refined by professional doctors to obtain multimodal regions of interest, including: first, judging whether there are holes inside the mask in the automatic segmentation results of each region of interest. Holes with an area of less than or equal to 20 pixels are defaulted to being mistakenly drawn during annotation, and the holes are filled; for holes with an area greater than 20 pixels, manually confirm whether they are intentionally hollowed out, and if not, fill them; second, calculate the number of connected domains in the original mask, remove small connected domains with a volume of less than 50 pixels, and for data that still have multiple connected domains after processing, manually confirm whether there are problems with the mask annotation, and remove them if they are mistakenly drawn; third, perform resolution sampling on the original image and the mask in the automatic segmentation result of the region of interest to fully align the two to obtain the region of interest of each modality.

[0069] In an embodiment, the feature extraction and screening module 130 is used to extract the radiomic features of the multimodal region of interest and normalize them, and then use a feature screening scheme constructed based on at least one of variance analysis, Spearman correlation, statistical test and feature selection to screen the normalized radiomic features to obtain the radiomic features selected for each modality.

[0070] For US images, radiomic features are extracted from the original standard planes (US-Color images and US-Gray images) corresponding to the ROI region machine. The radiomic features include first-order statistics, shape features, grayscale co-occurrence matrix features, etc. The specific extraction method is random selection to ensure that each patient's Doppler or B-mode ultrasound scan only retains a set of radiomic features extracted from a single US-Color image or US-Gray image. In order to solve the problem of incomplete US-Color images in the dataset, a post-processing image completion strategy is designed to ensure the integrity of the data. Including: when a patient has only US-Gray images, its color mode will be generated by copying the grayscale channel of a single US-Gray image as an RGB color space to obtain an RGB mode to complete the US-Color image. In this way, it is guaranteed that each patient should contain both color mode and grayscale mode.

[0071] For CT and MRI images, radiomic features were extracted from the images and their corresponding ROIs, respectively, and then filtered using a Gaussian filter (LoG) and a wavelet image. The extracted radiomic features include first-order features, grayscale co-occurrence matrix features, grayscale run length matrix features, grayscale size region matrix features, grayscale correlation matrix features, and adjacent grayscale difference matrix features. During the feature extraction process, the CT and MRI images were resampled to maintain the feature space dimensionality. Specifically, sitkBSpline interpolation was used to resample the CT images to [1, 1, 1] and the MRI images to [1.5, 1.5, 7].

[0072] The extracted radiomic features were also normalized. This process uses the z-score method to normalize each radiomic feature. This transforms the feature values into a standard normal distribution with zero mean and unit standard deviation, with a mean close to 0 and a standard deviation close to 1. This ensures that the features have a relatively uniform scale, helping to prevent unstable performance caused by the model being affected by feature scale.

[0073] The normalized radiomic features are then screened using a screening scheme. This screening scheme is based on at least one of analysis of variance (ANOVA), Spearman correlation, statistical tests, and feature selection (Lasso), where the statistical tests include Fisher score, Chi2 algorithm, or Relief algorithm. Based on this, the constructed feature screening scheme includes:

[0074] Option 1: ANOVA, Spearman correlation, Fisher score, and feature selection;

[0075] Option 2: ANOVA, Spearman correlation, Chi2 algorithm, and feature selection;

[0076] Option 3: ANOVA, Spearman correlation, Relief algorithm, and feature selection;

[0077] Option 4: ANOVA, Spearman correlation, and Fisher score;

[0078] Option 5: ANOVA, Spearman correlation, and Chi2 algorithm;

[0079] Solution 6: ANOVA, Spearman correlation, and Relief algorithm.

[0080] For each modality-normalized radiomic feature, variance analysis is used to remove low-variance features exceeding a preset variance threshold. Low-variance features do not carry much discriminatory information. Spearman correlation is used to identify and remove highly correlated features between groups exceeding a first correlation threshold c (e.g., c = 0.9). Highly correlated features may introduce redundancy and may cause multicollinearity in some models. The correlation between features and target labels is assessed using Fisher scores, Chi², or Relief algorithms, and low-correlation features below a second correlation threshold are removed. The target label is the classification label for each level of prediction task. Feature selection can be performed using the Lasso tool, which has two options: using features or not using features. Specifically, the Lasso is used to reduce the coefficients of less important features to zero and rank the remaining features. After these operations, the selected radiomic features are obtained for subsequent feature combination and classification prediction.

[0081] In an embodiment, the feature combination and classifier matching module 140 is used to determine the matching classifier, prediction task and corresponding optimal feature combination based on the classification effect of the feature combination formed by at least one type of radiomics features of at least one modality on each level of prediction task in the classifier.

[0082] During mid-term feature fusion, features corresponding to US-color images (USC) and US-gray images (USG) are combined into a single modality feature, features corresponding to CT images are used as a single modality feature, and features corresponding to the DWI and T2 modes contained in MRI images are divided into two modality features. During mid-term fusion, the four imaging modalities are fused together. First, features are extracted from the radiomics features of all modalities according to a predetermined feature type and quantity to form a feature combination. All features in the feature combination are then concatenated to form a feature vector. For example, a feature combination with a feature count of 30 for CT_DWI_USC involves selecting 10 features from each modality and concatenating them to create a 30-dimensional feature vector.

[0083] The data is then fed into various classifiers corresponding to the prediction tasks at each layer for inference prediction and training. These classifiers include SVC (Support Vector Classifier), XGB (XGBoost), LR (Logistic Regression), and RF (Random Forest). During training, smote (synthetic minority oversampling) is applied to the training data to address class imbalance, and GridSearchCV is used to traverse the parameters. Feature combinations are then screened and eliminated using the area under the curve (AUC), sensitivity (Sens), and specificity (Spec) metrics calculated based on the prediction results.

[0084] Specifically, the feature combinations are screened and removed using the three indicators AUC, Spec, and Sens calculated based on the prediction results, including:

[0085] First, the feature combinations whose AUC, Spec, and Sens indicators on the training set are all smaller than those on the validation set are eliminated; then, based on the set AUC threshold s1 (for example, setting s1 = 0.8), Spec threshold (for example, setting s2 = 0.8), and Sens threshold (for example, setting s2 = 0.8), the remaining feature combinations are further screened and removed. Specifically, the feature combinations that are greater than each threshold in both the validation set and the test set are retained. If there is no feature combination that meets the requirements, the Spec threshold and the Sens threshold are first reduced (for example, s2 is reduced by 0.05) and updated, and the feature combinations that are greater than the updated Spec threshold and Sens threshold (for example, s1 is reduced by 0.5) in both the validation set and the test set are retained. If there is no feature combination that meets the requirements, the AUC threshold is reduced and updated again, and the feature combinations that are greater than the updated AUC threshold in both the validation set and the test set are retained. The threshold updating and retention screening process are repeated until a feature combination that meets all thresholds is obtained.

[0086] Finally, the feature combination with the largest AUC index is selected from the remaining feature combinations after screening and removal as the optimal feature combination, and the classifier and prediction task corresponding to the optimal feature combination are used as the optimal classifier and prediction task that match the optimal feature combination. For example, for the first-level benign and non-benign classification tasks, the optimal feature combination selected is CT+MRI-DWI+MRI-T2+USC_USG, and the corresponding optimal classifier is XGB, with an AUC of 0.926. For another example, for the first-level borderline and malignant classification tasks, the optimal feature combination selected is CT+MRI-DWI, and the corresponding optimal classifier is SVC. For another example, for the second-level epithelial and non-epithelial classification, the optimal feature combination selected is MRI-DWI+MRI-T2, and the corresponding optimal classifier is XGB, with an AUC of 0.949.

[0087] In this embodiment, the accurate preoperative prediction module 150 is configured to combine at least one of the clinical data and text report with the optimal feature, and then use a matching classifier to perform an accurate prediction for the corresponding prediction task, thereby obtaining an accurate preoperative prediction result. The accurate preoperative prediction is performed by combining the clinical data and text report based on the optimal classifier and the optimal feature combination for each hierarchical classification task.

[0088] In one approach, for each hierarchical classification task, the vector corresponding to the clinical data and the best feature combination are combined and input into the corresponding best classifier for preoperative prediction.

[0089] In another approach, for each hierarchical classification task, the vector corresponding to the clinical data, the vector corresponding to the metastasis information extracted from the text report, and the best feature vector are combined and input into the corresponding best classifier for preoperative prediction.

[0090] Compared to prediction models that use a single imaging feature, the proposed system model can effectively improve the accuracy of predictions for ovarian cancer pathology, staging, and PDS surgical outcomes. Compared to similar studies in recent years, the new approach integrates three types of imaging, distinguishing itself from traditional radiomics methods. It can simultaneously predict ovarian cancer pathology, staging, and PDS outcomes, achieving high diagnostic performance. Therefore, the proposed system has certain advantages.

[0091] like Figure 5 As shown, the embodiment also provides a method for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features, comprising the following steps:

[0092] S510, obtaining and preprocessing clinical data, multimodal imaging data, and text reports related to ovarian cancer, wherein the multimodal imaging data includes US images, CT images, and MRI images;

[0093] S520, automatically segmenting the pre-processed multimodal image data into regions of interest using a segmentation model constructed based on a neural network, and obtaining multimodal regions of interest through manual review and refinement.

[0094] S530, extracting and normalizing the radiomic features of the multimodal regions of interest, and then performing feature screening on the normalized radiomic features using a feature screening scheme based on at least one of analysis of variance, Spearman correlation, statistical test, and feature selection to obtain selected radiomic features for each modality;

[0095] S540, determining a matching classifier, a prediction task, and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics feature of at least one modality on prediction tasks at each level in the classifier;

[0096] S550 , after combining at least one of the clinical data and the text report with the optimal feature, use the matching classifier to perform accurate prediction of the corresponding prediction task to obtain an accurate preoperative prediction result.

[0097] It should be noted that the device for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features provided in the above embodiment should be illustrated by the division of the above functional modules when performing accurate preoperative prediction of ovarian cancer. The above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the terminal or server is divided into different functional modules to complete all or part of the functions described above. In addition, the device for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features provided in the above embodiment and the embodiment of the method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features belong to the same concept. The specific implementation process is detailed in the embodiment of the method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features, which will not be repeated here.

[0098] Based on the same inventive concept, an embodiment further provides a computing device comprising a memory and one or more processors. The memory stores executable code. When the one or more processors execute the executable code, the device is used to implement the above-mentioned method for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features. The method specifically comprises the following steps:

[0099] S510, obtaining and preprocessing clinical data, multimodal imaging data, and text reports related to ovarian cancer, wherein the multimodal imaging data includes US images, CT images, and MRI images;

[0100] S520, automatically segmenting the pre-processed multimodal image data into regions of interest using a segmentation model constructed based on a neural network, and obtaining multimodal regions of interest through manual review and refinement.

[0101] S530, extracting and normalizing the radiomic features of the multimodal regions of interest, and then performing feature screening on the normalized radiomic features using a feature screening scheme based on at least one of analysis of variance, Spearman correlation, statistical test, and feature selection to obtain selected radiomic features for each modality;

[0102] S540, determining a matching classifier, a prediction task, and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics feature of at least one modality on prediction tasks at each level in the classifier;

[0103] S550 , after combining at least one of the clinical data and the text report with the optimal feature, use the matching classifier to perform accurate prediction of the corresponding prediction task to obtain an accurate preoperative prediction result.

[0104] The computing device provided in the embodiment, at the hardware level, includes not only a processor and memory, but also hardware required for other services such as an internal bus, a network interface, and memory. The memory is a non-volatile memory, and the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned S510-S550 described method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features. Of course, in addition to software implementation, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0105] Based on the same inventive concept, an embodiment further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for accurately predicting ovarian cancer before surgery based on multimodal imaging genomics features is implemented, specifically comprising the following steps:

[0106] S510, obtaining and preprocessing clinical data, multimodal imaging data, and text reports related to ovarian cancer, wherein the multimodal imaging data includes US images, CT images, and MRI images;

[0107] S520, automatically segmenting the pre-processed multimodal image data into regions of interest using a segmentation model constructed based on a neural network, and obtaining multimodal regions of interest through manual review and refinement.

[0108] S530, extracting and normalizing the radiomic features of the multimodal regions of interest, and then performing feature screening on the normalized radiomic features using a feature screening scheme based on at least one of analysis of variance, Spearman correlation, statistical test, and feature selection to obtain selected radiomic features for each modality;

[0109] S540, determining a matching classifier, a prediction task, and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics feature of at least one modality on prediction tasks at each level in the classifier;

[0110] S550 , after combining at least one of the clinical data and the text report with the optimal feature, use the matching classifier to perform accurate prediction of the corresponding prediction task to obtain an accurate preoperative prediction result.

[0111] In the embodiment, computer-readable media includes permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data.

[0112] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A precise preoperative prediction system for ovarian cancer based on multimodal imaging omics features, characterized by: include: A data acquisition and preprocessing module, which is used to obtain and preprocess clinical data, multimodal imaging data, and text reports related to ovarian cancer, wherein the multimodal imaging data includes US images, CT images, and MRI images; The image data delineation module is used to automatically segment the regions of interest of the pre-processed multimodal image data based on the segmentation model constructed by the neural network, and obtain the multimodal regions of interest through manual review and refinement; A feature extraction and screening module is used to extract and normalize the radiomic features of multimodal regions of interest, and then screen the normalized radiomic features using a feature screening scheme based on at least one of analysis of variance, Spearman correlation, statistical test, and feature selection to obtain radiomic features selected for each modality; A feature combination and classifier matching module is used to determine the matching classifier, prediction task, and corresponding optimal feature combination based on the classification effect of the feature combination formed by at least one type of radiomics feature of at least one modality on the prediction tasks at each level in the classifier, so as to optimally match the multimodal radiomics features, classifiers, and prediction tasks at different levels, including: Extract features from the radiomics features of all modalities according to the predetermined feature types and quantities to form feature combinations; input each feature combination into various classifiers corresponding to the prediction tasks of each layer for inference and prediction, and screen and remove the feature combinations based on the three indicators of area under the curve AUC, sensitivity Sens, and specificity Spec calculated based on the prediction results, including: first, remove the feature combinations whose AUC, Spec, and Sens indicators on the extracted training set are all smaller than those on the validation set; then further screen and remove the remaining feature combinations based on the set AUC threshold, Spec threshold, and Sens threshold, specifically retaining the feature combinations in the validation set that are all greater than each threshold; if there is no feature combination that meets the requirements, first reduce and update the Spec threshold and the Sens threshold, and retain the feature combinations in the validation set that are simultaneously greater than the updated Spec threshold and the Sens threshold; if there is no feature combination that meets the requirements, then reduce and update the AUC threshold, and retain the feature combinations in the validation set that are greater than the updated AUC threshold, and repeat the threshold update and retention screening process until a feature combination that meets the requirements and is greater than all thresholds is obtained; The feature combination with the largest AUC index is selected from the remaining feature combinations after screening and removal as the best feature combination, and the classifier and prediction task corresponding to the best feature combination are used as the best classifier and prediction task matching the best feature combination; The prediction tasks at each level include the first-level classification task of benign versus malignant, the second-level classification of malignant tumors into epithelial ovarian cancer and non-epithelial ovarian cancer, the third-level classification of epithelial ovarian cancer into early and late stages, the fourth-level classification of late-stage epithelial ovarian cancer into high-grade serous carcinoma and non-high-grade serous carcinoma according to pathological tissue type, and the fifth-level classification of late-stage high-grade serous carcinoma into R0 and non-R0. The preoperative accurate prediction module is used to combine at least one of the clinical data and text reports with the best features, and then use the matching classifier to make accurate predictions for the corresponding prediction tasks to obtain accurate preoperative prediction results.

2. The system for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features according to claim 1, characterized in that: The segmentation model built based on the neural network automatically segments the regions of interest of the pre-processed multimodal image data, including: For US images, a segmentation model constructed by fine-tuning 2D nn-Unet is used to automatically segment the regions of interest of US images. Specifically, for the US-Color image corresponding to the color standard plane contained in the US image, each channel in its RGB is input into the segmentation model as a layer output, and the union of the three layers of automatic segmentation results corresponding to the three layers of input is taken as the automatic segmentation result of the region of interest of the US-Color image; for the US-Gray image corresponding to the gray standard plane contained in the US image, it is directly input into the segmentation model to obtain the automatic segmentation result of the region of interest.

3. The system for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features according to claim 1, characterized in that: The segmentation model built based on the neural network automatically segments the regions of interest of the pre-processed multimodal image data, including: For CT images and MRI images, a segmentation model constructed by fine-tuning the volume-based nn-Unet was used to automatically segment the regions of interest (ROIs) of CT images and MRI images, respectively. Specifically, CT images were input into the segmentation model as single-channel images for automatic ROI segmentation, while MRI images contained MRI-DWI images and MRI-T2 images, which were combined into dual-channel images, were input into the segmentation model for automatic ROI segmentation. When fine-tuning the volume-based nn-Unet, the volume-based nn-Unet was fine-tuned using the ovarian cancer dataset, the benign data subset extracted from the ovarian cancer dataset, and the non-benign data subset to obtain three different segmentation models. When segmenting CT images or MRI images, the segmentation results of the three segmentation models were combined to determine the final automatic segmentation result of the region of interest.

4. The system for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features according to claim 1, characterized in that: After manual review and refinement, multimodal regions of interest were obtained, including: The automatic segmentation results of each region of interest are manually reviewed and refined, including: first, judging whether there are holes inside the mask in the automatic segmentation results of each region of interest. Holes with an area of less than or equal to 20 pixels are defaulted to being mistakenly drawn during annotation, and the holes are filled; for holes with an area greater than 20 pixels, manually confirm whether they are intentionally hollowed out. If not, fill them; second, calculate the number of connected domains in the original mask, remove small connected domains with a volume of less than 50 pixels, and manually confirm whether there are problems with the mask annotation for data that still have multiple connected domains after processing. If there are mistakes, remove them; third, perform resolution sampling on the original image and the mask in the automatic segmentation results of the region of interest to align the two completely to obtain the region of interest of each modality.

5. The system for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features according to claim 1, characterized in that: Extract radiomic features of multimodal regions of interest, including: For US images, radiomic features are extracted from the US-Color image and US-Gray image contained in the US image. The radiomic features include first-order statistics, shape features, and grayscale co-occurrence matrix features. When the US-Color image is missing, the US-Gray image is used as the three channels to generate the RGB US-Color image. For CT images and MRI images, the extracted radiomic features include first-order features, grayscale co-occurrence matrix features, grayscale run length matrix features, grayscale size area matrix features, grayscale correlation matrix features, and adjacent gray tone difference matrix features. During the feature extraction process, CT images and MRI images are resampled to maintain the feature space dimension, and the extracted radiomic features are also filtered.

6. The system for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features according to claim 1, characterized in that: The statistical test includes Fisher score, Chi2 algorithm or Relief algorithm, and the constructed feature screening scheme includes: Option 1: ANOVA, Spearman correlation, Fisher score, and feature selection; Option 2: ANOVA, Spearman correlation, Chi2 algorithm, and feature selection; Option 3: ANOVA, Spearman correlation, Relief algorithm, and feature selection; Option 4: ANOVA, Spearman correlation, and Fisher score; Option 5: ANOVA, Spearman correlation, and Chi2 algorithm; Solution 6: ANOVA, Spearman correlation, and Relief algorithm; Among them, for each modality-normalized radiomics feature, variance analysis is used to remove low-variance features exceeding a preset variance threshold; Spearman correlation is used to identify and remove highly correlated features exceeding a first correlation threshold between groups; the correlation between features and target labels is evaluated using Fisher score, Chi2 or Relief algorithm, and low-correlation features below a second correlation threshold are removed; feature selection is used to reduce the coefficients of less important features to zero, and the remaining features are ranked by importance; The target labels are the classification labels of the prediction tasks at each level.

7. The system for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features according to claim 1, characterized in that: The classifiers include support vector machine, XGBoost, Logistic regression, and random forest.

8. A method for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features, characterized by: The following steps are involved: Obtain and preprocess clinical data, multimodal imaging data, and text reports related to ovarian cancer. The multimodal imaging data includes US images, CT images, and MRI images. The segmentation model built based on the neural network automatically segments the regions of interest of the pre-processed multimodal image data and obtains the multimodal regions of interest after manual review and refinement; After extracting and normalizing the radiomic features of the multimodal regions of interest, a feature screening scheme based on at least one of analysis of variance, Spearman correlation, statistical test, and feature selection is used to screen the normalized radiomic features to obtain the radiomic features selected for each modality; Determining a matching classifier, a prediction task, and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics feature of at least one modality on prediction tasks at each level in the classifier includes: Extract features from the radiomics features of all modalities according to the predetermined feature types and quantities to form feature combinations; input each feature combination into various classifiers corresponding to the prediction tasks of each layer for inference and prediction, and screen and remove the feature combinations based on the three indicators of area under the curve AUC, sensitivity Sens, and specificity Spec calculated based on the prediction results, including: first, remove the feature combinations whose AUC, Spec, and Sens indicators on the extracted training set are all smaller than those on the validation set; then further screen and remove the remaining feature combinations based on the set AUC threshold, Spec threshold, and Sens threshold, specifically retaining the feature combinations in the validation set that are all greater than each threshold; if there is no feature combination that meets the requirements, first reduce and update the Spec threshold and the Sens threshold, and retain the feature combinations in the validation set that are simultaneously greater than the updated Spec threshold and the Sens threshold; if there is no feature combination that meets the requirements, then reduce and update the AUC threshold, and retain the feature combinations in the validation set that are greater than the updated AUC threshold, and repeat the threshold update and retention screening process until a feature combination that meets the requirements and is greater than all thresholds is obtained; The feature combination with the largest AUC index is selected from the remaining feature combinations after screening and removal as the best feature combination, and the classifier and prediction task corresponding to the best feature combination are used as the best classifier and prediction task matching the best feature combination; The prediction tasks at each level include the first-level classification task of benign versus malignant, the second-level classification of malignant tumors into epithelial ovarian cancer and non-epithelial ovarian cancer, the third-level classification of epithelial ovarian cancer into early and late stages, the fourth-level classification of late-stage epithelial ovarian cancer into high-grade serous carcinoma and non-high-grade serous carcinoma according to pathological tissue type, and the fifth-level classification of late-stage high-grade serous carcinoma into R0 and non-R0. After combining at least one of the clinical data and text reports with the optimal features, the matching classifier is used to perform accurate prediction of the corresponding prediction task to obtain accurate preoperative prediction results.

Images