Ovarian cancer preoperative accurate prediction system and method based on multi-modal radiomics characteristics

Abstract

The invention discloses an ovarian cancer preoperative accurate prediction system and method based on multi-modal imageomics characteristics, and the method comprises the steps: taking a US image, a CT image and an MRI image as the basis, carrying out the radiomics characteristic extraction of the three types of modal image data, and carrying out the combined screening of the three types of modal image data, so as to determine a matched classifier, a prediction task and a corresponding optimal characteristic combination; in this way, the multi-modal radiomics features, the classifier and prediction tasks of different levels are optimally matched, then preoperative accurate diagnosis of ovarian cancer pathology, staging and accurate prediction of PDS effects are achieved through optimal matching, and the prediction process is close to the clinical practice diagnosis and treatment thought; and iconography detection suggestions with the highest diagnostic value are provided for different diagnosis and treatment layers.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent medical technology, and specifically relates to a system and method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features. Background Art

[0002] For the treatment of ovarian cancer, surgical resection combined with systemic chemotherapy, NACT (neoadjuvant chemotherapy), PARP inhibitors, immunotherapy and other treatment methods can be used, marking that it has entered the era of precision treatment. Accurate diagnosis of ovarian cancer such as clinical staging, pathological classification, heterogeneity, etc. is the basis for precision treatment. However, this information often comes from during or after surgery. Therefore, how to make an accurate diagnosis and prediction of ovarian cancer before surgery is a major medical problem currently facing clinical practice.

[0003] Ultrasound, CT (computed tomography), MRI (magnetic resonance imaging) and other imaging examinations play an important role in the screening and diagnosis of ovarian cancer. Ultrasound is mainly used to detect the initial assessment of the presence of space-occupying lesions in the ovaries. The color Doppler technology included in ultrasound identifies the nature of tumor angiogenesis. CT is mainly used to evaluate peritoneal metastasis and clinical staging. MRI displays soft tissue images with high resolution, which further clarifies the possible malignant changes shown by ultrasound. In the clinical practice of ovarian cancer, ultrasound, CT and MRI play a role of complementary analysis and cross-validation, especially transvaginal ultrasound and its Doppler imaging technology play a key role in the diagnosis of ovarian cancer and the analysis of tumor angiogenesis. At present, imaging diagnosis still mainly relies on doctors to obtain visual image information with their naked eyes, combined with relevant clinical information, and make a comprehensive judgment based on the prior knowledge of experts. Limited by the subjective images and knowledge of experts, there is a lack of a scheme in clinical practice that can accurately predict the pathological classification, clinical staging, NACT efficacy and PDS (primary tumor cell reduction surgery) results of ovarian cancer before surgery.

[0004] The integration of existing image images mainly refers to image fusion, that is, the physical space registration between different image pictures, providing spatial anatomical information of the tumor from different angles and dimensions. On the technical level, medical image registration applies image fusion technology by geometrically aligning two images and overlapping two input source images to produce a final image with redundant and complementary information. Two points must be met in the image fusion step: 1) all appropriate medical information present in the input image must appear in the synthesized image, and 2) the fused image should not have any additional information that does not exist in any input image. This fusion is applied to multi-sensor images obtained from imaging modalities from different sources, multi-focus images usually obtained from the same modality, and multi-modal images widely used in medicine. Obviously, it is difficult for local imaging of conventional ultrasound to fuse images with other images in the above manner.

[0005] Although artificial intelligence scientific algorithms have promoted the rapid development of medical image information mining, and intelligent auxiliary diagnosis methods and prediction / diagnosis models have been reported. However, there are few methods applied to clinical practice so far. The reason is that most of the existing research is based on a single imaging technology, while in clinical practice, experts conduct multiple imaging examinations and summarize information before making clinical decisions. The clinical implementation of artificial intelligence medical imaging still faces the technical challenge of "how to objectively and accurately integrate multiple imaging examination images and establish an auxiliary diagnosis paradigm that conforms to clinical practice methods." Summary of the invention

[0006] In view of the above, the purpose of the present invention is to provide a system and method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features, which realizes the integration of preoperative ultrasound, CT, and MRI images of ovarian cancer through a new way of integrating non-spatially aligned multimodal images, and accurately predicts the preoperative multi-level classification of ovarian cancer based on the integrated radiomics features.

[0007] To achieve the above-mentioned purpose of the invention, the embodiment provides a system for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features, comprising the following steps:

[0008] A data acquisition and preprocessing module, which is used to obtain and preprocess clinical data, multimodal imaging data, and text reports related to ovarian cancer, wherein the multimodal imaging data includes US images, CT images, and MRI images;

[0009] An image data delineation module is used to automatically segment the regions of interest of the pre-processed multimodal image data based on the segmentation model constructed by the neural network and obtain the multimodal regions of interest after manual review and refinement;

[0010] A feature extraction and screening module, which is used to extract the radiomic features of the multimodal region of interest and normalize them, and then use a feature screening scheme based on at least one of variance analysis, Spearman correlation, statistical test and feature selection to screen the normalized radiomic features to obtain the radiomic features selected for each modality;

[0011] A feature combination and classifier matching module, which is used to determine a matching classifier, a prediction task and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics feature of at least one modality on prediction tasks at each level in the classifier;

[0012] The preoperative accurate prediction module is used to combine at least one of the clinical data and text reports with the best features, and then use the matching classifier to make an accurate prediction of the corresponding prediction task to obtain an accurate preoperative prediction result.

[0013] Preferably, the segmentation model constructed based on the neural network automatically segments the regions of interest of the preprocessed multimodal image data, including:

[0014] For US images, the segmentation model constructed by fine-tuning 2D nn-Unet is used to automatically segment the region of interest of US images. Specifically, for the US-Color image corresponding to the color standard plane contained in the US image, each channel in its RGB is input into the segmentation model as a layer output, and the three layers of automatic segmentation results corresponding to the three layers of input are unioned as the automatic segmentation result of the region of interest of the US-Color image; for the US-Gray image corresponding to the gray standard plane contained in the US image, it is directly input into the segmentation model to obtain the automatic segmentation result of the region of interest.

[0015] Preferably, the segmentation model constructed based on the neural network automatically segments the regions of interest of the preprocessed multimodal image data, including:

[0016] For CT images and MRI images, the segmentation model constructed by fine-tuning the volume-based nn-Unet is used to automatically segment the regions of interest of CT images and MRI images respectively. Specifically, the CT image is input into the segmentation model as a single-channel image for automatic segmentation of the region of interest, and the MRI-DWI image and MRI-T2 contained in the MRI image are merged into a dual-channel image and input into the segmentation model for automatic segmentation of the region of interest;

[0017] When fine-tuning the volume-based nn-Unet, the volume-based nn-Unet is fine-tuned using the ovarian cancer dataset, the benign data subset extracted from the ovarian cancer dataset, and the non-benign data subset to obtain three different segmentation models. When segmenting CT images or MRI images, the segmentation results of the three segmentation models are respectively integrated to determine the final automatic segmentation result of the region of interest.

[0018] Preferably, after manual review and refinement, the multimodal region of interest is obtained, including:

[0019] Manual review and refinement are performed on the automatic segmentation results of each region of interest, including: first, determining whether there are holes inside the mask in the automatic segmentation results of each region of interest. Holes with an area less than or equal to 20 pixels are assumed to be mistakenly drawn during annotation, and the holes are filled; for holes with an area greater than 20 pixels, manual confirmation is performed to determine whether they are intentionally hollowed out. If not, they are filled; second, the number of connected domains in the original mask is calculated, and small connected domains with a volume less than 50 pixels are removed. For data that still has multiple connected domains after processing, manual confirmation is performed to determine whether there are problems with the mask annotation. If there are mistakes, they are removed; third, the original image and the mask in the automatic segmentation results of the region of interest are resolution sampled to align the two completely to obtain the region of interest for each modality.

[0020] Preferably, extracting radiomic features of multimodal regions of interest includes:

[0021] For US images, radiomic features are extracted from US-Color images and US-Gray images contained in US images. The radiomic features include first-order statistics, shape features, and grayscale co-occurrence matrix features. When US-Color images are missing, US-Gray images are used as three channels to generate RGB US-Color images.

[0022] For CT images and MRI images, the extracted radiomics features include first-order features, grayscale co-occurrence matrix features, grayscale run length matrix features, grayscale size area matrix features, grayscale correlation matrix features, and adjacent gray tone difference matrix features. During the feature extraction process, CT images and MRI images are resampled respectively to maintain the feature space dimension, and the extracted radiomics features are also filtered.

[0023] Preferably, the statistical test includes Fisher score, Chi2 algorithm or Relief algorithm, and the constructed feature screening scheme includes:

[0024] Solution 1: ANOVA, Spearman correlation, Fisher score, and feature selection;

[0025] Solution 2: ANOVA, Spearman correlation, Chi2 algorithm, and feature selection;

[0026] Solution 3: ANOVA, Spearman correlation, Relief algorithm, and feature selection;

[0027] Protocol 4: ANOVA, Spearman correlation, and Fisher score;

[0028] Solution 5: ANOVA, Spearman correlation, and Chi2 algorithm;

[0029] Solution 6: ANOVA, Spearman correlation, and Relief algorithm;

[0030] Among them, for each modality normalized radiomics feature, variance analysis is used to remove low-variance features that exceed the preset variance threshold; Spearman correlation is used to identify and remove highly correlated features that exceed the first correlation threshold between groups; Fisher score, Chi2 or Relief algorithm is used to evaluate the correlation between features and target labels, and low-correlation features below the second correlation threshold are deleted; feature selection is used to reduce the coefficients of less important features to zero, and the remaining features are ranked by importance;

[0031] The target labels are the classification labels of the prediction tasks at each level.

[0032] Preferably, determining the matching best classifier, prediction task and corresponding best feature combination based on the classification effect of the feature combination formed by at least one type of radiomics feature of at least one modality on the prediction tasks at each level in the classifier includes:

[0033] Extracting features from the radiomic features of all modalities according to predetermined feature types and quantities to form a feature combination;

[0034] Each feature combination is input into various classifiers corresponding to the prediction tasks at each layer for inference prediction, and the feature combinations are screened and removed using the three indicators of area under the curve AUC, sensitivity Sens, and specificity Spec calculated based on the prediction results;

[0035] The feature combination with the largest AUC index is selected from the remaining feature combinations after screening and removal as the best feature combination, and the classifier and prediction task corresponding to the best feature combination are used as the best classifier and prediction task matching the best feature combination;

[0036] The prediction tasks at each level include the first-level prediction task of classifying according to benign or malignant, the second-level prediction task of dividing malignant tumors into epithelial ovarian cancer and non-epithelial ovarian cancer, the third-level prediction task of dividing epithelial ovarian cancer into early (FIGO I&II) and late (FIGO III&IV), the fourth-level prediction task of dividing late-stage epithelial ovarian cancer into high-grade serous carcinoma and non-high-grade serous carcinoma according to pathological tissue type, and the fifth-level prediction task of dividing late-stage high-grade serous carcinoma into R0 and non-R0.

[0037] Preferably, the feature combination is screened and removed by using the three indicators of AUC, Spec, and Sens calculated based on the prediction results, including:

[0038] First, we remove the feature combinations whose AUC, Spec, and Sens indicators on the training set are all smaller than those on the validation set;

[0039] Then, based on the set AUC threshold, Spec threshold, and Sens threshold, the remaining feature combinations are further screened and removed. Specifically, the feature combinations that are greater than each threshold in both the validation set and the test set are retained. If there is no satisfactory feature combination, the Spec threshold and the Sens threshold are first reduced and updated, and the feature combinations that are greater than the updated Spec threshold and the Sens threshold in both the validation set and the test set are retained. If there is no satisfactory feature combination, the AUC threshold is reduced and updated again, and the feature combinations that are greater than the updated AUC threshold in the validation set and the test set are retained. The threshold update and retention screening process are repeated until a feature combination that satisfies all thresholds is obtained.

[0040] Preferably, the classifier includes support vector machine, XGBoost, Logistics regression, and random forest.

[0041] To achieve the above-mentioned purpose of the invention, the embodiment further provides a method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features, comprising the following steps:

[0042] Obtaining and preprocessing clinical data, multimodal imaging data, and text reports related to ovarian cancer, wherein the multimodal imaging data includes US images, CT images, and MRI images;

[0043] The segmentation model constructed based on the neural network automatically segments the regions of interest of the pre-processed multimodal image data and obtains the multimodal regions of interest after manual review and refinement;

[0044] After extracting and normalizing the radiomic features of the multimodal regions of interest, a feature screening scheme based on at least one of analysis of variance, Spearman correlation, statistical test, and feature selection is used to screen the normalized radiomic features to obtain the selected radiomic features for each modality;

[0045] Determining a matching classifier, a prediction task, and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics features of at least one modality on prediction tasks at each level in the classifier;

[0046] After combining at least one of the clinical data and text reports with the best features, the matching classifier is used to make accurate predictions of the corresponding prediction tasks to obtain accurate preoperative prediction results.

[0047] To achieve the above-mentioned purpose of the invention, an embodiment further provides a computing device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features.

[0048] To achieve the above-mentioned purpose of the invention, the embodiment further provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the above-mentioned method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] Based on US images, CT images and MRI images, the radiomics features of these three types of modal image data are extracted and combined to determine the matching classifier, prediction task and the corresponding optimal feature combination. In this way, the multimodal radiomics features, classifiers and prediction tasks at different levels are optimally matched, and then the optimal match is used to achieve accurate preoperative diagnosis of ovarian cancer pathology, staging and accurate prediction of PDS effect. This prediction process is close to the clinical practice diagnosis and treatment ideas, and the most diagnostically valuable imaging detection recommendations are proposed for different diagnosis and treatment stratifications. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 is a schematic diagram of the structure of a system for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features provided in an embodiment;

[0053] Figure 2 is a hierarchical distribution diagram of prediction tasks at each level provided by the embodiment;

[0054] Figure 3 is a schematic diagram of a process for segmenting a region of interest provided by an embodiment;

[0055] Figure 4 It is a flowchart of the execution of the system for accurate preoperative prediction of ovarian cancer provided by the embodiment;

[0056] Figure 5 It is a flow chart of a method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features provided in an embodiment. DETAILED DESCRIPTION

[0057] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0058] like Figure 1 and Figure 4 As shown, the ovarian cancer preoperative accurate prediction system 100 based on multimodal imaging genomics features provided in the embodiment includes a data acquisition and preprocessing module 110, an imaging data delineation module 120, a feature extraction and screening module 130, a feature combination and classifier matching module 140, and a preoperative accurate prediction module 150.

[0059] In an embodiment, the data acquisition and preprocessing module 110 is used to acquire clinical data, multimodal imaging data, and text reports related to ovarian cancer and perform preprocessing. Among them, the clinical data related to ovarian cancer include age, BMI, gravidity, parity, menopausal status, and six major tumor markers, wherein the six major tumor markers are CEA, AFP, SCC, CA125, CA199, and CA153. The multimodal imaging data includes three modalities: US images, CT images, and MRI images. The text report contains descriptions of the liver / spleen / pancreas, peritoneum, lymph nodes, medium-large ascites, etc. The descriptions in the text report are extracted as metastasis information, and the metastasis information is used for subsequent preoperative prediction tasks.

[0060] In the embodiment, according to the clinical diagnosis and treatment ideas of ovarian cancer masses, the ovarian cancer prediction task is divided into multi-level prediction tasks, and the corresponding hierarchical relationships are as follows: Figure 2 As shown in the figure, the first layer is stratified according to benign and malignant, the second layer divides malignant tumors into epithelial ovarian cancer and non-epithelial ovarian cancer, the third layer divides epithelial ovarian cancer into early (FIGO I&II) and late (FIGOIII&IV), the fourth layer divides late-stage epithelial ovarian cancer into high-grade serous carcinoma and non-high-grade serous carcinoma according to pathological tissue type, and the fifth layer divides late-stage high-grade serous carcinoma into R0 and non-R0. Subsequently, the obtained clinical data, text reports and multimodal data are used to perform prediction tasks at each level to achieve accurate preoperative prediction of ovarian cancer.

[0061] The obtained clinical data and text reports were preprocessed, including: unified representation, data set segmentation, missing data completion and standardization. First, the clinical variables in each clinical data were converted into numerical values. For example, "yes" was converted to "1" and "no" was converted to "0". For missing data in tumor markers and metastasis information, the median of the features in each data set was used to complete them. The z-score was normalized using the mean and standard deviation in the training set.

[0062] The obtained multimodal data is preprocessed, including: for CT images and MRI images, the standard format is DICOM, and they are converted into Nifty format for processing. For US images, the US-Color image corresponding to the color standard plane and the US-Gray image corresponding to the gray standard plane are in DICOM format, and the corresponding label image format is NRRD. The preprocessing for US images may include: 1) Color space alignment. DICOM images can contain three different color spaces: RGB, YBR_FULL_422, and MONOCHROME2. Images in other color spaces are aligned to the RGB color space to ensure the consistency of color representation. 2) Cropping of double planes in US images (US-Color images or US-Gray images). In order to maintain the consistency of training data, a double plane and its corresponding label should be divided into two independent parts. Specifically, the cropping line is found by calculating the minimum pixel value of each column of the double plane, and then the double plane is separated according to the cropping line. 3) Labeling the US image based on the cropping line. Rename the cropped US images and labels by adding suffixes “-1” and “-2” to their original file names, where “-1” represents the left image and “-2” represents the right image.

[0063] In the embodiment, the image data delineation module 120 is used to automatically segment the region of interest of the pre-processed multimodal image data based on the segmentation model constructed by the neural network, and obtain the multimodal region of interest through manual review and refinement. The region of interest is abbreviated as ROI region, which is specifically the lesion region of interest for preoperative prediction of ovarian masses, etc. The segmentation process is as follows: Figure 3 shown.

[0064] The segmentation model constructed by fine-tuning 2D nn-Unet was used to automatically segment the region of interest of US images. The segmentation model was trained before being used. The specific training included loading the US image into the segmentation software, and having ultrasound physicians with more than 5 years of work experience delineate the ROI area in the US image, outline the boundary of the entire tumor mass on the US-Gray image, and outline the solid components of the mass on the US-Color image, including solid components, papillary protrusions, and cystic components. The US-Color image and US-Gray image containing ovarian tumors were used as input samples, and the manually outlined ROI area was used as a label to fine-tune the 2D nn-Unet to obtain the segmentation model.

[0065] When applying the trained segmentation model to perform US image segmentation, it specifically includes: for the US-Color image corresponding to the color standard plane contained in the US image, each channel in its RGB is input into the segmentation model as a layer of output, and the three layers of automatic segmentation results corresponding to the three layers of input are unioned as the automatic segmentation result of the region of interest of the US-Color image; for the US-Gray image corresponding to the gray standard plane contained in the US image, it is directly input into the segmentation model to obtain the automatic segmentation result of the region of interest.

[0066] For CT images and MRI images, the segmentation model constructed by fine-tuning the volume-based nn-Unet is used to automatically segment the regions of interest for CT images and MRI images, respectively. Specifically, the CT image is input as a single-channel image into the segmentation model for automatic segmentation of the region of interest, and the MRI-DWI image and MRI-T2 contained in the MRI image are merged into a dual-channel image and input into the segmentation model for automatic segmentation of the region of interest.

[0067] When fine-tuning the volume-based nn-Unet, the artificial ROI area is determined by manual delineation. Specifically, the enhanced CT and MRI images are loaded into the segmentation software, and the ROI area is delineated by a radiologist with 10 years of work experience. When outlining the ovarian mass, it is necessary to ensure that the mass is on the same side. If there are multiple masses, only one mass with the largest diameter and the highest degree of malignancy is retained. CT images and MRI images are used as input samples, and the volume-based nn-Unet is fine-tuned with the ROI area as a label. In the embodiment, the volume-based nn-Unet is also fine-tuned using an ovarian cancer data set, a benign data subset extracted from the ovarian cancer data set, and a non-benign data subset to obtain three different segmentation models. When segmenting the CT image or MRI image, the segmentation results of the three segmentation models are respectively integrated to determine the final ROI area automatic segmentation result. When specifically integrated, the three segmentation results can be averaged and then binarized to determine the final segmented ROI area, and the threshold value can be selected as 0.5 when binarizing.

[0068] After automatic segmentation, if there are inconsistencies in shape and spacing, they need to be re-outlined by professional doctors, and the multi-modal regions of interest are also obtained through manual review and refinement by professional doctors, including: first, judging whether there are holes inside the mask in the automatic segmentation results of each region of interest. Holes with an area less than or equal to 20 pixels are defaulted to being mistakenly drawn during annotation, and the holes are filled; for holes with an area greater than 20 pixels, manually confirm whether they are intentionally hollowed out, and if not, fill them; second, calculate the number of connected domains in the original mask, remove small connected domains with a volume less than 50 pixels, and manually confirm whether there are problems with the mask annotation for data that still have multiple connected domains after processing, and remove them if they are mistakenly drawn; third, perform resolution sampling on the original image and the mask in the automatic segmentation results of the region of interest to align the two completely, and obtain the region of interest of each modality.

[0069] In an embodiment, the feature extraction and screening module 130 is used to extract the radiomic features of the multimodal region of interest and normalize them, and then use a feature screening scheme constructed based on at least one of variance analysis, Spearman correlation, statistical test and feature selection to perform feature screening on the normalized radiomic features to obtain the radiomic features selected for each modality.

[0070] For US images, radiomic features are extracted from the original standard planes (US-Color images and US-Gray images) corresponding to the ROI area machines. The radiomic features include first-order statistics, shape features, grayscale co-occurrence matrix features, etc. The specific extraction adopts a random selection method to ensure that each patient's Doppler or B-mode ultrasound scan only retains a set of radiomic features extracted from a single US-Color image or US-Gray image. In order to solve the problem of incomplete US-Color images in the dataset, a post-processing image completion strategy is designed to ensure the integrity of the data. Including: when a patient has only US-Gray images, its color mode will be copied by copying the grayscale channel of a single US-Gray image as an RGB color space to obtain an RGB mode to complete the US-Color image. In this way, it is guaranteed that each patient should contain both color mode and grayscale mode.

[0071] For CT images and MRI images, radiomic features are extracted from the images and the corresponding ROI regions, and then filtered by Gaussian filter (LoG) and wavelet image. The extracted radiomic features include first-order features, grayscale co-occurrence matrix features, grayscale run length matrix features, grayscale size region matrix features, grayscale correlation matrix features, and adjacent gray tone difference matrix features. During the feature extraction process, CT images and MRI images are resampled to maintain the feature space dimension. When resampling, sitkBSpline interpolation can be used to resample the CT image from [1,1,1] and the interval of the MRI image can be resampled to [1.5,1.5,7].

[0072] The extracted radiomic features are also subjected to feature normalization. Feature normalization is used to normalize each radiomic feature separately using the z-score standardization method, which can convert the feature value into a standard normal distribution with zero mean and unit standard deviation, with a mean close to 0 and a standard deviation close to 1, so that the feature has a relatively uniform scale, which helps to avoid the problem of unstable performance caused by the model being affected by the feature scale.

[0073] The normalized radiomics features are further subjected to a constructed screening scheme for feature screening. The feature screening scheme is constructed based on at least one of analysis of variance (ANOVA), Spearman correlation (Spearman), statistical test and feature selection (Lasso), wherein the statistical test includes Fisher score, Chi2 algorithm or Relief algorithm. Based on this, the constructed feature screening scheme includes:

[0074] Solution 1: ANOVA, Spearman correlation, Fisher score, and feature selection;

[0075] Solution 2: ANOVA, Spearman correlation, Chi2 algorithm, and feature selection;

[0076] Solution 3: ANOVA, Spearman correlation, Relief algorithm, and feature selection;

[0077] Protocol 4: ANOVA, Spearman correlation, and Fisher score;

[0078] Solution 5: ANOVA, Spearman correlation, and Chi2 algorithm;

[0079] Solution 6: ANOVA, Spearman correlation, and Relief algorithm.

[0080] Among them, for each modality normalized radiomics feature, variance analysis is used to remove low-variance features that exceed the preset variance threshold. Low-variance features do not carry too much discriminatory information; Spearman correlation is used to identify and remove highly correlated features that exceed the first correlation threshold c (e.g., c=0.9) between groups. Highly correlated features may introduce redundancy and may cause multicollinearity problems in some models; Fisher score, Chi2 or Relief algorithm are used to evaluate the correlation between features and target labels, and low-correlation features below the second correlation threshold are deleted, where the target label is the classification label of each level prediction task; feature selection can use the Lasso tool, which has two options: use features or not use features. Specifically, Lasso is used to reduce the coefficients of less important features to zero and to rank the remaining features. After these operations, the selected radiomics features are obtained for subsequent feature combination and classification prediction.

[0081] In an embodiment, the feature combination and classifier matching module 140 is used to determine the matching classifier, prediction task and corresponding optimal feature combination based on the classification effect of the feature combination formed by at least one type of radiomics features of at least one modality on each level of prediction task in the classifier.

[0082] In the mid-term fusion of features, the features corresponding to the US-color image (USC) and the US-gray image (USG) are combined into one modality feature, the features corresponding to the CT image are used as one modality feature, and the features corresponding to the DWI and T2 modes contained in the MRI image are divided into two modality features. In the mid-term fusion, the four imaging modalities are multi-image fused. First, features are extracted from the radiomics features of all modalities according to the predetermined feature types and quantities to form a feature combination, and all features of the feature combination are connected to form a feature vector. For example, a feature combination with a feature count of 30 for CT_DWI_USC involves selecting 10 features from each modality, connecting these features, and creating a 30-dimensional feature vector.

[0083] Then, the prediction tasks at each layer are input into various classifiers for inference prediction and training, including SVC (Support Vector Classifier), XGB (XGBoost), LR (Logistic Regression), and RF (Random Forest). During training, smote (synthetic minority oversampling technology) is applied to the training data to solve the class imbalance problem, and GridSearchCV is used to traverse the parameters. Then, the feature combination is screened and removed based on the three indicators of area under the curve AUC, sensitivity Sens, and specificity Spec calculated based on the prediction results.

[0084] Specifically, the feature combinations are screened and removed based on the three indicators of AUC, Spec, and Sens calculated based on the prediction results, including:

[0085] First, the feature combinations whose AUC, Spec, and Sens indicators on the extracted training set are all smaller than those on the validation set are eliminated; then, based on the set AUC threshold s1 (for example, s1=0.8), Spec threshold (for example, s2=0.8), and Sens threshold (for example, s2=0.8), the remaining feature combinations are further screened and removed. Specifically, the feature combinations that are greater than each threshold in both the validation set and the test set are retained. If there is no satisfactory feature combination, the Spec threshold and the Sens threshold are first reduced (for example, s2 is reduced by 0.05) and updated, and the feature combinations that are greater than the updated Spec threshold and Sens threshold (for example, s1 is reduced by 0.5) in both the validation set and the test set are retained. If there is no satisfactory feature combination, the AUC threshold is reduced and updated again, and the feature combinations that are greater than the updated AUC threshold in the validation set and the test set are retained. The threshold update and retention screening process are repeated until a feature combination that satisfies all thresholds is obtained.

[0086] Finally, the feature combination with the largest AUC index is selected from the remaining feature combinations after screening and removal as the best feature combination, and the classifier and prediction task corresponding to the best feature combination are used as the best classifier and prediction task matching the best feature combination. For example, for the first-level benign and non-benign classification tasks, the best feature combination selected is CT+MRI-DWI+MRI-T2+USC_USG, the corresponding best classifier is XGB, and the AUC can reach 0.926. For another example, for the first-level borderline and malignant classification tasks, the best feature combination selected is CT+MRI-DWI, and the corresponding best classifier is SVC. For another example, for the second-level epithelial and non-epithelial classification, it is believed that the best feature combination selected is MRI-DWI+MRI-T2, the corresponding best classifier is XGB, and the AUC can reach 0.949.

[0087] In an embodiment, the preoperative accurate prediction module 150 is used to combine at least one of the clinical data and the text report with the best feature, and then use the matching classifier to make an accurate prediction of the corresponding prediction task to obtain a preoperative accurate prediction result. Preoperative accurate prediction is made based on the best classifier and the best feature combination for each level classification task matching, combined with clinical data and text reports.

[0088] In one approach, for each hierarchical classification task, the vector corresponding to the clinical data and the best feature combination are combined and input into the corresponding best classifier for preoperative prediction.

[0089] In another approach, for each hierarchical classification task, the vector corresponding to the clinical data, the vector corresponding to the metastasis information extracted from the text report, and the best feature vector are combined and input into the corresponding best classifier for preoperative prediction.

[0090] Compared with the prediction model using a single image feature, the system model of the present invention can effectively improve the prediction accuracy of ovarian cancer pathology, staging, and PDS surgical outcome. Compared with similar studies in recent years, the new method integrates three types of images, which is different from the traditional imaging genomics method, and can simultaneously predict ovarian cancer pathology, staging, and PDS outcomes, and can achieve higher diagnostic performance. Therefore, the system of the present invention has certain advantages.

[0091] like Figure 5 As shown, the embodiment also provides a method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features, comprising the following steps:

[0092] S510, obtaining clinical data, multimodal imaging data, and text reports related to ovarian cancer and performing preprocessing, wherein the multimodal imaging data includes US images, CT images, and MRI images;

[0093] S520, automatically segmenting the regions of interest of the preprocessed multimodal image data based on the segmentation model constructed by the neural network, and obtaining the multimodal regions of interest through manual review and refinement;

[0094] S530, extracting and normalizing the radiomic features of the multimodal region of interest, and then using a feature screening scheme constructed based on at least one of variance analysis, Spearman correlation, statistical test, and feature selection to perform feature screening on the normalized radiomic features to obtain radiomic features selected for each modality;

[0095] S540, determining a matching classifier, a prediction task, and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics features of at least one modality on prediction tasks at each level in the classifier;

[0096] S550, after combining at least one of the clinical data and the text report with the best feature, use the matching classifier to perform accurate prediction of the corresponding prediction task to obtain an accurate preoperative prediction result.

[0097] It should be noted that the device for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features provided in the above embodiment should be illustrated by the division of the above functional modules when performing accurate preoperative prediction of ovarian cancer. The above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the terminal or server is divided into different functional modules to complete all or part of the functions described above. In addition, the device for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features provided in the above embodiment and the embodiment of the method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features belong to the same concept. The specific implementation process is detailed in the embodiment of the method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features, which will not be repeated here.

[0098] Based on the same inventive concept, an embodiment further provides a computing device, including a memory and one or more processors, wherein executable code is stored in the memory, and when the one or more processors execute the executable code, the computing device is used to implement the above-mentioned method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features, which specifically includes the following steps:

[0099] S510, obtaining clinical data, multimodal imaging data, and text reports related to ovarian cancer and performing preprocessing, wherein the multimodal imaging data includes US images, CT images, and MRI images;

[0100] S520, automatically segmenting the regions of interest of the preprocessed multimodal image data based on the segmentation model constructed by the neural network, and obtaining the multimodal regions of interest through manual review and refinement;

[0101] S530, extracting and normalizing the radiomic features of the multimodal region of interest, and then using a feature screening scheme constructed based on at least one of variance analysis, Spearman correlation, statistical test, and feature selection to perform feature screening on the normalized radiomic features to obtain radiomic features selected for each modality;

[0102] S540, determining a matching classifier, a prediction task, and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics features of at least one modality on prediction tasks at each level in the classifier;

[0103] S550, after combining at least one of the clinical data and the text report with the best feature, use the matching classifier to perform accurate prediction of the corresponding prediction task to obtain an accurate preoperative prediction result.

[0104] The computing device provided in the embodiment, in addition to the processor and memory, also includes hardware required for other services such as internal bus, network interface, memory, etc. at the hardware level. The memory is a non-volatile memory, and the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned S510-S550 described in the method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features. Of course, in addition to the software implementation method, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0105] Based on the same inventive concept, an embodiment further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned method for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features is implemented, which specifically includes the following steps:

[0106] S510, obtaining clinical data, multimodal imaging data, and text reports related to ovarian cancer and performing preprocessing, wherein the multimodal imaging data includes US images, CT images, and MRI images;

[0107] S520, automatically segmenting the regions of interest of the preprocessed multimodal image data based on the segmentation model constructed by the neural network, and obtaining the multimodal regions of interest through manual review and refinement;

[0108] S530, extracting and normalizing the radiomic features of the multimodal region of interest, and then using a feature screening scheme constructed based on at least one of variance analysis, Spearman correlation, statistical test, and feature selection to perform feature screening on the normalized radiomic features to obtain radiomic features selected for each modality;

[0109] S540, determining a matching classifier, a prediction task, and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics features of at least one modality on prediction tasks at each level in the classifier;

[0110] S550, after combining at least one of the clinical data and the text report with the best feature, use the matching classifier to perform accurate prediction of the corresponding prediction task to obtain an accurate preoperative prediction result.

[0111] In the embodiment, computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data.

[0112] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An accurate preoperative prediction system for ovarian cancer based on multimodal imaging omics features, characterized in that: include: A data acquisition and preprocessing module, which is used to obtain and preprocess clinical data, multimodal imaging data, and text reports related to ovarian cancer, wherein the multimodal imaging data includes US images, CT images, and MRI images; An image data delineation module is used to automatically segment the regions of interest of the pre-processed multimodal image data based on the segmentation model constructed by the neural network and obtain the multimodal regions of interest after manual review and refinement; A feature extraction and screening module, which is used to extract the radiomic features of the multimodal region of interest and normalize them, and then use a feature screening scheme based on at least one of variance analysis, Spearman correlation, statistical test and feature selection to screen the normalized radiomic features to obtain the radiomic features selected for each modality; A feature combination and classifier matching module, which is used to determine a matching classifier, a prediction task and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics feature of at least one modality on prediction tasks at each level in the classifier; The preoperative accurate prediction module is used to combine at least one of the clinical data and text reports with the best features, and then use the matching classifier to make an accurate prediction of the corresponding prediction task to obtain an accurate preoperative prediction result.

2. The ovarian cancer preoperative accurate prediction system based on multimodal imaging genomics features according to claim 1, characterized in that: The segmentation model built based on the neural network automatically segments the regions of interest of the preprocessed multimodal image data, including: For US images, the segmentation model constructed by fine-tuning 2D nn-Unet is used to automatically segment the region of interest of US images. Specifically, for the US-Color image corresponding to the color standard plane contained in the US image, each channel in its RGB is input into the segmentation model as a layer output, and the three layers of automatic segmentation results corresponding to the three layers of input are unioned as the automatic segmentation result of the region of interest of the US-Color image; for the US-Gray image corresponding to the gray standard plane contained in the US image, it is directly input into the segmentation model to obtain the automatic segmentation result of the region of interest.

3. The accurate preoperative prediction system for ovarian cancer based on multimodal imaging genomics features according to claim 1, characterized in that: The segmentation model built based on the neural network automatically segments the regions of interest of the preprocessed multimodal image data, including: For CT images and MRI images, the segmentation model constructed by fine-tuning the volume-based nn-Unet is used to automatically segment the regions of interest of CT images and MRI images respectively. Specifically, the CT image is input into the segmentation model as a single-channel image for automatic segmentation of the region of interest, and the MRI-DWI image and MRI-T2 contained in the MRI image are merged into a dual-channel image and input into the segmentation model for automatic segmentation of the region of interest; When fine-tuning the volume-based nn-Unet, the volume-based nn-Unet is fine-tuned using the ovarian cancer dataset, the benign data subset extracted from the ovarian cancer dataset, and the non-benign data subset to obtain three different segmentation models. When segmenting CT images or MRI images, the segmentation results of the three segmentation models are respectively integrated to determine the final automatic segmentation result of the region of interest.

4. The system for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features according to claim 1, characterized in that: After manual review and refinement, multimodal regions of interest were obtained, including: Manual review and refinement are performed on the automatic segmentation results of each region of interest, including: first, determining whether there are holes inside the mask in the automatic segmentation results of each region of interest. Holes with an area less than or equal to 20 pixels are assumed to be mistakenly drawn during annotation, and the holes are filled; for holes with an area greater than 20 pixels, manual confirmation is performed to determine whether they are intentionally hollowed out. If not, they are filled; second, the number of connected domains in the original mask is calculated, and small connected domains with a volume less than 50 pixels are removed. For data that still has multiple connected domains after processing, manual confirmation is performed to determine whether there are problems with the mask annotation. If there are mistakes, they are removed; third, the original image and the mask in the automatic segmentation results of the region of interest are resolution sampled to align the two completely to obtain the region of interest for each modality.

5. The system for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features according to claim 1, characterized in that: Extract radiomic features of multimodal regions of interest, including: For US images, radiomic features are extracted from US-Color images and US-Gray images contained in US images. The radiomic features include first-order statistics, shape features, and grayscale co-occurrence matrix features. When US-Color images are missing, US-Gray images are used as three channels to generate RGB US-Color images. For CT images and MRI images, the extracted radiomics features include first-order features, grayscale co-occurrence matrix features, grayscale run length matrix features, grayscale size area matrix features, grayscale correlation matrix features, and adjacent gray tone difference matrix features. During the feature extraction process, CT images and MRI images are resampled respectively to maintain the feature space dimension, and the extracted radiomics features are also filtered.

6. The accurate preoperative prediction system for ovarian cancer based on multimodal imaging genomics features according to claim 1, characterized in that: The statistical test includes Fisher score, Chi2 algorithm or Relief algorithm, and the constructed feature screening scheme includes: Solution 1: ANOVA, Spearman correlation, Fisher score, and feature selection; Solution 2: ANOVA, Spearman correlation, Chi2 algorithm, and feature selection; Solution 3: ANOVA, Spearman correlation, Relief algorithm, and feature selection; Protocol 4: ANOVA, Spearman correlation, and Fisher score; Solution 5: ANOVA, Spearman correlation, and Chi2 algorithm; Solution 6: ANOVA, Spearman correlation, and Relief algorithm; Among them, for each modality normalized radiomics feature, variance analysis is used to remove low-variance features that exceed the preset variance threshold; Spearman correlation is used to identify and remove highly correlated features that exceed the first correlation threshold between groups; Fisher score, Chi2 or Relief algorithm is used to evaluate the correlation between features and target labels, and low-correlation features below the second correlation threshold are deleted; feature selection is used to reduce the coefficients of less important features to zero, and the remaining features are ranked by importance; The target labels are the classification labels of the prediction tasks at each level.

7. The system for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features according to claim 1, characterized in that: Determining the matching best classifier, prediction task and corresponding best feature combination based on the classification effect of the feature combination formed by at least one type of radiomics feature of at least one modality on the prediction tasks at each level in the classifier, including: Extracting features from the radiomic features of all modalities according to predetermined feature types and quantities to form a feature combination; Each feature combination is input into various classifiers corresponding to the prediction tasks at each layer for inference prediction, and the feature combinations are screened and removed using the three indicators of area under the curve AUC, sensitivity Sens, and specificity Spec calculated based on the prediction results; The feature combination with the largest AUC index is selected from the remaining feature combinations after screening and removal as the best feature combination, and the classifier and prediction task corresponding to the best feature combination are used as the best classifier and prediction task matching the best feature combination; The prediction tasks at each level include the first-level prediction task of classifying according to benign or malignant, the second-level prediction task of dividing malignant tumors into epithelial ovarian cancer and non-epithelial ovarian cancer, the third-level prediction task of dividing epithelial ovarian cancer into early and late stages, the fourth-level prediction task of dividing late-stage epithelial ovarian cancer into high-grade serous carcinoma and non-high-grade serous carcinoma according to pathological tissue type, and the fifth-level prediction task of dividing late-stage high-grade serous carcinoma into R0 and non-R0.

8. The system for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features according to claim 7, characterized in that: The feature combinations are screened and removed based on the three indicators of AUC, Spec, and Sens calculated based on the prediction results, including: First, we remove the feature combinations whose AUC, Spec, and Sens indicators on the training set are all smaller than those on the validation set; Then, based on the set AUC threshold, Spec threshold, and Sens threshold, the remaining feature combinations are further screened and removed. Specifically, the feature combinations that are greater than each threshold in both the validation set and the test set are retained. If there is no satisfactory feature combination, the Spec threshold and the Sens threshold are first reduced and updated, and the feature combinations that are greater than the updated Spec threshold and the Sens threshold in both the validation set and the test set are retained. If there is no satisfactory feature combination, the AUC threshold is reduced and updated again, and the feature combinations that are greater than the updated AUC threshold in the validation set and the test set are retained. The threshold update and retention screening process are repeated until a feature combination that satisfies all thresholds is obtained.

9. The system for accurate preoperative prediction of ovarian cancer based on multimodal imaging genomics features according to claim 1, characterized in that: The classifiers include support vector machine, XGBoost, Logistics regression, and random forest.

10. A method for accurate preoperative prediction of ovarian cancer based on multimodal imaging omics features, characterized in that: The following steps are involved: Obtaining and preprocessing clinical data, multimodal imaging data, and text reports related to ovarian cancer, wherein the multimodal imaging data includes US images, CT images, and MRI images; The segmentation model constructed based on the neural network automatically segments the regions of interest of the pre-processed multimodal image data and obtains the multimodal regions of interest after manual review and refinement; After extracting and normalizing the radiomic features of the multimodal regions of interest, a feature screening scheme based on at least one of analysis of variance, Spearman correlation, statistical test, and feature selection is used to screen the normalized radiomic features to obtain the selected radiomic features for each modality; Determining a matching classifier, a prediction task, and a corresponding optimal feature combination based on the classification effect of a feature combination formed by at least one type of radiomics features of at least one modality on prediction tasks at each level in the classifier; After combining at least one of the clinical data and text reports with the best features, the matching classifier is used to make accurate predictions of the corresponding prediction tasks to obtain accurate preoperative prediction results.

Images