Multi-mode-based prediction device for recurrence risk of patient suffering from cavity face type breast cancer

Abstract

The invention discloses a multi-modal-based prediction device for the recurrence risk of a patient with cavity and facial breast cancer, the device comprises one or more pre-trained multi-modal models, and the device is suitable for: in response to the received multi-modal information of a patient to be predicted, determining a target multi-modal model to be used from the one or more multi-modal models; obtaining a prediction feature set from the multi-modal information according to a training feature set adopted by the target multi-modal model during training; and inputting the prediction feature set into the target multi-modal model, and determining a recurrence risk score of the to-be-predicted patient.

Description

Technical Field

[0001] The present invention relates to the field of tumor detection, and in particular to a device for predicting the recurrence risk of patients with luminal-facial breast cancer based on multimodality. Background Art

[0002] Hormone receptor-positive (HR+) / human epidermal growth factor receptor 2-negative (HER2, human epidermal growth factor receptor 2) breast cancer (also known as luminal breast cancer) is a major subtype of breast cancer, which has a persistent risk of recurrence. Up to 20% of luminal breast cancer patients will experience recurrence and metastasis due to resistance to standard endocrine therapy; and advances in biomarker discovery, multi-omics technology, and prognostic prediction models have made it possible to predict tumor treatment resistance and recurrence risk. Currently, there are a variety of multi-gene detection tools that provide a hierarchical classification strategy for dividing patients into low-risk and high-risk groups, and help identify early luminal breast cancer patients who may benefit from chemotherapy.

[0003] However, the existing prediction technology still has some problems and shortcomings: the current multi-gene expression detection tools for predicting the prognosis of cavitary breast cancer cannot fully reflect the complex biological behavior of the tumor, and the applicable population is also relatively limited. Previous detection tools are mostly only applicable to patients with early breast cancer, such as Oncotype DX, which is applicable to newly diagnosed breast cancer patients with negative lymph node metastasis in stages I, II or IIIA, or postmenopausal patients with invasive breast cancer with positive lymph nodes. MammaPrint is suitable for patients with early breast cancer with negative lymph nodes or 1-3 positive lymph nodes. The narrow range of applicable populations limits its clinical application. For example, when predicting the prognosis of patients with lymph node-positive cavitary breast cancer, its consistency index (C-index, an indicator commonly used in survival analysis to evaluate the accuracy of the model in ranking prediction) is only 0.56 to 0.63, which has obvious defects and shortcomings.

[0004] Therefore, a new multimodality-based prediction device for the risk of recurrence in patients with luminal breast cancer is needed. Summary of the invention

[0005] According to one aspect of the present invention, a device for predicting the recurrence risk of patients with cavitary breast cancer based on multimodality is provided, wherein the device includes one or more pre-trained multimodal models, and the device is suitable for: in response to receiving multimodal information of a patient to be predicted, determining a target multimodal model to be used from one or more multimodal models; acquiring a prediction feature set from the multimodal information according to a training feature set used when training the target multimodal model; and inputting the prediction feature set into the target multimodal model to determine the recurrence risk score of the patient to be predicted.

[0006] Optionally, in the device according to the present invention, the multimodal information includes information of multiple modalities among clinical information, immunohistochemistry information, metabolome data, pathology group data, transcriptome data, genomic data and copy number variation data.

[0007] Optionally, in the device according to the present invention, the multimodal model pre-trained in the multimodal prediction device for the recurrence risk of patients with cavitary breast cancer includes a full-modal model, and the step of determining the first training feature set of the full-modal model includes: combining the features of each modality to obtain a feature pair of each modality, wherein the feature pair includes two features of the modality; calculating the correlation coefficient between the features of the feature pair; calculating the univariate COX score for the features in the first feature pair whose correlation coefficient is greater than a preset threshold; adding the feature with the highest univariate COX score in the first feature pair to the screening feature set of the modality, and adding the feature in the second feature pair whose correlation coefficient is less than the preset threshold to the screening feature set of the modality; calculating the univariate COX score for the features in each screening feature set; merging the screening feature sets of metabolome data, pathology group data, transcriptome data, genomic data and copy number variation data, determining a first preset number of features with the highest univariate COX scores therefrom and adding them to the first training feature set of the full-modal model, and adding all the features in the clinical information and immunohistochemistry information to the first training feature set.

[0008] Optionally, in the device according to the present invention, the multimodal model pre-trained in the multimodal prediction device for the recurrence risk of patients with cavitary facial breast cancer also includes a simplified model, and the step of determining the second training feature set of the simplified model includes: determining the feature importance score of each feature in the first training feature set; respectively determining a second preset number of features with the highest feature importance scores from the metabolome data, transcriptome data, genomic data and copy number variation data of the first training feature set, adding them to the second training feature set, and adding all features of the clinical information, immunohistochemistry information and pathology group data in the second training feature set to the second training feature set.

[0009] Optionally, in the device according to the present invention, determining the feature importance score of each feature in the first training feature set includes: determining a target feature for which the feature importance score is to be calculated, taking the target feature from the first training feature set to obtain a first training feature set lacking the target feature; constructing a full modal model lacking the target feature based on the first training feature set lacking the target feature; calculating a first consistency index of the full modal model and a second consistency index of the full modal model lacking the target feature; and taking the difference between the second consistency index and the first consistency index as the feature importance score of the target feature.

[0010] Optionally, in the device according to the present invention, the training step of the multimodal model includes: determining the training set features of each first test model, the training set features of the first test model including features of one or more modalities; inputting the training set features of each first test model into a COX proportional hazard model, a survival support vector machine model, a random survival forest model, a DeepSurv nonlinear model and a gradient boosting survival model for training to obtain multiple second test models; calculating the average cross-validation consistency index for each second test model, and determining one or more second test models with the largest average cross-validation consistency index as one or more multimodal models trained in the device.

[0011] Optionally, in the device according to the present invention, the training set features of each first test model are input into a COX proportional hazard model, a survival support vector machine model, a random survival forest model, a DeepSurv nonlinear model and a gradient boosting survival model for training, and multiple second test models are obtained, including: inputting the training set features of each first test model into a COX proportional hazard model, a survival support vector machine model, a random survival forest model, a DeepSurv nonlinear model and a gradient boosting survival model, and using a 5-fold cross-validation search to determine the model hyperparameters so as to maximize the average cross-validation consistency index of each second test model.

[0012] Optionally, the device according to the present invention is further adapted to: group the patients to be predicted according to the recurrence risk scores of the patients to be predicted, wherein the groups of the patients to be predicted include a low-risk group and a high-risk group.

[0013] Optionally, the device according to the present invention is further suitable for: determining the recurrence risk score of each training sample according to the training set for training the multimodal model; determining the recurrence risk score threshold according to the recurrence risk score of each training sample; if the recurrence risk score of the patient to be predicted is greater than the recurrence risk score threshold, the patient group is a high-risk group; if the recurrence risk score of the patient to be predicted is less than the recurrence risk score threshold, the patient group is a low-risk group.

[0014] According to another aspect of the present invention, a computing device is provided, comprising: one or more processors; a memory; and a device for predicting the recurrence risk of patients with cavitary-facial breast cancer based on multiple modalities.

[0015] According to another aspect of the present invention, there is provided a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions which, when executed by a computing device, cause the computing device to execute the method executed in the apparatus for predicting the risk of recurrence of patients with cavitary breast cancer based on multimodality according to the present invention.

[0016] The multimodal recurrence risk prediction device for cavitary facial breast cancer patients of the present invention integrates multimodal information and combines machine learning methods to determine the recurrence risk score for patients, thereby predicting the recurrence risk of cavitary facial breast cancer patients, thereby predicting the patient's prognosis, improving the accuracy and efficiency of the prediction, and using multimodal features to enable the prediction device to be applicable to a wider range of patients and assist in clinical treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] To achieve the above and related purposes, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles disclosed herein can be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The above and other purposes, features and advantages of the present disclosure will become more apparent by reading the following detailed description in conjunction with the accompanying drawings. Throughout the present disclosure, the same reference numerals generally refer to the same parts or elements.

[0018] Figure 1 A schematic diagram of acquiring multimodal information of a patient according to an exemplary embodiment of the present invention is shown;

[0019] Figure 2 A schematic diagram of a multimodal model for training and predicting the risk of recurrence in patients with cavitary-facial breast cancer according to an exemplary embodiment of the present invention is shown;

[0020] Figure 3 shows consistency index and cross-validation consistency index of multiple models according to an exemplary embodiment of the present invention;

[0021] Figure 4 A schematic diagram showing the survival rates of high-risk group and low-risk group predicted by CIMPTGV model according to an exemplary embodiment of the present invention;

[0022] Figure 5 A schematic diagram showing the difference in multi-omics features of high-risk and low-risk groups predicted by the CIMPTGV model according to an exemplary embodiment of the present invention;

[0023] Figure 6 A schematic diagram showing different modal correlations in a CIMPTGV model according to an exemplary embodiment of the present invention is shown;

[0024] Figure 7a A schematic diagram of training a simplified multimodal model according to an exemplary embodiment of the present invention is shown;

[0025] Figure 7b A schematic diagram showing a simplified feature in a multimodal model according to an exemplary embodiment of the present invention is shown;

[0026] Figure 7cA schematic diagram showing the prediction of patient survival status of a simplified model according to an exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to enable the scope of the present disclosure to be fully conveyed to those skilled in the art. The same reference numerals generally refer to the same components or elements.

[0028] At present, the multi-gene expression detection tool for predicting the prognosis of cavitary facial breast cancer mainly predicts the patient's prognosis by detecting gene expression. Based on only a few genes, it cannot fully reflect the complex biological behavior of the tumor. Therefore, the present invention integrates information from multiple modalities and combines machine learning methods to stratify high-risk and low-risk recurrence risk patients to achieve the prediction of the recurrence risk of cavitary facial breast cancer patients, thereby predicting the patient's prognosis, improving the accuracy and efficiency of the prediction, and assisting clinical treatment. Multiple modalities include: clinical information, immunohistochemistry information, metabolome data, pathology group data, transcriptome data, genomic data and copy number variation data.

[0029] Figure 1 FIG. 2 shows a schematic diagram of obtaining multimodal information of a patient according to an exemplary embodiment of the present invention. Figure 1 As shown, the clinical information of the patients was collected, including age, body mass index (BMI, body Mass Index), tumor thrombus status, menstrual status, tumor size stage and lymph node stage.

[0030] The immunohistochemical information of tumor samples was obtained by the pathology department, including the status of estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2), which could be confirmed by using immunohistochemical analysis and in situ hybridization.

[0031] Immunohistochemical information can be expressed using the following parameters: ER percentage, PR percentage and ki-67 proliferation index, etc.

[0032] Metabolomic data may include polar metabolites and lipid metabolite abundances detected by mass spectrometry.

[0033] Genomic data collected included gene mutation status, tumor mutation burden (TMB) score, and homologous recombination deficiency (HRD) score. The TMB score was obtained by calculating the total number of nonsynonymous mutations per million base pairs (Mb) of genomic region, and the HRD score was calculated by the unweighted sum of the loss of heterozygosity (LOH) score, telomeric allelic imbalance (TAI) score, and corrected large-scale state transition (LSTm) score.

[0034] Copy number variation data collected chromosome segment amplification and deletion scores.

[0035] The transcriptome data collected pathway scores contained in the KEGG database, GOBP database, and HALLMARK database calculated based on the ssGSEA algorithm.

[0036] The pathology group data includes pathological features. Pathology extraction includes: First, based on the deep learning Hover-net algorithm, the morphological and texture features of the nucleus of each individual cell in the whole slide image (WSI) are extracted, and the tissue component features of each WSI are calculated. Then, the topological features are extracted based on the image algorithm to characterize the intercellular relationship at single-cell resolution. Finally, the pathological features at the patient level are obtained by calculating the average values ​​of the nuclear morphological features, nuclear texture features, and topological features of each cell type within each patient.

[0037] After acquiring the multimodal information, a multimodal model for predicting the recurrence risk of cavitary facial breast cancer patients is trained based on the multimodal information. According to one embodiment of the present invention, a multimodal prediction device for predicting the recurrence risk of cavitary facial breast cancer patients includes one or more multimodal models.

[0038] Figure 2 FIG. 2 is a schematic diagram showing a multimodal model for training and predicting the risk of recurrence in patients with cavitary-facial breast cancer according to an exemplary embodiment of the present invention. Figure 2 As shown, the constructed machine learning framework is used to integrate multimodal information to build a model, including the following steps:

[0039] Determine the training set features of each first test model, wherein the training set features of the test model include features of one or more modalities. First, perform cohort division to establish training sets and test sets; specifically, the training set features of each first test model are obtained by combining different modal information, such as clinical information, immunohistochemical information, and transcriptome data as a separate training set feature of the first test model, perform stratified sampling in the cohort corresponding to each first test model (patients with modal combination information corresponding to the model), use the patient's recurrence status as a stratification factor, and divide the training set and test set at a ratio of 4:1. Recurrence status includes recurrence and no recurrence. The recurrence status of patients with recurrence events is recurrence, and the recurrence status of patients without recurrence events is no recurrence. Recurrence events are determined by outpatient review and imaging examinations (such as CT, B-ultrasound, molybdenum target, MRI, PET / CT, etc.), and recurrence-free survival (RFS) is defined as the time interval from the patient's diagnosis to the first recurrence of the primary breast cancer in the same side or adjacent area, or the new tumor of the contralateral breast cancer, or the death of the patient.

[0040] Subsequently, the training set features of each first test model are input into the COX proportional hazard model, the survival support vector machine model, the random survival forest model, the DeepSurv nonlinear model and the gradient boosting survival model for training to obtain multiple second test models; including: inputting the training set features of each first test model into the COX proportional hazard model, the survival support vector machine model, the random survival forest model, the DeepSurv nonlinear model and the gradient boosting survival model, and using 5-fold cross-validation search to determine the model hyperparameters to maximize the average cross-validation consistency index of each second test model.

[0041] An average cross-validation consistency index is calculated for each second test model, and one or more second test models with the largest average cross-validation consistency index are determined as one or more multimodal models trained in the device.

[0042] According to one embodiment of the present invention, the present invention also performs feature extraction: feature extraction is performed in each modality of each first test model, and a full-modality model is trained. The step of determining the first training feature set of the full-modality model includes: combining the features of each modality to obtain a feature pair of each modality, wherein the feature pair includes two features of the modality; calculating the correlation coefficient between the features of the feature pair; calculating the univariate COX score for the features in the first feature pair whose correlation coefficient is greater than a preset threshold; adding the feature with the highest univariate COX score in the first feature pair to the screening feature set of the modality, and adding the features in the second feature pair whose correlation coefficient is less than the preset threshold to the screening feature set of the modality, so as to remove collinearity and retain only one of the two strongly positively correlated features that is of great significance to the prediction result.

[0043] According to one embodiment of the present invention, the preset threshold of the correlation coefficient greater than can be set to 0.9; then the univariate COX score is calculated for each feature in the screening feature set; the screening feature sets of metabolome data, pathology group data, transcriptome data, genomic data and copy number variation data are merged, and the first preset number of features with the highest univariate COX scores are determined and added to the first training feature set of the full modality model, and all features of the clinical information and immunohistochemical information are added to the first training feature set. According to one embodiment of the present invention, the first preset threshold can be set to 15.

[0044] According to one embodiment of the present invention, feature dimension reduction is performed when selecting features to train a full-modality model: feature dimension reduction is performed separately in each modality, and in each modality, the spearman correlation coefficient between features is calculated, and feature pairs with correlations higher than 0.9 are selected. The COXPHSurvivalAnalysis in python scikit-survival is used to retain features with higher scores and delete features with lower scores. The above steps are performed separately in each modality and a screening feature set for each modality is obtained. Then, the selectkbest method in scikit-learn is used to further screen the screening feature set in each modality. In order to balance the features between different dimensions and reduce model complexity and training time, all features in the two modalities of clinical information and immunohistochemistry information (IHC, Immunohistochemistry) are included, and the top 15 features with the highest univariate COX risk score are selected from other modalities.

[0045] According to one embodiment of the present invention, the present invention can also train a simplified model, and the step of determining the second training feature set of the simplified model includes: determining the feature importance score of each feature in the first training feature set; respectively determining a second preset number of features with the highest feature importance scores from the metabolome data, transcriptome data, genomic data, and copy number variation data of the first training feature set, adding them to the second training feature set, and adding all features of the clinical information, immunohistochemistry information, and pathology group data in the second training feature set to the second training feature set. According to one embodiment of the present invention, the second preset threshold can be set to 2 or 3.

[0046] Determining the feature importance score of each feature in the first training feature set includes: determining a target feature for which the feature importance score is to be calculated, taking the target feature from the first training feature set to obtain a first training feature set lacking the target feature; constructing a full modal model lacking the target feature based on the first training feature set lacking the target feature; calculating a first consistency index of the full modal model and a second consistency index of the full modal model lacking the target feature; and using the difference between the second consistency index and the first consistency index as the feature importance score of the target feature.

[0047] According to one embodiment of the present invention, the method of associating the full modal model according to the first training feature cluster and the method of associating the simplified model according to the second training feature cluster includes: inputting the training set features of the first training feature cluster or the second training feature set into the COX proportional hazard model, the survival support vector machine model, the random survival forest model, the DeepSurv nonlinear model and the gradient boosting survival model for training to obtain test models of multiple full modal models or test models of multiple simplified models; calculating the average cross-validation consistency index for the test model of each full modal model, and determining the test model of the full modal model with the largest average cross-validation consistency index as the full modal model trained in the device; or calculating the average cross-validation consistency index for the test model of each simplified model, and determining the test model of the simplified model with the largest average cross-validation consistency index as the simplified model trained in the device.

[0048] According to one embodiment of the present invention, the present invention also performs model training and verification: the training feature set of each first training model obtained after the above screening is input into the COX proportional hazard model, the survival support vector machine model, the random survival forest model, the DeepSurv nonlinear model and the gradient boosting survival model, and a random 1000 5-fold cross-validation search is performed to optimize the hyperparameters so as to maximize the average cross-validation consistency index, and obtain multiple second test models. Among the multiple second test models, one or more models with the highest average cross-validation consistency index are selected as the final multi-objective modal model, and the final multi-objective modal model is tested in an independent test set of the model to evaluate its performance.

[0049] Subsequently, the final confirmation of the model was carried out: through the above training, multiple multimodal models and single-modal models were obtained, among which the full-modal CIMPTGV model (a model constructed by integrating clinical information, immunohistochemical information, metabolomics data, pathology group data, transcriptome data, genomic data, and copy number variation data) had the best consistency index in both the training set and the independent test set.

[0050] Figure 3 Graph 1 shows the consistency index and cross-validation consistency index of multiple models according to an exemplary embodiment of the present invention. Figure 3 As shown, the consistency index of the full-modality CIMPTGV model is significantly better than other multimodal models and single-modal models. Table 1 shows the included features of the CIMPTGV model, and Table 2 shows the specific parameters of the model generated according to one or more modalities.

[0051] Table 1

[0052]

[0053]

[0054]

[0055] Table 2

[0056]

[0057]

[0058]

[0059]

[0060]

[0061]

[0062] By applying multimodal models, such as the CIMPTGV model, recurrence risk prediction scores were generated in both training and test sets.

[0063] According to one embodiment of the present invention, the present invention also groups the patients to be predicted according to the recurrence risk scores of the patients to be predicted, and the groups of the patients to be predicted include a low-risk group and a high-risk group; specifically: determine the recurrence risk score of each training sample according to the training set for training the multimodal model; determine the recurrence risk score threshold according to the recurrence risk score of each training sample; if the recurrence risk score of the patient to be predicted is greater than the recurrence risk score threshold, the patient's group is a high-risk group; if the recurrence risk score of the patient to be predicted is less than the recurrence risk score threshold, the patient's group is a low-risk group. The present invention can set the upper quartile of the predicted risk score in the training set as a threshold to distinguish patients into a high-risk group and a low-risk group. Figure 4 FIG. 4 is a schematic diagram showing the survival rates of high-risk group and low-risk group predicted by CIMPTGV model according to an exemplary embodiment of the present invention. Figure 4 As shown in the figure, the CIMPTGV model can effectively distinguish patients with different conditions in terms of recurrence-free survival, overall survival, and distant metastasis-free survival in both the training set and the test set. Multivariate COX regression analysis further explained the independent prediction effect of the CIMPTGV model. In addition, subgroup analysis also showed that the CIMPTGV model can significantly distinguish high-risk recurrence patients with cavitary breast cancer at different stages and with different clinical characteristics, indicating the powerful effectiveness of the CIMPTGV model in predicting the risk of recurrence in patients with cavitary breast cancer.

[0064] Figure 5 FIG. 4 is a schematic diagram showing the difference in multi-omics features of high-risk and low-risk groups predicted by the CIMPTGV model according to an exemplary embodiment of the present invention. Figure 5 As shown in the figure, the high-risk group predicted by the CIMPTGV model in the entire cohort (including training and test sets) covers almost all recurrent patients, and patients in the high-risk group have higher tumor stages and ki-67 proliferation indexes. In biological characteristics, the high-risk group downregulates lipid metabolism, highly expresses the MYC pathway, and has a larger number of tumor cells. The above shows that there are significant changes in multiple dimensional characteristics in the high-risk group for recurrence of luminal breast cancer.

[0065] Figure 6 FIG. 4 is a schematic diagram showing different modal associations in a CIMPTGV model according to an exemplary embodiment of the present invention. Figure 6As shown in the figure, by analyzing the eigenvalue and the CIMPTGV model risk score, it was found that the HRD score was significantly positively correlated, and the HRD score of the high-risk group was significantly increased. HRD can lead to genomic instability and ultimately promote tumor progression. HRD is significantly positively correlated with the amplification of the 11q13.1 chromosome segment. Some genes related to the malignant biological behavior of cancer, such as fibroblast growth factor 4 (FGF4), fibroblast growth factor 3 (FGF3) and cortactin (CTTN), are located in this amplified segment and are significantly amplified in the HRD high score group. At the pathological level, patients with high HRD scores have a higher degree of tumor cell contact and a larger number of tumor cells. In addition, the HRD high score group has a higher pathological morphological heterogeneity, which is closely related to its high risk of recurrence. At the metabolomics level, patients with high HRD scores show a high enrichment of nucleic acid metabolites, indicating that genomic instability in high-risk populations is closely related to abnormal tumor metabolic function.

[0066] In order to further enhance the clinical applicability of the model, the present invention also sets up a simplified multimodal model based on the performance of the CIMPTGV model. Figure 7a FIG. 2 shows a schematic diagram of training a simplified multimodal model according to an exemplary embodiment of the present invention. Figure 7a As shown, features are first selected from each modality, wherein all features in clinical information and immunohistochemical information are selected, and for other modalities, features with high feature importance scores are selected.

[0067] Figure 7b A schematic diagram of simplifying features in a multimodal model according to an exemplary embodiment of the present invention is shown. The simplified model S-CIMPTGV is obtained by training based on the selected features. The present invention further verifies the model effectiveness of the simplified model. Figure 7c FIG. 4 is a schematic diagram showing a patient survival prediction of a simplified model according to an exemplary embodiment of the present invention. Figure 7c As shown in the figure, the prediction effect of the simplified model (S-CIMPTGV, mean area under the curve (AUC, Area Under the Curve) = 0.840) was slightly lower than that of the CIMPTGV model (mean AUC = 0.886), but the S-CIMPTGV model was superior to other dimensions commonly used in clinical practice. The Kaplan-Meier survival curve further illustrated the significant stratification of the predicted high-risk and low-risk groups, showing the practicality of the simplified model in clinical decision-making.

[0068] According to one embodiment of the present invention, in response to receiving multimodal information of a patient to be predicted, when determining the target multimodal model to be used from one or more multimodal models, the most accurate model, such as a full modal model, can be selected from one or more multimodal models, and a simplified model or other model can also be selected based on the principle of highest prediction efficiency. The target multimodal model can also be determined by formulating according to setting information from one or more multimodal models. Subsequently, a prediction feature set is obtained from the multimodal information based on the training feature set used in training the target multimodal model; the prediction feature set is input into the target multimodal model to determine the recurrence risk score of the patient to be predicted.

[0069] According to one embodiment of the present invention, the present invention collects multi-omics information of 579 patients with cavitary facial breast cancer who visited a hospital, wherein 200 patients have clinical information, immunohistochemistry information, metabolomics information, pathological information, transcriptomics information, genomic information and copy number variation information, and collects a multi-omics feature set of these 200 patients, which includes a total of 85 features including 7 dimensions, and randomly splits these 200 patients into a training set and a validation set in a ratio of 4:1 with recurrence status as a stratification factor.

[0070] After the training set features are constructed, they are used as the input matrix, and the random survival forest algorithm in the scikit-survival (v.0.22.1) library of Python is used as the model algorithm input, where the hyperparameters are selected as 'n_estimators': 10, 'min_samples_split': 3, 'min_samples_leaf': 2, 'max_features': 0.5, 'max_depth': 5. After the model training is completed, the risk score of each patient calculated by the model in the training set is exported.

[0071] Traditional methods rely on single gene testing, which cannot reflect the complex biological information of tumors and is applicable to a narrow population. The multimodal model constructed by the present invention comprehensively integrates the characteristics of different modalities and is applicable to patients with cavitary facial breast cancer at all stages. Traditional methods are not effective in predicting breast cancer patients with lymph node metastasis. The multimodal model constructed by the present invention can significantly distinguish the risk of recurrence of cavitary facial breast cancer at all stages, with a consistency index of up to 0.869 in the independent test set.

[0072] It should be noted that the storage medium (computer-readable medium) mentioned above in the present application may be a computer-readable signal medium or a non-transitory computer-readable storage medium or any combination of the above two. The non-transitory computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of non-transitory computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0073] In the present application, a non-transitory computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a non-transitory computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0074] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0075] The above description is only a partial embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

[0076] In addition, although each operation is described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the application. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0077] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A multimodal prediction device for the risk of recurrence of luminal breast cancer patients, the device comprising one or more pre-trained multimodal models, the device being suitable for: In response to receiving multimodal information of a patient to be predicted, determining a target multimodal model to be used from one or more multimodal models; Acquiring a prediction feature set from the multimodal information according to a training feature set used when training the target multimodal model; The prediction feature set is input into the target multimodal model to determine the recurrence risk score of the patient to be predicted.

2. The device according to claim 1, wherein: The multimodal information includes information of multiple modalities among clinical information, immunohistochemical information, metabolome data, pathology group data, transcriptome data, genomic data and copy number variation data.

3. The device as claimed in claim 2, wherein: The multimodal model pre-trained in the multimodal-based prediction device for the risk of recurrence of cavitary breast cancer patients includes a full-modal model, and the step of determining the first training feature set of the full-modal model includes: The features of each mode are combined to obtain a feature pair of each mode, wherein the feature pair includes two features of the mode; Calculating the correlation coefficient between the features of the feature pair; Calculate the univariate COX score for the features in the first feature pair whose correlation coefficient is greater than the preset threshold; Adding the feature with the highest single factor COX score in the first feature pair to the screening feature set of the modality, and adding the features in the second feature pair with a correlation coefficient less than a preset threshold to the screening feature set of the modality; Univariate COX scores were calculated for each feature in the screening feature set; The screening feature sets of metabolome data, pathology group data, transcriptome data, genomic data and copy number variation data are merged, and a first preset number of features with the highest univariate COX scores are determined and added to the first training feature set of the full-modality model, and all features in the clinical information and immunohistochemistry information are added to the first training feature set.

4. The device according to claim 3, wherein: The multimodal model pre-trained in the multimodal-based prediction device for the risk of recurrence of cavitary-facial breast cancer patients also includes a simplified model, and the step of determining the second training feature set of the simplified model includes: determining a feature importance score for each feature in the first training feature set; A second preset number of features with the highest feature importance scores are determined from the metabolome data, transcriptome data, genomic data and copy number variation data of the first training feature set, and added to the second training feature set, and all features of the clinical information, immunohistochemistry information and pathology group data in the second training feature set are added to the second training feature set.

5. The device according to claim 4, wherein: Determining the feature importance score of each feature in the first training feature set includes: Determine a target feature for which a feature importance score is to be calculated, and take the target feature out of the first training feature set to obtain a first training feature set lacking the target feature; Constructing a full-modal model lacking the target feature according to the first training feature set lacking the target feature; Calculating a first consistency index of the full modal model and a second consistency index of the full modal model lacking the target feature; The difference between the second consistency index and the first consistency index is used as the feature importance score of the target special diagnosis.

6. The device according to claim 1, wherein: The training steps of the multimodal model include: Determine a training set feature of each first test model, wherein the training set feature of the first test model includes features of one or more modalities; Inputting the training set features of each first test model into a COX proportional hazard model, a survival support vector machine model, a random survival forest model, a DeepSurv nonlinear model, and a gradient boosting survival model for training to obtain multiple second test models; An average cross-validation consistency index is calculated for each second test model, and one or more second test models with the largest average cross-validation consistency index are determined as one or more multimodal models trained in the device.

7. The device according to claim 6, wherein: The training set features of each first test model are input into the COX proportional hazard model, the survival support vector machine model, the random survival forest model, the DeepSurv nonlinear model and the gradient boosting survival model for training, and the plurality of second test models are obtained, including: The training set features of each first-test model were input into the COX proportional hazards model, survival support vector machine model, random survival forest model, DeepSurv nonlinear model, and gradient boosting survival model, and a 5-fold cross-validation search was used to determine the model hyperparameters to maximize the average cross-validation consistency index of each second-test model.

8. The device according to claim 1, further adapted to: The patients to be predicted are grouped according to the recurrence risk scores of the patients to be predicted, and the groups of the patients to be predicted include a low-risk group and a high-risk group.

9. The device according to claim 8, further adapted to: determining a recurrence risk score for each training sample based on a training set for training the multimodal model; Determining a recurrence risk score threshold according to the recurrence risk score of each training sample; If the recurrence risk score of the patient to be predicted is greater than the recurrence risk score threshold, the patient group is a high-risk group; If the recurrence risk score of the patient to be predicted is less than the recurrence risk score threshold, the patient group is a low-risk group.

10. A computing device comprising: one or more processors; Memory; as well as A multi-cancer auxiliary determination device as described in any one of claims 1-9.

11. A computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to execute the method executed in a multi-cancer auxiliary determination apparatus according to claims 1 to 9.

Images