Patient pooling based on machine learning model
By identifying patient pools with similar attributes and using machine learning models to conduct clinical predictions, the inaccuracy of machine learning models in the existing technology in prognosis and treatment recommendations is solved, and the correlation between survival prediction and treatment decisions of new patients is improved. Validity.
Patent Information
- Application Number
- CN202380032232.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-01
- Filing Date
- 2023-03-31
- Publication Date
- 2024-11-15
AI Technical Summary
Existing machine learning models when predicting patients' medical outcomes may be inaccurate prognosis and treatment recommendations due to the lack of data training and insufficient data weighting for specific patient types, resulting in inaccurate prognosis and treatment recommendations, which cannot effectively improve the prognosis of new patients.
By identifying patient groups with similar attributes (patient pools), using machine learning models such as the random survival forest model, clinical predictions are made based on the attributes of new patients, and output attributes and medical outcomes of patient pools similar to those of new patients, in support of clinical support, using machine learning models such as the random survival forest model, and perform clinical predictions based on the attributes of new patients, and output attributes and medical outcomes of patient pools similar to those of new patients, to support clinical decision making.
Improved survival prediction relevance in new patients, provided potential treatment insights to improve prognosis, and helped clinicians develop action plans that improve survival probability of new patients.
Smart Images

Figure CN118974839A_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to U.S. patent application serial number 63 / 362,373 filed on April 1, 2022, the disclosure of which is incorporated herein by reference in its entirety. Background Art
[0003] Predictive machine learning models trained using real-world clinical data offer great potential for providing patients and their clinicians with patient-specific information about diagnosis, prognosis, or optimal treatment options. Machine learning models can be trained to make clinical predictions to predict a patient's medical outcomes such as, for example, a patient's probability of survival as a function of time since diagnosis (e.g., advanced cancer), survival time of new patients since diagnosis, other types of prognosis, etc. Predictions can be provided to patients to, for example, improve their ability to plan for their future, which can improve their quality of life.
[0004] Many machine learning models are developed using extensive data from multiple types of patients with different conditions, treatment modalities applicable to those conditions, and prognoses. These models may not be adequately trained with data applicable to a specific patient type and / or may be weighted based on data that may be less valuable for a specific patient type, resulting in inaccurate prognoses and / or recommended treatments. Therefore, improved methods for utilizing machine learning models to enhance patient care are needed. Summary of the invention
[0005] Disclosed herein are techniques for facilitating clinical decision making for a patient based on identifying a group of patients having similar attributes to the patient. The patient group can be identified using information from a predictive machine learning model that makes clinical predictions for the patient. At least some of the attributes of the patient group can be output to support clinical decision making. The attributes can include, for example, archival data of the patient, results of one or more laboratory tests of the patient, biopsy image data of the patient, molecular biomarkers of the patient, tumor site of the patient, and tumor stage of the patient.
[0006] Specifically, the clinical decision support system can use a machine learning model to make clinical predictions for a patient based on the patient's attributes. For example, the machine learning model may include a random survival forest (RSF) model that predicts the patient's probability of survival as a function of the time elapsed since diagnosis. The clinical decision support system can also identify a group of patients with specific attributes similar to the patient's specific attributes (e.g., a "patient pool based on similarity"). The patient pool based on similarity may include patients with health conditions comparable to the patient, and can be identified based on a subset of attributes that are most relevant to the clinical prediction made by the machine learning model (e.g., the probability of survival at a specific time point since diagnosis). The clinical decision support system can then obtain information about the attributes of the patient pool based on similarity.
[0007] The clinical decision support system can output a predicted probability of survival for a new patient and the attributes of the new patient that are determined to be most relevant to the prediction. Through similarity-based patient pooling, the clinical decision support system can output a survival function for a similarity-based patient pool and a summary of the attributes of the patients in the similarity-based patient pool. This facilitates comparison between the attributes of the new patient and the attributes of the similarity-based patient pool, focusing on the attributes that are most relevant to the prediction of survival for the new patient. Studying the relationship between attributes and survival in the similarity-based patient pool can help clinicians determine a course of action (e.g., treatment) to improve the probability of survival for the new patient.
[0008] In some embodiments, a computer-implemented method for facilitating clinical decision making includes: receiving first data corresponding to multiple features of a first patient, each feature representing an attribute among multiple attributes; inputting the first data into a machine learning model to generate a result of a clinical prediction for the first patient, the machine learning model being associated with multiple feature importance metrics, the multiple feature importance metrics defining the relevance of each of the multiple features to the clinical prediction; obtaining second data corresponding to multiple features of each patient in the patient group based on a degree of similarity between the first patient and the patient group with respect to at least some of the multiple features, the degree of similarity being based on the first data, the second data, and the multiple feature data importance metrics; generating content based on the result of the clinical prediction and at least a portion of the second data; and outputting the content so as to make a clinical decision for the first patient based on the content.
[0009] In some embodiments, the plurality of attributes includes at least one of: archival data of the patient, results of one or more laboratory tests of the first patient, biopsy image data of the first patient, molecular biomarkers of the first patient, tumor site of the first patient, or tumor stage of the first patient.
[0010] In some embodiments, the clinical prediction includes at least one of: the probability of survival of the first patient at a predetermined time from when the first patient was diagnosed with the tumor, the survival time of the first patient from when the first patient was diagnosed with the tumor, or the outcome of receiving treatment.
[0011] In some embodiments, the machine learning model includes a random forest survival model, the random forest survival model includes f decision trees, each of the decision trees is configured to process a subset of the first subset of the data to generate a cumulative survival probability; and wherein the survival rate of the patient at a predetermined time is determined based on an average of the cumulative survival probabilities output by multiple decision trees.
[0012] In some embodiments, the patient group is a first patient group; wherein the first patient group is selected from a second patient group; and wherein the machine learning model is trained based on patient data of the second patient group.
[0013] In some embodiments, the method further includes: ranking the multiple features based on the relevance of each of the multiple features to the clinical prediction; determining a subset of the multiple features based on the ranking; and determining the first patient group based on the degree of similarity between the first patient and the first patient group with respect to the subset of the multiple features.
[0014] In some embodiments, the first patient group is selected from the second patient group based on a degree of similarity between the first patient and the first patient group with respect to a subset of the plurality of features exceeding a threshold.
[0015] In some embodiments, the first group of patients is selected from the second group of patients based on selecting a threshold number of patients having a highest degree of similarity to the first patient with respect to a subset of the plurality of features.
[0016] In some embodiments, the method further includes: calculating a weighted aggregate degree of similarity based on summing the scaled degrees of similarity for each of at least some of the multiple features, each degree of similarity being scaled by a weight based on a relevance of the features; and identifying the patient group based on the weighted aggregate degree of similarity between the first patient and each of the patients in the patient group.
[0017] In some embodiments, a feature importance measure for a feature is determined based on a relationship between an error of a clinically predicted outcome generated by a machine learning model for a second patient in a first patient group; wherein the clinically predicted outcome is generated from multiple values of the feature for the second patient; and wherein the error is calculated based on comparing the clinically predicted outcome with an actual clinical outcome for the second patient.
[0018] In some embodiments, the content includes at least one of: a median survival time for the first patient group, or a Kaplan-Meier survival curve for the first patient group.
[0019] In some embodiments, the content includes values for one or more of a first subset of a plurality of features of the first patient, the first group of patients, and the second group of patients.
[0020] In some embodiments, a computer product includes a computer readable medium storing a plurality of instructions for controlling a computer system to perform the operations of any of the above methods.
[0021] In some embodiments, a system includes a computer product as described herein; and
[0022] One or more processors for executing instructions stored on a computer-readable medium.
[0023] In some embodiments, a system comprises means for performing any of the methods described herein.
[0024] In some embodiments, the system is configured to perform any of the methods described herein.
[0025] In some embodiments, the system includes modules that respectively perform the steps of any of the methods described herein.
[0026] These and other exemplary embodiments are described in detail below.For example, other embodiments relate to systems, devices, and computer-readable media associated with the methods described herein.
[0027] The nature and advantages of examples of the present disclosure may be better understood with reference to the following detailed description and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] A detailed description is given with reference to the accompanying drawings.
[0029] Figure 1A , Figure 1B and Figure 1C Exemplary techniques for facilitating clinical decision making based on clinical predictions are presented in accordance with certain aspects of the present disclosure.
[0030] Figure 2A , Figure 2B , Figure 2C , Figure 2D , Figure 2E as well as Figure 2F An improved clinical decision system capable of machine learning-based patient pooling according to certain aspects of the present disclosure is presented.
[0031] Figure 3 Methods for performing machine learning-based patient pooling operations according to certain aspects of the present disclosure are presented.
[0032] Figure 4An exemplary computer system is presented that can be used to implement the techniques disclosed herein.
[0033] Figure 5 Shows examples of how patient data from the patient pool can be used.
[0034] Figure 6 Another example of how patient data from the patient pool can be used is shown. DETAILED DESCRIPTION
[0035] As described above, predictive machine learning models can be trained to make clinical predictions to predict medical outcomes for new patients. A new patient can be any living patient, and a patient for whom a clinical decision is being made. For example, a random survival forest (RSF) model can be trained based on data from previous patients and their survival statistics to predict the probability of survival for a new patient as a function of time since diagnosis (e.g., advanced cancer). Predictions can be provided to new patients, for example, to improve their ability to plan for the future. This is likely to improve the quality of life of patients.
[0036] Although clinical predictions provided by predictive machine learning models can provide valuable information to new patients, clinical prediction results themselves may not provide insights into how to improve the prognosis of new patients. For example, a prediction that a patient has a certain probability of survival at a specific time point may not provide information about potential clinical decisions to improve the patient's probability of survival at that time point.
[0037] On the other hand, the medical journey of previous patients (whose data and survival statistics are used to train predictive machine learning models) can provide valuable insights into potential clinical decisions to improve the prognosis of new patients. For example, a machine learning model such as an RSF model can output a prediction of the probability that a new patient will survive to a specific time point since diagnosis. There may be a first patient group (e.g., Group A) whose survival probability with respect to time is similar to the survival probability predicted by the model for new patients, and a second patient group (e.g., Group B) whose survival probability is much lower than the survival probability predicted by the model for new patients. If Group A has a common biomarker with the new patient, and Group B does not have the biomarker, it can be determined that the biomarker can be associated with the survival probability of the new patient. A treatment decision can then be made to target the biomarker. However, as described above, although predictive machine learning models can play a role in predicting a patient's prognosis based on the patient's attributes, machine learning models generally do not identify other patient groups whose medical outcomes are similar to the patient's prognosis. In addition to providing clinical prediction results, machine learning models generally do not provide additional information that can be used to improve the patient's prognosis.
[0038] Disclosed herein is a technique for facilitating clinical decision making for new patients based on identifying a group of patients with similar attributes to the new patient (hereinafter, a "similarity-based patient pool"). A predictive machine learning model is provided to make clinical predictions for new patients who are alive or whose future survival is unknown. The similarity-based patient pool can be identified from a previous group of patients whose data and survival statistics are used to train the predictive machine learning model. At least some of the attributes of the similarity-based patient pool can be output to support clinical decision making. The attributes may include, for example, archival data of the patient, results of one or more laboratory tests of the patient, biopsy image data of the patient, molecular biomarkers of the patient, tumor site of the patient, and tumor stage of the patient.
[0039] In some examples, the clinical decision support system can use a machine learning model to make clinical predictions for new patients based on the attributes of the new patients. For example, a random survival forest (RSF) model can be used to predict the survival probability of a patient as a function of the time elapsed since diagnosis. The clinical decision support system can also identify a patient pool based on similarity with specific attributes similar to the specific attributes of the new patient. The patient pool based on similarity can be identified based on a subset of attributes that are determined to be most relevant to the clinical prediction performed by the machine learning model (e.g., the probability of survival at a specific time point since diagnosis) and share similar values with the new patient. The clinical decision support system can output the attributes and medical outcomes of the patient pool based on similarity, as well as the attributes and clinical prediction results of the patient, to facilitate clinical decision making for the patient. In some examples, the patient pool based on similarity can include patients whose attributes and survival statistics are included in the training data for training the machine learning model. In some examples, the patient pool based on similarity can also include patients whose data are not used to train the machine learning model.
[0040] Specifically, the clinical decision support system can receive first data corresponding to the attributes of the new patient. The attributes can include various archival information such as the patient's age and gender. Each attribute can be represented as a feature, which can include one or more vectors for input into the machine learning model. In some examples, the attribute can be represented as multiple features. The attribute can also include the patient's medical history (e.g., which (some) treatments the patient has received), the patient's habits (e.g., whether the patient smokes), and the category of the patient's laboratory test results (e.g., white blood cell count, hemoglobin count, platelet count, hematocrit count, red blood cell count, creatinine count, lymphocyte count, protein, bilirubin, calcium, sodium, potassium, glucose measurement). The attribute can also indicate the measurement of various biomarkers for different cancer types, such as estrogen receptor (ER) for breast cancer, progesterone receptor (PR), human epidermal growth factor receptor 2 (HER2), epidermal growth factor receptor (EGFR or HER1), ALK (anaplastic lymphoma kinase) for lung cancer, KRAS gene for lung cancer and colorectal cancer, BRAF gene for colorectal cancer, etc. The attribute data can be processed by the clinical decision support system, or before being input into the clinical decision support system, to create multiple features containing attribute information in a format (e.g., a vector) that can be interpreted by a machine learning model.
[0041] The clinical decision support system may include a machine learning model that can be trained based on data from previous patients to make clinical predictions for new patients. Predictions can be made based on inputting the attributes of the new patient into the machine learning model. For example, the machine learning model may include an RSF model that can output a predicted survival function based on the first data as a clinical prediction. The survival function can be used to obtain the probability that the new patient survives to a predetermined time (e.g., 500 days, 1000 days, 1500 days, etc.) after the new patient is diagnosed with a medical condition (e.g., advanced cancer). Alternatively, assuming survival to this time, the risk function provides the risk of death as a function of time. Another example of a survival function is a cumulative risk function (CHF), which provides a cumulative risk as a function of time. The survival function about time can be used to generate a patient-specific survival graph for a new patient.
[0042] As part of the training operation, multiple feature importance metrics associated with the machine learning model can also be obtained, wherein the feature importance metric defines the relevance of each feature to a clinical prediction (e.g., survival rate at a specific time point). In one example, an out-of-bag (OOB) sample of a sample of training patient data that is not used to build an RSF model can be input to a decision tree to calculate a prediction error such as a consistency index (c-index). Then, for those samples, the values of the feature can be arranged, and the prediction error of each decision tree can be calculated for the arranged values of the feature. The original importance score of the feature can be calculated based on the average difference in the prediction error among the trees for the arranged values. A higher original importance score can indicate that the feature is more relevant to the predicted survival function, while a lower original importance score can indicate that the feature is less relevant to the predicted survival function. At the end of the training operation, the features can be sorted based on the importance scores of the features, wherein the higher the relevance, the higher the feature ranking.
[0043] Based on the attributes of the new patient, the clinical decision support system can identify a group of patients from the patient database that are similar to the new patient in terms of the highest ranked features. This group can be called a similarity-based patient pool. The first step in selecting patients to form a similarity-based patient pool is to calculate the similarity between the new patient and each patient in the database based on the highest ranked features. Then, the patients who form the similarity-based patient pool are selected based on the criteria, and two examples of this operation are shown below. In the first example, patients can be selected based on the similarity with the new patient exceeding a threshold. In the second example, the patients in the database are sorted according to the similarity between the patients in the database and the new patient, and a predetermined number of patients with the highest ranking are selected. Therefore, it can be considered that the similarity-based patient pool not only has a similar health condition to the patient, but is also similar to the patient in terms of the features that are most relevant to clinical predictions.
[0044] The clinical decision support system can then output the attributes and medical outcomes of the similarity-based patient pool, as well as the attributes and clinical predictions for the new patient. This may help facilitate clinical decision making for the new patient. For example, the clinical decision support system can output a summary of the attributes of the similarity-based patient pool, as well as a comparison of the attributes between the new patient and the similarity-based patient pool (particularly the attributes corresponding to the highest ranked features). The output of the clinical decision support allows the clinician to study the relevant attributes and determine a course of action (e.g., treatment) to improve the probability of survival for the new patient.
[0045] As an illustrative example, a feature corresponding to a biomarker attribute (e.g., epidermal growth factor receptor (EGFR)) can be one of the highest ranked features for the RSF model. Assume that a new patient is EGFR-positive, and the clinical decision support system can output an EGFR-positive result for a similarity-based patient pool. If the similarity between the predicted survival function for the new patient and the predicted survival function of EGFR-positive patients from the similarity-based patient pool is higher than the similarity between the predicted survival function of EGFR-negative patients from the patient pool, it can be determined that a treatment targeting EGFR can be used to improve the new patient's probability of survival.
[0046] Using the disclosed techniques, similarity-based patient pools can be identified that not only have similar health conditions as the new patient, but are also similar with respect to the attributes / conditions that are most correlated with the clinical prediction. The correlation of the attributes with the clinical prediction makes it more likely that the medical journeys of the patients in the similarity-based patient pools can provide insights into potential treatments that can improve the prognosis for the new patient. These insights can be supported by the statistics and medical histories of a relatively large population of patients. For example, specific biomarkers in common between the similarity-based patient pool and the new patient can be studied to determine whether targeted treatments can improve the new patient's probability of survival.
[0047] I. Examples of Clinical Predictions and Applications
[0048] Figure 1A and Figure 1B Examples of clinical predictions that may be provided by embodiments of the present disclosure are presented. Figure 1A A mechanism for predicting a patient's cumulative probability of survival with respect to the time since a diagnosis of cancer was made is demonstrated, and Figure 1B An example application of survival probability prediction is shown. Figure 1A , chart 100 shows an example of a Kaplan-Meier (KM) graph, which provides a study of survival statistics between patients with a type of cancer (e.g., lung cancer). Patients can receive specific treatments. The KM graph shows the cumulative survival probability of a group of patients since the patient was diagnosed with cancer with respect to the measured time. In the case where the patient receives treatment, the KM graph also shows the cumulative survival probability of the patient's response to the treatment. Over time, some patients may die and the survival probability decreases. Some other patients may be censored (deleted) from the graph due to other events unrelated to the study events (e.g., they moved to a different state, so they changed hospitals). Censored events are represented by slash marks in the KM graph. The length of each horizontal line represents the time interval during which they did not die, and the survival estimate at a given point represents the cumulative probability of surviving to that time.
[0049] exist Figure 1A , graph 100 includes two KM plots of cumulative survival probabilities for different cohorts A and B of patients (e.g., cohorts of patients with different characteristics, receiving different treatments, etc.). Figure 1A In the study, the median survival time (the time when the cumulative survival probability first dropped below 50%) was approximately 11 months for cohort A and approximately 6.5 months for cohort B. For example, the probability that a patient in cohort A will survive for at least 8 months is approximately 70% (0.7), while the probability that a patient in cohort B will survive for at least 8 months is approximately 30% (0.3).
[0050] Figure 1B An exemplary application for survival prediction of patients is shown. Figure 1B As shown, data 102 of patient 103 can be input into clinical decision support tool 104 to generate survival prediction 106. Data 102 can include different attributes, such as archival data, medical history data, biomarkers, laboratory test result data, etc. Clinical decision support tool 104 can generate various information 108 to assist clinicians in giving care / treatment to patient 103 based on survival prediction 106. For example, in order to assist in the care of patient 103, clinical decision support tool 104 can generate information 108 to indicate, for example, the patient's life expectancy. Information 108 can contribute to the discussion between clinicians and patients 103 about the evaluation of the patient's prognosis and treatment options and the planning of the patient's life events. Two illustrative examples are given below. If clinical decision support tool 104 predicts that patient 103 has a relatively high probability of survival within 5 years, patient 103 may decide to accept aggressive treatment that is more physically demanding and has more severe side effects. But if clinical decision support tool 104 indicates that patient 103 has a relatively low probability of survival within 5 years, patient 103 may decide to forgo treatment or receive alternative treatments and plan care and life events for the remainder of his life.
[0051] Although survival predictions can provide valuable information to patients and clinicians, survival prediction results alone may not provide insights into how to improve a patient's prognosis. For example, a prediction that a patient 103 has a certain probability of survival beyond a specific time point may not provide information about potential treatments to improve the patient's likelihood of survival at that time point.
[0052] Making clinical decisions is a complex task in which clinicians must infer a diagnosis or treatment plan. Clinicians aim to match the best treatment based on their education, research, and personal experience. They usually operate on a per-patient basis and do not have digital solutions at hand that can assist them in fully exploiting the potential of medical knowledge obtained from real-world data (RWD). On the other hand, the increasing number of RWDs provides an opportunity to make complementary decisions using evidence-based, holistic information. Patient similarity is an essential component of studying the most and least effective treatments based on RWDs of similar individuals with comparable health conditions.
[0053] Figure 1C Examples of clinical decision making and clinical prediction outcomes based on RWD are shown. Figure 1C A graph 120 is shown that combines the first patient group ( Figure 1C KM diagram 122 of the second patient group ( Figure 1C 103 ) and a KM graph 124 of the patient 103 (labeled as "Group B"), and a survival prediction result 126 of the patient 103. The survival prediction result 126 of the patient 103 may be a function of time, wherein the predicted cumulative survival probability decreases over time. Figure 1C As shown, the predicted cumulative survival probability function 126 of patient 103 is more similar to the KM graph 124 of group B than to the KM graph 122 of group A.
[0054] Graph 130 shows an exemplary distribution of positive epidermal growth factor receptor (EGFR) between patient 103, group A patients (corresponding to KM graph 124) and group B patients (corresponding to KM graph 122). Patient 103 (corresponding to predicted cumulative survival probability function 126) has positive EGFR, so the bar for patient 103 in graph 130 is 100%. Approximately 60% of patients in group A have positive EGFR results, while less than 5% of patients in group B have positive EGFR results (all from graph 130). It should also be noted that cumulative survival curve 124 overlaps with curve 126, while curve 122 is much lower.
[0055] From the graphs 120 and 130, it can be determined that the patients in group A, who have a similar cumulative survival curve to patient 103 (as evidenced by the similarity between KM graph 124 and prediction result 126), have an EGFR positivity rate of about 60%. In contrast, group B, whose KM graph 122 shows a much lower survival probability than the prediction result 126 of patient 103, has an EGFR positivity rate of only 5%. This may indicate that the presence of EGFR may be an important factor in determining the survival probability of patient 103. Further studies such as studying treatments targeting EGFR may then be conducted based on this observation.
[0056] While such observations are useful and can provide insights into treatment options to improve the survival probability of patient 103, observations cannot generally be drawn solely from the survival probability prediction 106. For example, the prediction does not identify other patients with similar survival statistics. The prediction also does not identify other patients with similar health conditions as patient 103.
[0057] II. Similarity-Based Patient Pooling Using Machine Learning Models
[0058] Figure 2A An example of a clinical decision support system 200 for making clinical predictions for patients and identifying similarity-based patient pools (based on patient attributes involved in the clinical predictions) is shown. Figure 2A As shown, the clinical decision support system 200 includes a clinical prediction module 202, a patient pool determination module 204, and a portal 205. The clinical prediction module 202 may include a machine learning prediction model 206. The clinical prediction module 202 may receive patient data 208 corresponding to a plurality of features of a patient 210, and use the machine learning prediction model 206 to make a clinical prediction 212 for the patient based on the patient data 208. The patient 210 may be a new patient. The features of the patient data 208 may represent various attributes of the patient 210, including, for example, archival data 208a, medical history data 208b, biomarkers 208c, laboratory test result data 208d, etc. The clinical prediction 212 may include, for example, a survival probability of the patient. The survival probability may indicate the likelihood that the patient will survive to a predetermined time (e.g., 500 days, 1000 days, 1500 days, etc.) after the patient is diagnosed with a medical condition (e.g., advanced cancer). The clinical decision support system 200 may be a computer system (such as Figure 4 A software system executed on a computer system 10).
[0059] In addition, the patient pool determination module 204 can be connected to a patient database 214 storing patient data of a group of patients. As described below, the patient data of the patient database 214 can be used to train the machine learning prediction model 206. The patient pool determination module 204 can identify a patient pool and its patient data 216 having similar attributes to the patient 210 from the patient database 214. The patient pool determination module 204 can identify the patient pool based on these patients who are similar to the patient 210 in terms of a subset of attributes that are most relevant to the clinical prediction made by the machine learning prediction model 206. The clinical decision support system can then obtain the patient data 216 corresponding to the patient pool from the patient database 214. The portal 205 can perform additional processing of the patient data 216 (e.g., comparison between the patient data 216 of the patient pool and the patient data 208 of the patient 210).
[0060] Figure 2B A table 220 providing examples of attributes is shown, and the attributes include archival data 208a, medical history data 208b, biomarkers 208c, and laboratory test result data 208d. For example, archival data 208a can include various categories of information such as age, gender, and race. Medical history data 208b can include various categories of information, such as diagnosis results including cancer stage, histology, Charlson comorbidity index (CCI) (which predicts the risk of death based on the presence of specific comorbid conditions), Eastern Cooperative Oncology Group (ECOG) values (which describe the patient's functional level in terms of self-care ability, daily activities, and physical abilities). Medical history data 208b can also include other information, such as the patient's habits (e.g., whether the patient smokes). The laboratory test results 208c may include different categories of laboratory test results for the patient, such as white blood cell count, hemoglobin count, platelet count, hematocrit count, red blood cell count, creatinine count, lymphocyte count, protein, bilirubin, calcium, sodium, potassium, alkaline phosphatase, carbon dioxide, monocytes, chloride, lactate dehydrogenase, glucose, etc. The biomarker data 208d may include measurements of various biomarkers for different cancer types, such as estrogen receptor (ER), progesterone receptor (PR), human epidermal growth factor receptor 2 (HER2), epidermal growth factor receptor (EGFR or HER1) for breast cancer, ALK (anaplastic lymphoma kinase) for lung cancer, KRAS gene for lung cancer and colorectal cancer, BRAF gene for colorectal cancer, etc. It is understood that biopsy image data such as can also be included. Figure 2B Other attributes of the clinical data not shown are provided to the machine learning prediction model 206 to make clinical predictions.
[0061] In Table 220, each attribute can be represented by a continuous numeric feature, a binary feature (values can be one or zero), or some one-hot vector indicating a numerical value from a set of possible categories of the attribute. For example, age can be represented as a continuous numeric feature. As another example, the attribute corresponding to the test result of the biomarker ER can be one-hot encoded. Such an attribute can be associated with the following data categories: biomarker result positive, biomarker result negative, biomarker result invalid, and biomarker not tested. One-hot encoding can generate four features, each corresponding to one of the above categories. For each patient, only one of the four features will have a value of 1 (the feature corresponding to the category of its attribute), and the values of the other three will be 0. The following table illustrates this, where an example of four patients is given, each with a different category of the ER biomarker attribute.
[0062]
[0063] Table 1: Exemplary properties of the biomarker ER
[0064] A. Random Survival Forest
[0065] Figure 2A The machine learning prediction model 206 can be implemented using various techniques such as a random survival forest (RSF) model. Figure 2C An example of RSF model 230 is shown. Figure 2C As shown, the random survival forest model 230 may include multiple decision trees, including, for example, decision trees 232 and 234. Each decision tree may include multiple nodes, including a root node (e.g., root node 232a of decision tree 232, root node 234a of decision tree 234) and child nodes (e.g., child nodes 232b, 232c, 232d and 232e of decision tree 232, child nodes 234b and 234c of decision tree 234). Each parent node with child nodes (e.g., nodes 232a, 232b and 234a) may be associated with a predetermined classification criterion to classify a patient into one of its child nodes. Child nodes without child nodes are terminal nodes, which include nodes 232d and 232e (of decision tree 232) and nodes 234b and 234c (of decision tree 234). Node survival is calculated at each terminal node of each tree. When used to predict the survival of patient 210, patient 210 is assigned to the terminal node of each tree based on data 208 of patient 210. For example, decision tree 232 may output cumulative survival probability value 236, while decision tree 234 may output cumulative survival probability value 238. The survival probability of patient 210 may be calculated by averaging the node survival of each of the terminal nodes to which the patient is assigned. For example, average survival probability value 240 may be calculated based on the average between survival probability values 236, 238 and the survival probability values output by other decision trees.
[0066] Each decision tree can be assigned to process a different subset of features. For example, Figure 2C As shown, the patient data 242 includes a set of features {S0, S1, S2, S3, S4, . . . S n}. Each feature can represent Figure 2B, or any other attribute described herein. Decision tree 232 may be assigned to process features S0 and S1, decision tree 234 may be assigned to process feature S2, and other decision trees may be assigned to process other feature subsets. The parent node in the decision tree may then compare a subset of patient data 242 corresponding to one or more of the assigned features with one or more thresholds to classify patient 210 into one of its child nodes. For example, referring to decision tree 232, if the patient data for feature S0 exceeds threshold x0, root node 232a may classify the patient into child node 232b; otherwise, it is assigned to terminal node 232c. Child node 232b may further classify the parent node into one of terminal nodes 232d or 232c based on the patient data for feature S1. Based on which terminal node the patient is classified into based on features S0 and S1, decision tree 232 may output a cumulative survival probability of 10%, 20%, or 30%. Additionally, the decision tree 234 may also output a cumulative probability of survival of 50% or 90%, based on which terminal node the patient is classified into based on feature S2.
[0067] Can be built Figure 2C The RSF model 230 of the machine learning prediction model 206 is used to determine the cumulative survival probability from diagnosis to a predetermined time (e.g., 1 year, 3 years, 5 years, etc.). The machine learning prediction model 206 may include multiple RSF models. Figure 2A , the clinical prediction module 202 may receive the time 222 as an input for determining the cumulative survival probability. Then, the clinical prediction module 202 may select the RSF model trained for the time 222 to calculate the cumulative survival probability up to the time 222.
[0068] B. Training Operation
[0069] A training operation may be performed to generate each decision tree in the RSF model, a subset of features assigned to each decision tree, classification criteria at each parent node of the decision tree, and output values at each terminal node. Figure 2D An example of a training operation is shown. The training operation may be performed by a training module 250, which may be a clinical decision support tool 200 ( Figure 2A) is part of or external to the patient database 214. The training operation can be performed with patient data from a larger patient population from the patient database 214. As described above, the RSF model can be trained to determine the cumulative survival probability from diagnosis to a predetermined time. The training data used to train a specific RSF model to determine the cumulative survival probability from diagnosis to a predetermined time (e.g., 1, 3, 5, etc. years) may include deaths and censorings up to the predetermined time, and all surviving patients at the predetermined time will then be censored. In some examples, the training data may include deaths and censorings up to a longer period of time (e.g., 5 years of death and censoring data is used for an RSF model that outputs a cumulative survival probability of up to 3 years).
[0070] Specifically, the patient database 214 can store the attributes of the patients shown in the table 220. The training module 250 performs a process of randomly sampling the patient data 252 with replacement for the root node of each tree in the RSF model. The process of random sampling with replacement is generally referred to as "bootstrapping", and because all trees are combined / aggregated to form a random forest, the process is also called "bagging". Each tree is also assigned a random subset of features. Then, as part of the training operation, the root node (and each subsequent parent node) can be split into child nodes in a recursive node splitting process. During the node splitting process, a node including a subset of patients can be split into two child nodes based on a threshold for a subset of features. The features and their thresholds at each split are selected to maximize the difference in the probability of survival between the two child nodes (e.g., based on a log-rank test).
[0071] As an example, in Figure 2CDuring the training of the decision tree 232 of , it can be determined that the bootstrap sample of the patient data is divided into two groups based on the feature S0 and the threshold x0, and the difference in the survival probability between the two groups can be maximized, so that selecting different features (e.g., S1) or setting different thresholds for the feature S0 will result in a smaller difference between the survival probabilities of the two groups, and a child node 232a can be generated. Then the process can be repeated for the child node 232a to generate additional child nodes, until, for example, the threshold minimum number of patients is reached in a particular child node. Once the splitting process has stopped, all nodes without child nodes can become terminal nodes. For example, in at least one of the terminal nodes 232d and 232e, the number of patients reaches the threshold minimum number, and therefore the root splitting operation stops at these nodes. The output at each of these terminal nodes can be calculated from the outcome data of the patient classified into the terminal node. The training operation can be repeated to generate a decision tree for outputting the survival probability at different times, so that the RSF model 230 can output a survival function that predicts the survival probability of the patient at different times.
[0072] refer to Figure 2E , the training module 250 may also determine a feature importance metric 260 associated with the machine learning prediction model 206. The feature importance metric 260 may define the relevance of each feature by studying the impact of each feature on the error of the machine learning prediction model 206. Through the machine learning prediction model 206, the feature importance metric 260 may be determined for the survival probability up to a predetermined time (e.g., 3 years), and different feature importance metrics 260 may be determined for the survival probability predictions up to different predetermined times (e.g., 3 years, 5 years, 7 years, etc.).
[0073] In one embodiment, to determine the feature importance metric 260, the training module 250 may obtain a set of out-of-bag (OOB) samples of the patient data 252 from the patient database 214. The OOB samples of each tree may include Figure 2D Samples of patient data that were not included in the bootstrap sample for the tree in the training process. For those samples, the values of the features can be ranked, and the prediction error rate 262 for each decision tree that processed the OOB sample with the ranked values of the feature can be obtained. For the feature, a raw importance score 264 can be calculated based on, for example, averaging the differences in the prediction error rate outputs of each decision tree. This process can be repeated for each feature to calculate a respective raw importance score 264 for each feature. A high raw importance score can indicate that the feature is more relevant to survival prediction, while a low raw importance score can indicate that the feature is less relevant. At the end of the training operation, the features can be ranked based on their importance scores, with features with higher relevance ranking higher.
[0074] In one example, the training module 250 can calculate the prediction error rate 262 based on calculating a consistency index (c-index). The consistency index for the OOB sample can be calculated based on a pairwise comparison of the estimated cumulative hazard function (CHF) of the model and the actual death time between the patients in the OOB sample. For each pair of patients, if the relative survival probability of the pair of patients at a given time point matches the order of the death time of the pair of patients, then the pair of patients is consistent, otherwise the pair of patients is inconsistent. For example, if the CHF estimate of the first patient in the pair of patients is higher than the CHF estimate of the second patient in the pair of patients, and the first patient dies before the second patient, then the pair of patients is consistent. Otherwise, the pair of patients is inconsistent. The c-index can be calculated based on the following equation:
[0075]
[0076] The prediction error rate can then be calculated as the inverse of the C-index. Since the survival probability may change over time, the prediction error rate and thus the raw importance score may also change over time. Therefore, if Figure 2E As shown, feature importance metric 260 may include different raw importance scores 264 for different times 266 .
[0077] C. Similarity-based patient pooling
[0078] Based on the feature importance metric 260 and the patient data 208 of the patient 210 , the patient pool determination module 204 can identify a pool of patients and their patient data 216 from a patient database 214 that have similar attributes to the patient 210 . Figure 2F 2 shows exemplary internal components and operations of the patient pool determination module 204. Figure 2F As shown, the patient pool determination module 204 includes a feature weight selection module 270 and a similarity determination module 272 .
[0079] The feature weight selection module 270 can sort the features by feature importance value 260 and select the x features with the highest feature importance values (x can be a predetermined number, such as 20, or based on a rule, such as all features whose importance values are greater than the average importance value across all features). The set of x highest ranked features can be denoted as E. The feature weight selection module 270 can then fit the RSF using only the features in E and recalculate the feature importance values of these features based on this new RSF. These new raw feature importance values are scaled according to the following equation, with feature k denoted as w k :
[0080] ∑ k∈E w k =1; w k>0 for k∈E (Equation 2)
[0081] Then, using the scaled feature importance value, w k , used as a weight in the similarity determination module 272.
[0082] The similarity determination module 272 can then identify patients in the patient database 214 that are similar to the patient 210 based on the scaled feature importance values / weights 274. The similarity determination module 272 can determine the similarity between two patients (x i and x j ) is the weighted aggregate similarity s(x) between i , x j ):
[0083] s(x i ,x j )=∑ k w k s ijk (Equation 3)
[0084] In Equation 3, s ijk Represents two patients x i and x j The degree of similarity between them in terms of feature k, and w k is the scaled feature importance value 274. More important features may have similarity degrees associated with larger weights. In the case where feature k is represented by a binary value or a one-hot encoded vector, if feature k of both patients is one, or the one-hot vectors match exactly, then the similarity degree s ijk can be 1; otherwise s ijk The value of is taken as zero. In addition, the value of the characteristic k is taken from the range R k In the case of , the similarity degree s can be calculated based on the following equation ijk :
[0085]
[0086] The similarity determination module 272 can calculate the weighted aggregate similarity s(x) between the patient 210 and the patients represented by the patient database 214 using Equation 3. i , x j), and selects a similarity-based patient pool based on the weighted aggregate similarity. The similarity-based patient pool may be considered to have not only similar health conditions as the patient, but also similar features with the patient in terms of the features with the highest correlation to the clinical prediction. In one example, the similarity determination module 272 may select a similarity-based patient pool based on the degree of similarity between the similarity-based patient pool calculated according to Equation 3 or 4 and the patient 210 exceeding a similarity threshold 280. In another example, the similarity determination module 272 may select a predetermined number of patients with the highest degree of similarity to the patient 210 as part of the similarity-based patient pool, defined based on a pool size threshold 282.
[0087] The similarity determination module 272 can then obtain attributes and medical outcomes of the patient pool based on similarity and output them as part of the patient data 216 to facilitate clinical decision making for the patient 210. Figure 1C , the similarity determination module 272 can identify the patient pool and patient data 216 based on similarity, and a KM curve 124 can be generated for the patient pool based on similarity. The portal 205 can compare the patient data 216 of the patient pool, the patient data 208, and the patient data of the training patient set in the patient database 214 for each feature present in each set of data, and output the comparison result. Based on the comparison, the clinician can determine that the EGFR positivity of the patient pool is much higher than the EGFR positivity of the training patient set, and further research on EFGR (e.g., treatment targeting EFGR) can be performed to improve the patient's survival rate.
[0088] III. Methods
[0089] Figure 3 An example of a method 300 for facilitating clinical decision making is shown. The method 300 may be performed by, for example Figure 2A 200 clinical decision support tools were performed.
[0090] In step 302, a clinical decision support tool may receive first data corresponding to a plurality of features of a first patient (e.g., a new patient), wherein each feature represents an attribute of a plurality of attributes. The first data may be input through a computer interface such as portal 205, or directly from a patient database such as patient database 214. The first patient may be a new patient such as patient 210.
[0091] refer to Figure 2A, the first data may include patient data 208 corresponding to the attributes of the new patient. The attributes may include various archival information such as the patient's age and gender. Each attribute may be represented as one or more features, and each feature may be represented as a vector for input into a machine learning model. The attributes may also include the patient's medical history (e.g., which treatment(s) the patient has received), the patient's habits (e.g., whether the patient smokes), the category of the patient's laboratory test results (e.g., white blood cell count, hemoglobin count, platelet count, hematocrit count, red blood cell count, creatinine count, lymphocyte count, protein, bilirubin, calcium, sodium, potassium, glucose measurement). The attributes may also indicate the measurement of various biomarkers for different cancer types, such as estrogen receptor (ER), progesterone receptor (PR), human epidermal growth factor receptor 2 (HER2), epidermal growth factor receptor (EGFR or HER1) for breast cancer, ALK (anaplastic lymphoma kinase) for lung cancer, KRAS gene for lung cancer and colorectal cancer, BRAF gene for colorectal cancer, etc.
[0092] In step 304, the clinical decision support tool may input the first data into a machine learning model to generate a result of a clinical prediction for the first patient, the machine learning model being associated with a plurality of feature importance metrics defining a relevance of each of a plurality of features to the clinical prediction.
[0093] refer to Figure 2A , the clinical decision support tool may include a machine learning prediction model 206 to make clinical predictions 212 based on the patient's data. The machine learning model 206 may include Figure 2C The RSF model 230 of the present invention can output a predicted survival function based on the patient's data as a clinical prediction. The clinical prediction 212 can include, for example, a patient's survival probability. The survival probability can indicate the likelihood that the patient will survive to a predetermined time (e.g., 500 days, 1000 days, 1500 days, etc.) after the patient is diagnosed with a medical condition (e.g., advanced cancer). As described above, in some examples, the machine learning prediction model 206 can include multiple RSF models configured to predict survival probabilities as of different predetermined times. One of the RSF models can be selected to predict the survival probability as of an input time based on the input time.
[0094] In addition, the machine learning prediction model 206 is also associated with a plurality of feature importance metrics, such as feature importance metrics 260. Figure 2E, the feature importance metric 260 can define the relevance of each feature by studying the impact of each feature on the error of the machine learning prediction model 206, and can be determined by a training module 250 based on a set of out-of-bag (OOB) samples of patient data 252 from the patient database 214. OOB samples may include samples of patient data that are not involved in the bagging process for building the RSF model 230. For those samples, the values of the features can be arranged, and the prediction error rate of each decision tree from processing the OOB samples with the arranged values of the feature can be obtained. The prediction error rate can be calculated based on the consistency index (c-index) based on the above equation 1. For this feature, the original importance score can be calculated based on, for example, averaging the difference of the prediction error rate output of each decision tree. This process can be repeated for each feature to calculate the respective original importance score for each feature. A high original importance score can indicate that the feature is more relevant to survival prediction, while a low original importance score can indicate that the relevance of the feature is low. At the end of the training operation, the features can be ranked based on the importance scores of the features, where the features with higher relevance are ranked higher.
[0095] In step 306, the clinical decision support tool may obtain second data corresponding to multiple features of each patient in the patient group based on a degree of similarity between the first patient and the patient group with respect to at least some of the multiple features, the degree of similarity being based on the first data, the second data, and multiple feature importance measures.
[0096] Specifically, the second data can be obtained by the patient pool determination module 204 of the clinical decision support tool 200, and the patient pool determination module includes a feature weight selection module 270 and a similarity determination module 272. Figure 2F , the feature weight selection module 270 can sort the features by feature importance value 260, and select x features with the highest feature importance values (x can be a predetermined number, such as 20, or based on a rule, such as all features whose importance values are greater than the average importance value across all features). The set of x highest ranked features can be recorded as E. The feature weight selection module 270 can then fit the RSF using only the features in E, and recalculate the feature importance values of these features based on this new RSF. The degree of similarity between the first patient and the other patients can be calculated based on equations 3 to 4 above. In the first example, a patient can be selected based on a similarity with a new patient exceeding a threshold. In the second example, the patients in the database are sorted according to their similarity to the first patient, and a predetermined number of patients with the highest rankings are selected.
[0097] In step 308 , the clinical decision support tool may generate content based on the results of the clinical prediction and at least a portion of the second data.
[0098] Specifically, in some examples, the content may include output summary statistics (e.g., median survival time) of the patient pool (patient group), KM curve of the patient pool, etc. In some examples, comparisons may be made between the patient data of the patient group, the patient data of the first patient, and the patient data of a training patient set (e.g., patients represented by a training data set for training a machine learning model) to generate a comparison result.
[0099] In step 310, the clinical decision support tool may output the content so as to make a clinical decision for the first patient based on the content. Figure 1C , the content may indicate that the EGFR positivity of the patient pool is much higher than that of the training patient set, and further research on EFGR (e.g., treatment targeting EFGR) may be performed to improve patient survival.
[0100] IV. Computer System
[0101] Any computer system mentioned herein may utilize any suitable number of subsystems. Examples of such subsystems are Figure 4 10 is shown in FIG. 10. In some embodiments, the computer system includes a single computer device, wherein the subsystem can be a component of the computer device. In other embodiments, the computer system can include multiple computer devices, each of which is a subsystem with internal components. The computer system can include desktop computers and portable computers, tablet computers, mobile phones and other mobile devices. In some embodiments, cloud infrastructure (e.g., Amazon Web Services), graphics processing units (GPUs), etc. can be used to implement the disclosed technology.
[0102] Figure 4 The subsystems shown are interconnected via a system bus 75. Additional subsystems are shown, such as a printer 74, a keyboard 78, a storage device 79, a monitor 76 (which is coupled to a display adapter 82), etc. Peripheral devices and input / output (I / O) devices coupled to the I / O controller 71 may be coupled to the I / O controller 71 by any number of means known in the art, such as an input / output (I / O) port 77 (e.g., USB, ) is connected to the computer system. For example, I / O port 77 or external interface 81 (e.g., Ethernet, Wi-Fi, etc.) can be used to connect the computer system 10 to a wide area network, such as the Internet, a mouse input device, or a scanner. The interconnection through the system bus 75 allows the central processor 73 to communicate with each subsystem and control the execution of multiple instructions from the system memory 72 or storage device 79 (e.g., a fixed disk, such as a hard drive, or an optical disk), as well as the exchange of information between the subsystems. The system memory 72 and / or the storage device 79 may include a computer readable medium. Another subsystem is a data collection device 85, such as a camera, a microphone, an accelerometer, etc. Any data mentioned herein can be output from one component to another component and can be output to a user.
[0103] The computer system may include multiple identical components or subsystems, for example connected together via external interface 81 or via internal interface. In some embodiments, the computer system, subsystem or device may communicate via a network. In this case, one computer may be considered a client and another computer may be considered a server, wherein each computer may be considered part of the same computer system. The client and server may each include multiple systems, subsystems or components.
[0104] Aspects of the embodiments may be implemented in a modular or integrated manner using hardware (e.g., an application specific integrated circuit or a field programmable gate array) and / or computer software with a general programmable processor in the form of control logic. As used herein, a processor includes a single-core processor, a multi-core processor on the same integrated chip, or multiple processing units on a single circuit board or networked. Based on the disclosure and inspiration provided herein, one of ordinary skill in the art will know and understand other ways and / or methods of implementing embodiments of the present invention using hardware and a combination of hardware and software.
[0105] Any suitable computer language, such as, for example, Java, C, C++, C#, Objective-C, Swift, or a scripting language, such as Perl or Python, can be used to implement any software component or function described in this application as a software code executed by a processor using, for example, conventional techniques or object-oriented techniques. The software code can be stored on a computer-readable medium as a series of instructions or commands for storage and / or transmission. Suitable non-transitory computer-readable media may include random access memory (RAM), read-only memory (ROM), magnetic media (such as a hard drive or floppy disk), or optical media (such as a compact disk (CD) or DVD (digital versatile disk), flash memory, etc.). The computer-readable medium may be any combination of such storage or transmission devices.
[0106] Such programs can also be encoded and transmitted using carrier signals, which are adjusted to be suitable for transmission via wired networks, optical networks and / or wireless networks that meet various protocols including the Internet. In this way, computer-readable media can be created using data signals encoded through such programs. Computer-readable media encoded with program code can be packaged with compatible devices, or provided separately from other devices (e.g., downloaded via the Internet). Any such computer-readable media can reside on or inside a single computer product (e.g., a hard drive, a CD, or an entire computer system), and can be present on or inside different computer products in a system or network. A computer system can include a monitor, a printer, or other suitable displays for providing any result mentioned herein to a user.
[0107] V. Examples
[0108] Figure 5 An example of how patient data 500 from a patient pool can be used is shown. Building patient data from a patient pool involves cohort construction, and any suitable type of cohort construction method can be used, such as the similarity-based patient pooling method described herein. For example, from a combination of Figure 2A and Figure 2F The method for generating patient data based on similarity-based patient pools can be used in the examples described herein. Other methods for cohort construction may also be applicable.
[0109] like Figure 5 As shown, patient data 500 from a patient pool can be accessed, processed, and / or used by a disease journey information tool 502 to automatically extract useful data about a patient's disease journey from a patient's electronic health record and / or other patient database. In some embodiments, the disease journey information tool 502 may include a patient care information extraction module 504, a patient health status extraction module 506, and an additional patient treatment and service module 508. Various embodiments of the disease journey information tool 502 may include any combination of one or more of the modules described herein.
[0110] In some embodiments, the patient care information extraction module 504 includes an algorithm for extracting information about the order in which patients are cared for in a cohort. This extracted information can be displayed to the user to assist and facilitate the user in learning from the disease journey of a particular cohort. This extracted information can also be used for risk factor analysis of a cohort.
[0111] In some embodiments, the patient health status extraction module 506 may include an algorithm for extracting information about the patient's health from the patient data. For example, the measure of some patients' health status includes, for example, patient-reported outcomes or experiences, such as reported symptoms, disabilities, various aspects of health status, health concepts, etc. In some embodiments, a more customized display may be provided to the user, which may be used to view the health status of the patient as a patient group from an individual level and in a cohort. For example, the user may be provided with overall statistics on how patients in an identified cohort report their treatment.
[0112] In some embodiments, the additional patient treatments and services module 508 may include algorithms for extracting information about non-medical additional services (i.e., non-pharmaceutical and non-drug services) such as, for example, rehabilitation, psychotherapy, physical therapy, and occupational therapy. In some embodiments, this extracted information can be used to determine which additional non-medical interventions benefit a particular patient cohort, such as a rehabilitation clinic, psychotherapy, physical therapy, and / or occupational therapy.
[0113] Figure 6 Another example of how patient data 600 from a patient pool may be used is shown. Figure 6 As shown, patient data 600 from a patient pool may be accessed, processed, and used by a suggestion tool 602 to suggest common terms for entry into data fields of a spreadsheet or record. In some embodiments, the suggestion tool 602 may include a common electronic medical record (EMR) term extraction module 604 and a common diagnostic test extraction module 606. Various embodiments of the suggestion tool 602 may include any combination of one or more of the modules described herein.
[0114] In some embodiments, the common EMR term extraction module 604 may include an algorithm for extracting common terms for data fields in the EMR system, and then the algorithm uses these extracted common terms as suggestions to the user who is filling out the EMR. For example, in some embodiments, one or more of the data fields in the EMR that the user is filling out can be automatically filled in with a pre-selected text field based on the highest frequency of common terms extracted in the queue. In some embodiments, when a text field is selected, a ranked list of common terms can be provided to the user instead, where the list is sorted based on the frequency of the terms appearing in the queue. In some embodiments, common EMR terms can be extracted from EMRs from a patient pool in the queue. In other embodiments, common EMR terms can be extracted from a wider EMR data set formed from a larger patient pool. In some embodiments, common EMR terms can be extracted from one or more EMR data sets.
[0115] In some embodiments, the common diagnostic test extraction module 606 may include an algorithm for extracting common diagnostic tests for use in the diagnostic test recommendation system. In some embodiments, the data set used to extract common diagnostic tests may be limited to data from a cohort patient pool. Figure 2A and Figure 2F The method of generating patient data from a similarity-based patient pool can be used to construct a cohort of patients with similar characteristics to a particular patient. The recommendation system identifies diagnostic tests to be performed by the cohort. The system can use the extracted information to recommend these tests to be considered for a particular patient.
[0116] In some embodiments, the algorithm used by the data extraction module described herein may be, but is not limited to, a process mining algorithm, a deep learning algorithm, and a sequence alignment method.
[0117] Any method described herein can be performed in whole or in part by a computer system including one or more processors, which can be configured to perform steps. Therefore, embodiments can be directed to a computer system configured to perform the steps of any method described herein, and may have different components for performing corresponding steps or corresponding step groups. Although presented in numbered steps, the steps of the method herein can be performed simultaneously or in different orders. In addition, some steps can be used together with some steps in other methods. In addition, all or part of the steps can be optional. In addition, any step of any method can be performed with modules, units, circuits or other devices for performing these steps.
[0118] Without departing from the spirit and scope of the embodiments of the present invention, the specific details of the specific embodiments may be combined in any suitable manner. However, other embodiments of the present invention may be directed to specific embodiments related to each individual aspect, or specific combinations of these individual aspects.
[0119] The above description of exemplary embodiments of the present invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form described, and many modifications and variations are possible in light of the above teaching.
[0120] Reference to "a," "an," or "the" is intended to mean "one or more," unless specifically indicated to the contrary. The use of "or" is intended to mean "inclusive or," not "exclusive or," unless specifically indicated to the contrary. Reference to a "first" component does not necessarily require that a second component be provided. Furthermore, reference to a "first" or "second" component does not limit the referenced component to a specific location, unless explicitly stated.
[0121] All patents, patent applications, publications, and specifications mentioned herein are incorporated by reference in their entirety for all purposes. None is admitted to be prior art.
Claims
1. A computer-implemented method for facilitating clinical decision making, the computer-implemented method comprising: receiving first data corresponding to a plurality of features of a first patient, each feature representing an attribute of a plurality of attributes; inputting the first data into a machine learning model to generate a result of a clinical prediction for the first patient, the machine learning model being associated with a plurality of feature importance metrics defining a relevance of each of the plurality of features to the clinical prediction; obtaining second data corresponding to the plurality of features for each patient in the group of patients based on a degree of similarity between the first patient and the group of patients with respect to at least some of the plurality of features, the degree of similarity being based on the first data, the second data, and a plurality of feature data importance measures; generating content based on the result of the clinical prediction and at least a portion of the second data; as well as The content is outputted so as to make a clinical decision for the first patient based on the content.
2. The method of claim 1, wherein the plurality of attributes comprises at least one of: archival data of a patient, results of one or more laboratory tests of the first patient, biopsy image data of the first patient, molecular biomarkers of the first patient, tumor site of the first patient, or tumor stage of the first patient. 3 . The method of claim 1 , wherein the plurality of attributes comprises one or more attributes representing measurements of biomarkers for different cancer types.
4. The method of claim 1, wherein the clinical prediction comprises at least one of: a probability of survival of the first patient at a predetermined time from when the first patient was diagnosed with a tumor, a survival time of the first patient from when the first patient was diagnosed with a tumor, or an outcome of receiving treatment.
5. The method of claim 4, wherein the machine learning model comprises a random forest survival model, the random forest survival model comprising f decision trees, the decision trees each configured to process a subset of the first subset of the data to generate a cumulative survival probability; and The survival rate of the patient at the predetermined time is determined based on an average value of the cumulative survival probabilities output by a plurality of decision trees.
6. The method of claim 1, wherein the patient group is a first patient group; wherein the first patient group is selected from a second patient group; and Wherein the machine learning model is trained based on the patient data of the second patient group.
7. The method according to claim 6, further comprising: ranking the plurality of features based on the relevance of each feature in the plurality of features to the clinical prediction; determining a subset of the plurality of features based on the ranking; as well as The first patient group is determined based on a degree of similarity between the first patient and the first patient group with respect to the subset of the plurality of features.
8. The method of claim 7, wherein the first patient group is selected from the second patient group based on the degree of similarity between the first patient and the first patient group with respect to the subset of the plurality of features exceeding a threshold.
9. The method of claim 7, wherein the first group of patients is selected from the second group of patients based on selecting a threshold number of patients having a highest degree of similarity to the first patient with respect to the subset of the plurality of features.
10. The method according to claim 1, further comprising: calculating a weighted aggregate degree of similarity based on summing the scaled degrees of similarity for each of the at least some of the plurality of features, each degree of similarity being scaled by a weight based on the degree of relevance of the features; as well as The group of patients is identified based on the weighted aggregate degree of similarity between the first patient and each patient in the group of patients.
11. The method of claim 1 , wherein a feature importance measure of a feature is determined based on a relationship between errors of an outcome of a clinical prediction generated by the machine learning model for a second patient in the first group of patients; wherein said outcome of the clinical prediction is generated from a plurality of values of said characteristic of said second patient; and wherein the error is calculated based on comparing the result of the clinical prediction with the actual clinical outcome of the second patient.
12. The method of claim 6, wherein the content comprises at least one of the following: a median survival time of the first patient group, or a Kaplan-Meier survival curve of the first patient group.
13. The method of claim 6, wherein the content comprises values for one or more of the first subset of the plurality of features for the first patient, the first group of patients, and the second group of patients.
14. A computer product comprising a computer readable medium storing a plurality of instructions for controlling a computer system to perform the operations of any of the above methods.
15. A system comprising: The computer product according to claim 16; as well as One or more processors for executing instructions stored on the computer-readable medium.
16. A system comprising means for performing any of the above methods.
17. A system configured to perform any of the above methods.
18. A system comprising modules respectively performing the steps of any one of the above methods.
