Machine learning-based methods and systems for predicting risk based on multi-modal health data

Abstract

An example method for predicting medical risk is described herein. The method includes receiving multi-modal health data for a subject, and inputting the multi-modal health data into a trained machine learning model. The method also includes predicting, using the trained machine learning model, a medical risk for the subject. The multi-modal health data for the subject includes patient-generated health data (PGHD), patient-reported outcome (PRO) data, and imaging data.

Description

MACHINE LEARNING-BASED METHODS AND SYSTEMS FOR PREDICTING RISK BASED ON MULTIMODAL HEALTH DATACROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. provisional patent application No. 63 / 511,708, filed on July 3, 2023, and titled "MACHINE LEARNING-BASED METHODS AND SYSTEMS FOR PREDICTING RISK BASED ON MULTI-MODAL HEALTH DATA," the disclosure of which is expressly incorporated herein by reference in its entirety.STATEMENT REGARDING FEDERALLY FUNDED RESEARCH

[0002] This invention was made with government support under Grant no. W81XWH-20-1- 0351 awarded by the Department of Defense. The government has certain rights in the invention.BACKGROUND

[0003] As oncology moves towards greater adoption of novel therapies, there is need to advance symptom monitoring and early detection of patient deterioration and toxicities. Each year, nearly 2,000,000 people in the U.S. develop cancer, and many of those individuals experience treatment-related toxicities or deterioration due to other factors. Improving the ability to predict which patients will develop toxicities or deterioration is expected to optimize treatment decision-making, aid in earlier detection and intervention, and ultimately improve survival.SUMMARY

[0004] Methods and systems for predicting, using machine learning models, medical risks based on multi-modal health data are described herein.

[0005] In some aspects, the techniques described herein relate to a method for predicting medical risk, the method including: receiving multi-modal health data for a subject; inputting the multimodal health data into a trained machine learning model; and predicting, using the trained machine learning model, a medical risk for the subject, wherein the multi-modal health data for the subject includes patient-generated health data (PGHD), patient-reported outcome (PRO) data, and medical imaging data.

[0006] In some aspects, the multi-modal health data for the subject further includes electronic medical record (EMR) data.

[0007] In some aspects, the multi-modal health data for the subject further includes clinical data.

[0008] In some aspects, the patient-generated health data is recorded by one or more wearable sensors.

[0009] In some aspects, the patient-generated health data includes at least one of sleep data, heart rate data, physical activity data, body temperature, weight, calorie intake data, or macronutrient intake data.

[0010] In some aspects, the PRO data includes a respective metric related to at least one of pain, fatigue, drowsiness, nausea, appetite, shortness of breath, depression, anxiety, constipation, diarrhea, sleep, distress, or suicidal ideation.

[0011] In some aspects, the medical imaging data includes one or more imaging features.

[0012] In some aspects, the medical risk for the subject is related to an unplanned medical encounter; a change in medication; an adverse reaction to a drug, pharmaceutical, substance, compound, or treatment; a development of a disease or condition; a relapse or recurrence of a disease or condition; or a death.

[0013] In some aspects, the method further includes administering a treatment to the subject based upon the predicted medical risk for the subject.

[0014] In some aspects, the step of administering the treatment to the subject includes changing a dosage amount or dosage schedule for a drug, pharmaceutical, substance, or compound; changing a type of a drug, pharmaceutical, substance, or compound; or changing an amount, frequency, or type of treatment.

[0015] In some aspects, the method further includes selecting a plurality of features from the multi-modal health data, wherein the selected features are input into the trained machine learning model.

[0016] In some aspects, the trained machine learning model is a supervised learning model.

[0017] In some aspects, the trained machine learning model is a logistic regression model, a logistic classifier model, a support vector machine, or an artificial neural network.

[0018] In some aspects, the techniques described herein relate to a system for predicting medical risk, the system including: a computing device including a processor and a memory, the memory having computer-executable instructions stored thereon that, when executed by the processor, cause the processor to: receive multi-modal health data for a subject; input the multi-modal health data into a trained machine learning model; and receive a medical risk for the subject, the medical risk being predicted by the trained machine learning model, wherein the multi-modal health data for the subject includes patient-generated health data (PGHD), patient-reported outcome (PRO) data, and medical imaging data.

[0019] In some aspects, the multi-modal health data for the subject further includes electronic medical record (EMR) data.

[0020] In some aspects, the multi-modal health data for the subject further includes clinical data.

[0021] In some aspects, the patient-generated health data is recorded by one or more wearable sensors.

[0022] In some aspects, the patient-generated health data includes at least one of sleep data, heart rate data, physical activity data, body temperature, weight, calorie intake data, or macronutrient intake data.

[0023] In some aspects, the PRO data includes a respective metric related to at least one of pain, fatigue, drowsiness, nausea, appetite, shortness of breath, depression, anxiety, constipation, diarrhea, sleep, distress, or suicidal ideation.

[0024] In some aspects, the medical imaging data includes one or more imaging features.

[0025] In some aspects, the medical risk for the subject is related to an unplanned medical encounter; a change in medication; an adverse reaction to a drug, pharmaceutical, substance, compound, or treatment; a development of a disease or condition; a relapse or recurrence of a disease or condition; or a death.

[0026] In some aspects, the trained machine learning model is a supervised learning model.

[0027] In some aspects, the trained machine learning model is a logistic regression model, a logistic classifier model, a support vector machine, or an artificial neural network.

[0028] In some aspects, the techniques described herein relate to a method for training a machine learning model, the method including: receiving multi-modal health data for each of a plurality of subjects; processing, using a data integration machine learning model, the multi-modal health data to create a data set; storing the data set in a data repository; and training a machine learning model with the data set, wherein the trained machine learning model is configured to predict a medical risk for a subject, and wherein the multi-modal health data for the subject includes patient-generated health data(PGHD), patient-reported outcome (PRO) data, and medical imaging data.

[0029] In some aspects, the techniques described herein relate to a method for predicting treatment-related symptoms in a subject, including: receiving multi-modal health data for the subject; inputting the multi-modal health data into a trained machine learning model; and predicting, using the trained machine learning model, a risk of immunotherapy-induced lung inflammation in the subject, wherein the multi-modal health data for the subject includes patient-generated health data (PGHD), patient-reported outcome (PRO) data, and medical imaging data.

[0030] In some aspects, the patient-generated health data is recorded by one or more wearable sensors.

[0031] In some aspects, the patient-generated health data includes at least one of sleep data, heart rate data, or physical activity data.

[0032] In some aspects, the PRO data includes a respective metric related to at least one of wheezing, pain, fatigue, drowsiness, nausea, appetite, shortness of breath, depression, anxiety, constipation, diarrhea, sleep, distress, or suicidal ideation.

[0033] In some aspects, the medical imaging data includes one or more computed tomography (CT) scan, positron emission tomography (PET) scan, or CT / PET scan imaging features.

[0034] In some aspects, the method further includes predicting, using the trained machine learning model, a severity of the immunotherapy-induced lung inflammation in the subject.

[0035] In some aspects, the multi-modal health data for the subject includes pre-treatment data only.

[0036] In some aspects, the multi-modal health data for the subject includes post-treatment data.

[0037] In some aspects, the multi-modal health data for the subject further includes pretreatment data.

[0038] In some aspects, the techniques described herein relate to a method for predicting medical risk for a subject having non-small cell lung cancer (NSCLC), the method including: receiving multi-modal health data for the subject having NSCLC, wherein the multi-modal health data for the subject includes patient-generated health data (PGHD) and patient-reported outcome (PRO) data; inputting the multi-modal health data into a trained machine learning model; and predicting, using the trained machine learning model, a medical risk related to treatment of the subject having NSCLC.

[0039] In some aspects, the multi-modal health data for the subject further includes electronic medical record (EMR) data.

[0040] In some aspects, the multi-modal health data for the subject further includes clinical data.

[0041] In some aspects, the PGHD is recorded by one or more wearable sensors.

[0042] In some aspects, the PGHD includes at least one of sleep data, heart rate data, physical activity data, body temperature, weight, calorie intake data, or macronutrient intake data.

[0043] In some aspects, the PRO data includes a respective metric related to at least one of pain, fatigue, drowsiness, nausea, appetite, shortness of breath, depression, anxiety, constipation, diarrhea, sleep, distress, or suicidal ideation.

[0044] In some aspects, the medical risk is a risk of healthcare utilization.

[0045] In some aspects, the risk of healthcare utilization is an urgent care or emergency room visit by the subject.

[0046] In some aspects, the method further includes administering a treatment to the subject based upon the predicted medical risk.

[0047] In some aspects, the step of administering the treatment to the subject includes changing a dosage amount or dosage schedule for a drug, pharmaceutical, substance, or compound;changing a type of a drug, pharmaceutical, substance, or compound; or changing an amount, frequency, or type of treatment.

[0048] In some aspects, the trained machine learning model is a supervised learning model.

[0049] In some aspects, the trained machine learning model is a Bayesian network.

[0050] In some aspects, the techniques described herein relate to a system for predicting medical risk for a subject having non-small cell lung cancer (NSCLC), the system including: a computing device including a processor and a memory, the memory having computer-executable instructions stored thereon that, when executed by the processor, cause the processor to: receive multi-modal health data for the subject having NSCLC, wherein the multi-modal health data for the subject includes patient-generated health data (PGHD) and patient-reported outcome (PRO) data; input the multi-modal health data into a trained machine learning model; and receive a medical risk for the subject, the medical risk being predicted by the trained machine learning model, wherein the medical risk is related to treatment of the subject having NSCLC.

[0051] It should be understood that the above-described subject matter may also be implemented as a computer-controlled apparatus, a computer process, a computing system, or an article of manufacture, such as a computer-readable storage medium.

[0052] Other systems, methods, features and / or advantages will be or may become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features and / or advantages be included within this description and be protected by the accompanying claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The components in the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding parts throughout the several views.

[0054] FIGURE 1 is a diagram illustrating a machine learning model for medical risk prediction according to an implementation described herein.

[0055] FIGURE 2 is a diagram illustrating an example machine learning model training method according to an implementation described herein.

[0056] FIGURE 3 is an example computing device.

[0057] FIGURE 4 is a table (Table 1) that shows NSCLC patients' available clinical, demographic data and additional patient-reported outcome (PRO) and wearable sensor data (WSD) before and during systemic therapy (ST) according to an Example described herein.

[0058] FIGURE 5 is a flow chart illustrating a process to obtain a Bayesian Network (BN) based on a simplified Human-in-the-Loop BN approach according to an Example described herein.

[0059] FIGURE 6 is a table (Table 2) that shows demographic and clinical characteristics of participants who received systemic therapy during the study (n = 58) according to an Example described herein.

[0060] FIGURE 7 illustrates a predictive model of urgent care or emergency department visits and prediction performance (AUC=0.86, 95% Cl: 0.76-0.95) according to an Example described herein.DETAILED DESCRIPTION

[0061] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. Methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present disclosure. As used in the specification, and in the appended claims, the singular forms "a," "an," "the" include plural referents unless the context clearly dictates otherwise. The term "comprising" and variations thereof as used herein is used synonymously with the term "including" and variations thereofand are open, non-limiting terms. The terms "optional" or "optionally" used herein mean that the subsequently described feature, event or circumstance may or may not occur, and that the description includes instances where said feature, event or circumstance occurs and instances where it does not. Ranges may be expressed herein as from "about" one particular value, and / or to "about" another particular value. When such a range is expressed, an aspect includes from the one particular value and / or to the other particular value. Similarly, when values are expressed as approximations, by use of the antecedent "about," it will be understood that the particular value forms another aspect. It will be further understood that the endpoints of each of the ranges are significant both in relation to the other endpoint, and independently of the other endpoint.

[0062] Described herein are machine learning-based methods and systems for predicting a subject's medical risk based on multi-modal health data. Multi-modal health data can include, but is not limited to, clinical data, patient-generated health data (PGHD), patient-reported outcome (PRO) data, and imaging data. Benefits of using machine learning-based models include, but are not limited to, that machine learning methods better handle integration and analysis of complex data patterns such as the multi-modal health data described herein. In some implementations described herein, imaging data is integrated with PGHD and / or PRO data to predict medical risk. For example, this disclosure contemplates that integrating imaging data with PGHD and / or PRO data facilitates more accurate prediction of medical risk including detection of symptom changes as compared to predicting risk based on imaging data alone. This allows for just-in-time interventions. Optionally, in some implementations, imaging data is integrated with PGHD, PRO data, and electronic medical record (EMR) data to predict medical risk. In other implementations described herein, clinical data is integrated with imaging data, PGHD, and / or PRO data to predict medical risk. For example, this disclosure contemplates that integrating clinical data with imaging data, PGHD, and / or PRO data facilitates more accurate predictionof medical risk including detection of symptom changes as compared to predicting risk based on clinical data alone. Additionally, methods for training a machine learning model are also described herein.

[0063] In some implementations described herein, the medical risk is associated with a disease such as cancer. Cancer is a disease caused by uncontrolled division of abnormal cells, e.g., a malignant growth. The cancer may be a solid or hematologic malignancies. A solid tumor is an abnormal mass of hyperproliferative or neoplastic cells from a tissue other than blood, bone marrow, or the lymphatic system, which may be benign or cancerous. In general, the tumors described herein are cancerous. For example, the tumors in the examples described herein are in the subject's lungs (i.e., non-small cell lung cancer (NSCLC)). This disclosure contemplates that the systems and methods described herein may be applicable to solid tumors in other cells, tissue, or organ of the subject. For example, although lung cancer is used in the examples, this disclosure contemplates that the systems and methods described herein can be applied to other tumors. Additionally, this disclosure contemplates that the systems and methods described herein may be applicable to hematologic malignancies. As used herein, the terms "hyperproliferative" and "neoplastic" refer to cells having the capacity for autonomous growth, i.e., an abnormal state or condition characterized by rapidly proliferating cell growth. Hyperproliferative and neoplastic disease states may be categorized as pathologic, i.e., characterizing or constituting a disease state, or may be categorized as non-pathologic, i.e., a deviation from normal but not associated with a disease state. The term is meant to include all types of solid cancerous growths, metastatic tissues or malignantly transformed cells, tissues, or organs, irrespective of histopathologic type or stage of invasiveness. "Pathologic hyperproliferative" cells occur in disease states characterized by malignant tumor growth. Examples of non-pathologic hyperproliferative cells include proliferation of cells associated with wound repair. Examples of solid tumors are sarcomas, carcinomas, and lymphomas. Leukemias (cancers of the blood) generally do not form solid tumors.

[0064] The term "carcinoma" is art recognized and refers to malignancies of epithelial or endocrine tissues including respiratory system carcinomas, gastrointestinal system carcinomas, genitourinary system carcinomas, testicular carcinomas, breast carcinomas, prostatic carcinomas, endocrine system carcinomas, and melanomas. In some implementations, the disease is lung carcinoma, rectal carcinoma, colon carcinoma, esophageal carcinoma, prostate carcinoma, head and neck carcinoma, or melanoma. Exemplary carcinomas include those forming from tissue of the cervix, lung, prostate, breast, head and neck, colon and ovary. The term also includes carcinosarcomas, e.g., which include malignant tumors composed of carcinomatous and sarcomatous tissues. An "adenocarcinoma" refers to a carcinoma derived from glandular tissue or in which the tumor cells form recognizable glandular structures. The term "sarcoma" is art recognized and refers to malignant tumors of mesenchymal derivation. The term "hematologic malignancies" is art recognized and refers to cancers that affect the blood, bone marrow, and lymph nodes such as leukemia (acute lymphocytic (ALL), chronic lymphocytic (CLL), acute myeloid (AML), chronic myeloid (CML)), myeloma, and lymphoma (Hodgkin's and non-Hodgkin's (NHL)).

[0065] Referring now to Fig. 1, a block diagram illustrating a machine learning model 100 is shown. In Fig. 1, the machine learning model 100 is operating in inference mode. The machine learning model 100 has therefore been trained with a data set (or "dataset"). The machine learning model 100 in Fig. 1 is a supervised learning model. Supervised learning models are known in the art. A supervised learning model "learns" a function that maps an input 120 (also known as feature or features) to an output 140 (also known as target or targets) during training with a labeled data set. Machine learning model training is discussed in further detail below.

[0066] Optionally, the machine learning model 100 in Fig. 1 is an artificial neural network. An artificial neural network (ANN) is a computing system including a plurality of interconnected neurons (e.g., also referred to as "nodes"). This disclosure contemplates that the nodes can be implementedusing a computing device (e.g., a processing unit and memory as described herein). The nodes can optionally be arranged in a plurality of layers such as input layer, output layer, and one or more hidden layers. Each node is connected to one or more other nodes in the ANN. For example, each layer is made of a plurality of nodes, where each node is connected to all nodes in the previous layer. The nodes in a given layer are not interconnected with one another, i.e., the nodes in a given layer function independently of one another. As used herein, nodes in the input layer receive data (sometimes referred to as "features" or input 120) from outside of the ANN, nodes in the hidden layer(s) modify the data between the input and output layers, and nodes in the output layer provide the results (sometimes referred to as "target" or output 140). Each node is configured to receive an input, implement an activation function (e.g., binary step, linear, sigmoid, tanh, or rectified linear unit (ReLU) function), and provide an output in accordance with the activation function. Additionally, each node is associated with a respective weight. ANNs are trained with a data set to minimize the cost function, which is a measure of the ANN'S performance (e.g., error such as LI or L2 loss). The training algorithm tunes the node weights and / or bias to minimize the cost function. This disclosure contemplates that any algorithm that finds the minimum of the cost function can be used for training the ANN. Training algorithms for ANNs include, but are not limited to, backpropagation. It should be understood that an artificial neural network is provided only as an example machine learning model. This disclosure contemplates that the machine learning model 100 can be other types of models including, but not limited to, a logistic regression model, a logistic classifier model, a support vector machine, or an artificial neural network. Machine learning models are known in the art and are therefore not described in further detail herein.

[0067] The machine learning model 100 is trained to map the input 120 to the output 140. In the examples described herein, the input 120 is multi-modal health data, and the output 140 is a medical risk for the subject. The multi-modal health data includes one or more "features" that are inputinto the machine learning model 100, which predicts the medical risk for the subject. The medical risk for the subject is therefore the "target" of the machine learning model 100.

[0068] As shown in Fig. 1, the multi-modal health data (e.g., the input 120) includes patientgenerated health data (PGHD), patient-reported outcome (PRO) data, and imaging data. Optionally, the multi-modal health data (e.g., the input 120) includes PGHD, PRO data, clinical data, and imaging data. It should be understood that the multi-modal health data (e.g., the input 120) can optionally include data from other sources as well. This disclosure contemplates that the multi-modal health data can be stored in a data storage medium accessible by a computing device (e.g., computing device 300 of Fig. 3). Optionally, in some implementations, the multi-modal health data is stored in one or more databases (see Fig. 2).

[0069] Patient-generated health data includes, but is not limited to, sleep data, heart rate data, physical activity data, body temperature, weight, calorie intake, or macronutrient intake. It should be understood that the types of PGHD described herein are provided only as examples. This disclosure contemplates that PGHD can include other types of information. In some implementations, the patientgenerated health data is optionally recorded by one or more wearable sensors. For example, a fitness tracking device such as a FITBIT activity tracker from Fitbit, Inc. of San Francisco, CA is a wearable sensor that is capable of tracking PGHD such as steps, heart rate, weight, calories, sleep, etc. It should be understood that fitness tracking devices are provided only as an example wearable sensor. This disclosure contemplates recording PGHD with other wearable sensors.

[0070] Additionally, patient-reported outcome data includes, but is not limited to, a respective metric related to at least one of pain, fatigue, drowsiness, nausea, appetite, shortness of breath, depression, anxiety, constipation, diarrhea, sleep, distress, or suicidal ideation. It should be understood that the types of PRO data described herein are provided only as examples. This disclosure contemplates that PRO data can include other types of information.

[0071] Additionally, the imaging data can include information obtained from medical images such as computed tomography (CT) scans, positron emission tomography (PET) scans, or PET / CT scans. CT, PET, or PET / CT scans are commonly used to image tumors. Although CT, PET, and PET / CT scans are provided as examples, this disclosure contemplates that the imaging data can be obtained from other medical imaging modalities including, but not limited to, ultrasound and magnetic resonance imaging (MRI), single photon emission computed tomography (SPECT). The imaging data can include one or more imaging features, such as geometry features, intensity features, texture features, and deep features. It should be understood that medical images can be analyzed to extract one or more imaging features, which include, but are not limited to, quantitative features and / or radiomic features. Quantitative and radiomic features are related to the size, shape, texture, intensity, etc. of objects of interest (e.g., tumors or portions thereof) captured by the medical image.

[0072] Optionally, in some implementations, the multi-modal health data for the subject further includes electronic medical record (EMR) data. In other words, EMR data can be integrated with image data, PGHD, and PRO data. EMR data can include, but is not limited to, clinical data such as the subject's age, demographics, disease factors (e.g., diagnosis, time since diagnosis, stage, change in stage, tumor volume, mutation status, molecular markers), comorbidities, and previous medical history (e.g., other diseases / disorders, time since diagnosis of other diseases / disorders). It should be understood that the types of EMR data described herein are provided only as examples. This disclosure contemplates that EMR data can include other types of information.

[0073] The multi-modal health data is the input 120 ("features") to the machine learning model 100. Raw data are pre-processed by, for example, splitting physical activity data and sleep data reported as a range of minutes into standardized increments of time (e.g., one-minute or half-minute intervals), recalculating sleep efficiency (i.e., the time in bed that is spent asleep, which would exclude time spent falling asleep), and calculating calories per micronutrient. In some implementations, splittingraw physical activity data includes splitting raw physical activity data for a range of minutes (e.g., 12 minutes spent in "very active" physical activity) into one-minute increments. In some implementations, splitting raw sleep data includes splitting raw sleep data for a range of minutes in 0.5-minute increments (e.g., 11.5 minutes of "deep sleep") into 0.5-minute increments. Variants of neural networks, such as, for example, autoencoders for patient-reported outcomes data, can be created for each data type to prepare the data for presentation. These neural networks ensure the information in each data type is captured faithfully and will represent the data in a way that does not overwhelm the subsequent algorithm. Optionally, in some implementations, a plurality of features are selected from the multimodal health data. The selected features may be a subset of features contained in the multi-modal health data shown to have predictive value. In these implementations, the selected features are the input 120 ("features") to the machine learning model 100. A medical risk for the subject is then predicted, using the machine learning model 100, based on the multi-modal health data. In other words, the medical risk for the subject is the output 140 of the machine learning model 100. In some implementations, this disclosure contemplates that the machine learning model 100 outputs a probability (e.g., logistic regression). In other implementations, this disclosure contemplates that the machine learning model performs a classification (e.g., logistic regression classification). In the examples described herein, the medical risk for the subject can be related to an unplanned medical encounter (e.g., ER, urgent care, or outpatient visit); a change in medication (e.g., dose reduction, dose delay, or change in medication prescribed); an adverse reaction to a drug, pharmaceutical, substance, compound, or treatment; a development of a disease or condition; a relapse or recurrence of a disease or condition; or a death. It should be understood that the types of medical risks described herein are provided only as examples. This disclosure contemplates that medical risks can include other than those described herein, which can include, but are not limited to, dose reduction, dose delay, change in medication, or dose-limiting toxicity.

[0074] Optionally, in some implementations, a treatment is administered to the subject based upon the predicted medical risk. Optionally, the step of administering the treatment to the subject comprises changing a dosage amount or dosage schedule for a drug, pharmaceutical, substance, or compound; changing a type of a drug, pharmaceutical, substance, or compound; or changing an amount, frequency, or type of treatment.

[0075] Referring now to Fig. 2, an example method for training a machine learning model is shown. The method includes receiving multi-modal health data for each of a plurality of subjects. For example, multi-modal health data can include, but is not limited to, patient-generated health data 202, patient-reported outcome data 204, imaging data 206, and electronic medical record data 208. Example PGHD data 202, PRO data 204, imaging data 206, and EMR data 208 are described above and therefore not described in further detail below. Additionally, as described above, PGHD data 202, PRO data 204, imaging data 206, and EMR data 208 can be stored in a data storage medium accessible by a computing device (e.g., computing device 300 of Fig. 3), and optionally stored in one or more databases. For example, PGHD data 202 is stored in a PGHD database 212 and PRO data 204 is stored in a PRO database 214. Additionally, imaging data 206 can be obtained from medical images (e.g., CT, PET, and / or PET / CT scans) that are stored in a medical imaging system 216 such as a picture archiving and communication system (PACS), which is known in the art. As described herein, the imaging data 206 includes one or more imaging features such as quantitative and / or radiomic features. Additionally, EMR data 208, which includes clinical data, can be stored in an EMR database 218, which is known in the art. It should be understood that PGHD data 202, PRO data 204, imaging data 206, and EMR data 208 are provided only as examples of multi-modal health data. This disclosure contemplates that multi-modal health data can include data from other sources.

[0076] The method also includes processing, using a data integration machine learning model 220, the multi-modal health data to create a data set. Custom architecture designs can becreated for each data type, and various neural networks can be used to represent each data type. As noted above, the data set includes multi-modal health data for each of a plurality of subjects. It should be understood that machine learning models are trained to find patterns present in large data sets. The data set can be stored in a data storage medium accessible by a computing device (e.g., computing device 300 of Fig. 3), and optionally stored in one or more databases. For example, the data set is stored in a data repository 222. This disclosure contemplates that the data set is a labeled data set, where the multi-modal health data is tagged with "labels." The labels associate the multi-modal health data samples with the target or targets. The targets can include, but are not limited to, medical risks such as dose reduction, dose delay, change in medication, dose-limiting toxicity, unplanned clinical encounter, disease onset, disease reoccurrence or relapse, or death. It should be understood that the targets described herein are provided only as examples. This disclosure contemplates that targets can include other than those described herein.

[0077] The method further includes training a supervised learning model 240 with the data set. As described herein, a supervised learning model is trained with the data set to minimize the cost function. Cost functions include, but are not limited to, errors such as LI loss or L2 loss. During training, the supervised learning model's node weights and / or bias are tuned to minimize the cost function. This disclosure contemplates that any algorithm that finds the minimum of the cost function can be used for training the supervised learning model 240. At the conclusion of training, the trained machine learning model 260 is configured to predict a medical risk for a subject. In other words, the trained machine learning model 260 is capable of operating in inference mode such that a target (or targets) (e.g., output 140 shown in Fig. 1) can be predicted based on the input feature set (e.g., input 120 shown in Fig. 1).

[0078] It should be appreciated that the logical operations described herein with respect to the various figures may be implemented (1) as a sequence of computer implemented acts or program modules (i.e., software) running on a computing device (e.g., the computing device described in Fig. 3),(2) as interconnected machine logic circuits or circuit modules (i.e., hardware) within the computing device and / or (3) a combination of software and hardware of the computing device. Thus, the logical operations discussed herein are not limited to any specific combination of hardware and software. The implementation is a matter of choice dependent on the performance and other requirements of the computing device. Accordingly, the logical operations described herein are referred to variously as operations, structural devices, acts, or modules. These operations, structural devices, acts and modules may be implemented in software, in firmware, in special purpose digital logic, and any combination thereof. It should also be appreciated that more or fewer operations may be performed than shown in the figures and described herein. These operations may also be performed in a different order than those described herein.

[0079] Referring to Fig. 3, an example computing device 300 upon which the methods described herein may be implemented is illustrated. It should be understood that the example computing device 300 is only one example of a suitable computing environment upon which the methods described herein may be implemented. Optionally, the computing device 300 can be a well- known computing system including, but not limited to, personal computers, servers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, network personal computers (PCs), minicomputers, mainframe computers, embedded systems, and / or distributed computing environments including a plurality of any of the above systems or devices. Distributed computing environments enable remote computing devices, which are connected to a communication network or other data transmission medium, to perform various tasks. In the distributed computing environment, the program modules, applications, and other data may be stored on local and / or remote computer storage media.

[0080] In its most basic configuration, computing device 300 typically includes at least one processing unit 306 and system memory 304. Depending on the exact configuration and type of computing device, system memory 304 may be volatile (such as random access memory (RAM)), non-volatile (such as read-only memory (ROM), flash memory, etc.), or some combination of the two. This most basic configuration is illustrated in Fig. 3 by dashed line 302. The processing unit 306 may be a standard programmable processor that performs arithmetic and logic operations necessary for operation of the computing device 300. The computing device 300 may also include a bus or other communication mechanism for communicating information among various components of the computing device 300.

[0081] Computing device 300 may have additional features / functionality. For example, computing device 300 may include additional storage such as removable storage 308 and nonremovable storage 310 including, but not limited to, magnetic or optical disks or tapes. Computing device 300 may also contain network connection(s) 316 that allow the device to communicate with other devices. Computing device 300 may also have input device(s) 314 such as a keyboard, mouse, touch screen, etc. Output device(s) 312 such as a display, speakers, printer, etc. may also be included. The additional devices may be connected to the bus in order to facilitate communication of data among the components of the computing device 300. All these devices are well known in the art and need not be discussed at length here.

[0082] The processing unit 306 may be configured to execute program code encoded in tangible, computer-readable media. Tangible, computer-readable media refers to any media that is capable of providing data that causes the computing device 300 (i.e., a machine) to operate in a particular fashion. Various computer-readable media may be utilized to provide instructions to the processing unit 306 for execution. Example tangible, computer-readable media may include, but is not limited to, volatile media, non-volatile media, removable media and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. System memory 304, removable storage 308, and nonremovable storage 310 are all examples of tangible, computer storage media. Example tangible,computer-readable recording media include, but are not limited to, an integrated circuit (e.g., field- programmable gate array or application-specific IC), a hard disk, an optical disk, a magneto-optical disk, a floppy disk, a magnetic tape, a holographic storage medium, a solid-state device, RAM, ROM, electrically erasable program read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices.

[0083] In an example implementation, the processing unit 306 may execute program code stored in the system memory 304. For example, the bus may carry data to the system memory 304, from which the processing unit 306 receives and executes instructions. The data received by the system memory 304 may optionally be stored on the removable storage 308 or the non-removable storage 310 before or after execution by the processing unit 306.

[0084] It should be understood that the various techniques described herein may be implemented in connection with hardware or software or, where appropriate, with a combination thereof. Thus, the methods and apparatuses of the presently disclosed subject matter, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium wherein, when the program code is loaded into and executed by a machine, such as a computing device, the machine becomes an apparatus for practicing the presently disclosed subject matter. In the case of program code execution on programmable computers, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. One or more programs may implement or utilize the processes described in connection with the presently disclosed subject matter, e.g., through the use of an application programming interface (API), reusable controls, or the like. Such programs may be implemented in a high level procedural or object-oriented programminglanguage to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language and it may be combined with hardware implementations.

[0085] Examples

[0086] The following examples are put forth so as to provide those of ordinary skill in the art with a complete disclosure and description of how the compounds, compositions, articles, devices and / or methods claimed herein are made and evaluated and are intended to be purely exemplary and are not intended to limit the disclosure. Efforts have been made to ensure accuracy with respect to numbers (e.g., amounts, temperature, etc.), but some errors and deviations should be accounted for. Unless indicated otherwise, parts are parts by weight, temperature is in °C or is at ambient temperature, and pressure is at or near atmospheric.

[0087] Example 1

[0088] Immunotherapy has prolonged survival1 5and become standard of care7for some patients with non-small cell lung cancer (NSCLC). For example, immune checkpoint inhibitors such as durvalumab are commonly prescribed for patients with NSCLC.8 10As the use of immune checkpoint inhibitors increases, there is urgent need for interventions that monitor immunotherapy-related symptoms, such as treatment induced lung inflammation (i.e., pneumonitis).11 13

[0089] Immune-related symptoms, such as pneumonitis, can negatively affect treatment and survival.14 18The PACIFIC trial found that pneumonitis was the most common reason for treatment discontinuation among NSCLC patients receiving durvalumab.6Real-word evidence suggests pneumonitis may occur in up to 18% of patients receiving immunotherapy.1617 19 23

[0090] Studies have identified potential risk factors of immunotherapy-related pneumonitis including clinical and treatment characteristics and radiomic features from CT and PET images. Patients with comorbidities and worse performance status may be at greater risk for pneumonitis.16-21There ismixed evidence on whether treatment regimen (e.g., combined chemotherapy and immunotherapy) and prior radiotherapy affects immunotherapy-related pneumonitis.16-17-23Findings from a recent pilot (n=32) suggest radiomic texture features (e.g., skewness and angular variance of sum of squares) from immunotherapy patients' CT scans were predictive of pneumonitis.24Although CT is provided as an example, PET images or PET / CT images may be used for predictions. While radiomics holds promise for predicting pneumonitis, this disclosure contemplates that there is value in integrating imaging data with other data, such as patient-generated health data (PGHD) (e.g., data collected with wearable sensors), clinical data, and patient-reported outcomes (PROs), to detect symptom changes and allow for just-in- time interventions.

[0091] Pneumonitis is associated not only with distinct radiomic features14but also unique symptoms that may be detectable through PROs and wearable sensors. Pneumonitis is associated with symptoms, such as coughing, wheezing, shortness of breath, and loss of appetite that have been captured through PROs and used to predict other treatment complications (e.g., post-surgery recovery).25-26Additionally, physical activity patterns (e.g., PGHD) from wearable devices have been correlated with patient-reported fatigue,27-29a pneumonitis symptom. Wearable devices may also detect sleep disturbance from coughing. While there may be value in leveraging diverse data sources, we are unaware of any studies using multi-modal data (i.e., imaging, wearables, PROs) to detect treatment- related toxicities or disease progression.

[0092] We addresses this gap by collecting data among patients receiving systemic therapy, including immune checkpoint inhibitors, for NSCLC. Patients are followed for 60 days after initiating therapy. PET / CT scan data (e.g., baseline, follow-up at 6-8 weeks) are obtained, clinical data are abstracted from electronic records, patient-reported outcomes are assessed, and daily biometrics (e.g., physical activity, sleep, resting heart rate) are captured via wrist-worn sensor. Machine learning algorithms are developed using pre-treatment data and data gathered during treatment to predict riskand severity of toxicities (e.g., pneumonitis), disease progression, or other unplanned medical encounters.

[0093] For example, a method for predicting treatment-related symptoms in a subject includes training a supervised learning model to predict a subject's health risk. The method also includes receiving multi-modal health data for the subject. The multi-modal health data for the subject includes patient-generated health data, patient-reported outcome data, and imaging data. As described herein, patient-generated health data is recorded by one or more wearable sensors. Alternatively or additionally, the patient-generated health data includes at least one of sleep data, heart rate data, or physical activity data. The patient-reported outcome data includes a respective metric related to at least one of wheezing, pain, fatigue, drowsiness, nausea, appetite, shortness of breath, depression, anxiety, constipation, diarrhea, sleep, distress, or suicidal ideation. The imaging data includes one or more CT scan, PET scan, or PET / CT scan imaging features. The multi-modal health data can be collected periodically (e.g., daily, weekly, monthly, or at other time interval).

[0094] The method also includes inputting the multi-modal health data into a trained machine learning model, and predicting, using the trained machine learning model, a risk of immunotherapy-induced lung inflammation in the subject. Optionally, in some implementations, the method further includes predicting, using the trained machine learning model, a severity of the immunotherapy-induced lung inflammation in the subject.

[0095] In some implementations, the multi-modal health data for the subject includes pretreatment data only. In these implementations, the machine learning model predicts pneumonitis risk and severity based on pre-treatment data (e.g., baseline PET / CT scan, clinical characteristics). In other implementations, the multi-modal health data for the subject includes post-treatment data. Optionally, the multi-modal health data for the subject further includes pre-treatment data. In these implementations, the machine learning model predicts pneumonitis risk and severity based onlongitudinal data gathered during treatment (e.g., CT scans, daily biometrics, weekly PROs during the first 6 months of treatment).

[0096] Example 2

[0097] Patients receiving systemic therapy (ST) for non-small cell lung cancer (NSCLC) experience high rates of toxicities with significant impact on patient outcomes. This study aimed to test a platform for prospectively collecting patient-reported outcome (PRO) data, wearable sensor data (WSD), and clinical data and to develop a machine learning algorithm to predict healthcare utilization, specifically urgent care visits.

[0098] NSCLC patients completed the PROMIS-57 PRO measure and began wearing a Fitbit to monitor patient-generated health data (PGHD) before or on the day they started ST through day 60. Demographic and clinical data were abstracted from the medical record during this period. Bayesian Networks (BNs) were used to develop predictive models for urgent care visits.

[0099] Patients (N=58) were on mostly female (57%), White (88%), and non-Hispanic (95%) patients with adenocarcinoma (69%) and an average age of 69 years old (range 35-89). Initial BN models trained on demographic and clinical data demonstrated moderate predictive accuracy for urgent care visits before ST (AUC=0.72, 95% Cl: 0.57-0.80) and during ST (AUC=0.81, 95% Cl: 0.63-0.89). Incorporating PRO and WSD yielded enhanced models with significantly improved performance (final AUC=0.86, 95% Cl: 0.76-0.95).

[0100] Multidimensional data sources, including demographic, clinical, PRO, and WSD can enhance ML predictive models to elucidate complex and interactive factors influencing NSCLC patient outcomes and healthcare utilization during the first 60 days of ST. Use of ML to predict and prevent treatment toxicities and healthcare utilization holds potential for substantially improving patient outcomes and enhance the quality of cancer care delivery.

[0101] Introduction

[0102] Patients receiving systemic therapy (ST) for non-small cell lung cancer (NSCLC) are at high risk of toxicities that can lead to urgent care or emergency department visits.1However, patient symptoms have been historically under-detected by clinicians,2and the field of predicting toxicities in oncology patients remains nascent. Improving oncologists' ability to detect, predict, and prevent patient toxicities could significantly improve patient outcomes in this population and help reduce healthcare system burden.

[0103] Patient-reported outcome (PRO) measures are increasingly being to monitor risk of toxicities and improve clinical outcomes by asking patients to self-report symptoms.3Indeed, the FDA now recognizes the critical role of PROs in clinical trials,4and clinical practice guidelines recommend use of PROs as an evidence-based approach to monitoring symptoms from the start of active treatment.5In a 2017 randomized controlled trial of 766 patients receiving chemotherapy, routine PRO monitoring at each clinic visit with nurse referral for elevated toxicities was associated with increased overall survival relative to usual care.6A 2022 machine learning study showed that augmenting electronic health record data with PROs improved prediction modeling for 180-day mortality among outpatients with any cancer type.7Notably, cancer-directed treatment was not required for study participation, and the role of treatment-related toxicities remains unknown. Beyond clinical outcomes, PROs have also been shown to predict healthcare utilization (e.g., emergency department visits and unplanned hospitalizations) among cancer survivors in longitudinal population-based research.8

[0104] Beyond PROs, wearable sensor data (WSD), such as daily physical activity, sleep patterns, and heart rate, represent another type of patient-generated health data (PGHD) now utilized for remote symptom monitoring ,9 10Building upon evidence of feasibility and acceptability of longitudinal WSD collection among cancer patients,11studies are increasingly investigating associations between WSD and clinical outcomes in oncology. A 2023 systematic review and meta-analysis of 34 oncology studies found moderate-to-strong evidence for associations between circadian rest-activitymetrics and overall survival.12However, examination of other WSD and patient outcome associations were hampered by methodological heterogeneity across studies, supporting the need for further research in this developing field.

[0105] While PGHD show promise for improving management of existing and emerging symptoms that are noticeable to patients, the next critical step to improving patient outcomes is to predict and prevent toxicities.9Limited research has focused on elucidating mechanisms to predict and prevent toxicities in cancer patients receiving ST, particularly those with NSCLC. Traditional predictive models often struggle to capture dynamic interactions among treatment modalities, patient characteristics, and disease progression. Moving beyond traditional models and integrating more diverse sources of data inclusive of PGHD along with clinical and biomarker-based predictors could substantially enhance effectiveness of predictive modeling.13

[0106] Machine learning (ML) is a versatile computational approach that offers a promising solution for synthesizing information from disparate data sources that conventional analytical methods may overlook.13Our team has already used ML to develop models predicting tumor local control and radiation-induced toxicities among NSCLC patients with satisfactory accuracy and interpretability.14The next step is to incorporate PGHD in this predictive modeling. By further leveraging ML algorithms, the collective power of PGHD, clinical data, biomarker profiles, and other data sources can be used to develop robust predictive models tailored to the complex interplay of factors influencing patient outcomes and healthcare utilization. Thus, this study aimed to (1) test a new platform for collecting and integrating PGHD among patients receiving ST for NSCLC, and (2) develop a ML algorithm to predict healthcare utilization, namely urgent care or emergency department visits, in these patients.

[0107] Materials and Methods

[0108] Patients treated at Moffitt Cancer Center (MCC) were recruited and enrolled from March 2021 to March 2022. The study protocol was approved by MCC's Institutional Review Board(Pro00048158). Eligible patients: 1) were > 18 years old; 2) were expected to begin ST (defined as chemotherapy, immunotherapy, chemo-immunotherapy, and / or targeted therapy) for NSCLC at MCC; c) had an Eastern Cooperative Oncology Group (ECOG) performance status of 0 to 2; d) had no documented or observable severe and untreated neurological, psychiatric, or physical limitations that would preclude participation in this study; and e) were capable of providing informed consent. No incentives were provided to participate.

[0109] Patients were enrolled shortly before initiating ST (pre-ST) and were subsequently monitored for 60 days after starting ST (during-ST). At pre-ST baseline and at the 60-day follow-up assessment, patients completed the Patient-Reported Outcomes Measurement Information System 57-item Profile (PROMIS-57 v2.1), which measures patient quality of life and well-being in the domains of Depression, Anxiety, Physical Function, Pain Interference, Fatigue, Sleep Disturbance, Ability to Participate in Social Roles and Activities along with a single-item question of Global Pain Intensity. During the course of the study period, patients wore a Fitbit device to collect WSD, (i.e., daily physical activity, sleep patterns, heart rate), and the Collaborative Data Services Core at MCC abstracted medical record data, including demographic and clinical variables (e.g., albumin, calcium) and urgent care utilization. Changes in albumin and calcium levels and changes in WSD from pre-ST to instance of an urgent care visit were calculated. Detailed information on all available measures of demographic, clinical, PRO, and WSD before and during ST are presented in Table 1 (Fig. 4), which shows NSCLC patients' available clinical, demographic data and additional patient-reported outcome (PRO) and wearable sensor data (WSD) before and during systemic therapy (ST).

[0110] We used the generative ML approach of Bayesian Networks (BNs), which has been used extensively in previous studies, including our own prior research with NSCLC patients.14BNs utilize a directed acyclic graph (DAG) to represent conditional independence relationships among variables, where nodes depict symptoms and edges signify causal effects. In the current study, weT1adopted a simplified human-in-the-loop BN approach from our previous work15to involve a human expert in the model building process and developed a predictive model for urgent care visits, as shown in Fig. 5, which is a flow chart illustrating a process to obtain a Bayesian Network (BN) based on a simplified Human-in-the-Loop BN approach.

[0111] This strategy integrates the rank of features based on unexplainable ML approaches (e.g., Random forests, Gradient Boosting Machines) with selected features from explainable ML models (e.g., BNs) to improve performance of prediction models. Given a rank list of features (with the most important feature at its top) that can be evenly divided into I intervals, i represents the index of these intervals starting from the top of the list ( / = 0, 1, 2, 3, ..., / ). First, features in the top rank interval are combined with features from the Markov blanket of "urgent care visit" to build the initial BN0and obtain its performance Area Under the Receiver Operating Characteristic Curve (AUC0), as shown in Fig. 5. Next, features in BN0are extracted and combined with features in the second rank interval to develop a new BNt. If the new model performs better than the previous BN model, features in the new BNrare combined with features in the next rank interval to develop the next BN (BN2) and so on; otherwise, the algorithm stops with previous BN as final prediction model. Following this BN development process, this approach identifies features that are highly correlated with "urgent care visit" and crucial for its prediction, while also constructing transparent biophysical pathways to illustrate relationships.

[0112] In the current study, the first step was to train a pre-treatment BN to predict the risk of urgent care visit using only demographic and clinical data available at pre-ST. The second step was to train a second BN incorporating demographic and clinical data collected during ST. In the third step, enhancement was achieved by training a pre-treatment BN that incorporated pre-ST PRO data.Finally, additional enhancement was achieved by including clinical data and WSD assessed during ST. Weevaluated the predictive performance of each BN using cross-validation and the AUC during the training phase of our study.

[0113] Results

[0114] Table 2 (Fig. 6) presents demographic and clinical characteristics of N=58 participants with NSCLC treated with ST at MCC who were included in the analysis. The majority were female (57%), White (88%), and non-Hispanic (95%) with an average age of 69 years old (range 35-89). Adenocarcinoma was the predominant diagnosis (69%).

[0115] BN prediction model training followed the step-wise approach described in Materials and Methods. The first step was to train a pre-treatment BN to predict the risk of urgent care visit among NSCLC patients using only demographic and clinical data available at pre-ST. The resultant BN prediction model included age at pre-ST, gender, marital status, cancer stage, BMI, adenocarcinoma diagnosis (yes / no), pre-ST albumin, and pre-ST calcium with urgent care visit prediction performance of AUC=0.72, 95% Cl: 0.57-0.80.

[0116] The second step was to train a second BN incorporating demographic and clinical data collected during ST. The resultant BN prediction model included age, gender, pre-ST albumin, pre-ST calcium, changes in albumin and calcium from pre-ST to urgent care visit, and receipt of targeted therapy, chemotherapy, and / or radiation therapy during the 60-day follow-up period with urgent care visit prediction performance of AUC=0.81, 95% Cl: 0.63-0.89.

[0117] In the third step, enhancement was achieved by training a pre-treatment BN that incorporated pre-ST PRO data. The resultant BN prediction model included the following pre-ST variables: age, BMI, Karnofsky performance status, PROMIS Global Pain, and PROMIS Pain Interference with urgent care visit prediction performance of AUC=0.78, 95% Cl: 0.66-0.88.

[0118] Finally, additional enhancement was achieved by including clinical data and WSD assessed during ST. Fig. 7 illustrates the final BN prediction model and its prediction performancewith 95% CL DeLong's test revealed that enhanced pre-ST and during-ST BNs significantly outperformed BNs with only pre-ST (p =0.002) and during-ST (p < 0.001) clinical and demographic data respectively.

[0119] Note that the model of Fig. 7 was based on clinical and demographic data collected before and during systemic therapy (ST), pre-ST patient-reported outcome data, and change in wearable sensor data (WSD) from pre-ST up to urgent care visit. Green or red arcs between two nodes denote positive or negative associations, while gray arcs indicate mixed associations.

[0120] Age = age at baseline (pre-ST); Calcium_0 = pre-ST calcium; albumin_during = change in albumin from pre-ST to urgent care visit, calcium_during = change in calcium from pre-ST to urgent care visit. "RSLP" indicates the change of wearable device data before NSCLC patients' urgent visit during the ST while "BT" denotes before initiating ST. Thus, SLP_HeartRate = change in heart rate from pre-ST to urgent care visit; RSLP SIeepEfficiency = change in sleep efficiency from pre-ST to urgent care visit. Finally, During_Targeted, During_Chemo, and During_Rad indicate whether patients received targeted therapy, chemotherapy, or radiation therapy during the 60-day follow-up period.

[0121] Discussion

[0122] Study findings shed light on the utility of BN models for predicting urgent care visits among NSCLC patients undergoing ST. Our analysis included NSCLC patients with demographic characteristics aligning with broader trends observed in NSCLC patient populations and reflecting the demographic distribution commonly encountered in clinical practice. This representation suggests that study results may be generalizable for other NSCLC patients.

[0123] By leveraging a diverse array of demographic, clinical, PRO, and WSD, our models exhibited promising predictive performance and provide critical insights into the complex interplay of factors influencing NSCLC patient outcomes and healthcare utilization during the first 60 days of ST. Our initial models, trained solely on demographic and clinical data, demonstrated moderatepredictive accuracy for urgent care visits before and during treatment. Subsequent incorporation of PRO and WSD yielded enhanced models with significantly improved performance.

[0124] Results underscore the importance of integrating multidimensional data sources to enhance predictive modeling capabilities and refine risk stratification strategies.

[0125] Several study limitations warrant consideration. Generalizability of study findings is limited by the relatively small sample size and single-center design. Available study measures do not represent an exhaustive list of all possible predictors of urgent care visits, and important predictors may consequently be missing from our models.

[0126] The BN models developed in this study represent a novel approach to data visualization and predictive modeling in oncology, particularly in the context of healthcare utilization, such as urgent care visits, among NSCLC patients undergoing ST. By harnessing multidimensional data sources and leveraging advanced analytical techniques, BNs offer a pathway towards personalized risk assessment and targeted intervention strategies in the era of precision medicine. Application of such ML tools could ultimately help improve patient outcomes and enhance the quality of cancer care delivery.

[0127] References - Example 1

[0128] 1. Antonia SJ, Borghaei H, Ramalingam SS, et al. Four-year survival with nivolumab in patients with previously treated advanced non-small-cell lung cancer: a pooled analysis. Lancet Oncol. 2019;20(10):1395-1408.

[0129] 2. Garon EB, Hellmann MD, Rizvi NA, et al. Five-Year Overall Survival forPatients With Advanced Non-Small-Cell Lung Cancer Treated With Pembrolizumab: Results From the Phase I KEYNOTE-OOl Study. J Clin Oncol. 2019;37(28):2518-2527.

[0130] 3. Leighl NB, Hellmann MD, Hui R, et aL Pembrolizumab in patients with advanced non-small-cell lung cancer (KEYNOTE-OOl): 3-year results from an open-label, phase 1 study.Lancet Respir Med. 2019;7(4):347-357.

[0131] 4. Reck M, Rodriguez-Abreu D, Robinson AG, et al. Updated Analysis ofKEYNOTE-024: Pembrolizumab Versus Platinum-Based Chemotherapy for Advanced Non-Small-Cell Lung Cancer With PD-L1 Tumor Proportion Score of 50% or Greater. J Clin Oncol. 2019;37(7):537-546.

[0132] 5. Herbst RS, Baas P, Kim DW, et al. Pembrolizumab versus docetaxel for previously treated, PD-Ll-positive, advanced non-small-cell lung cancer (KEYNOTE-010): a randomised controlled trial. Lancet. 2016;387(10027):1540-1550.

[0133] 6. Antonia SJ, Villegas A, Daniel D, et al. Durvalumab afterChemoradiotherapy in Stage III Non-Small-Cell Lung Cancer. N Engl J Med. 2017;377(20):1919-1929.

[0134] 7. Ettinger DS, Wood DE, Aggarwal C, et al. NCCN Guidelines Insights: Non¬Small Cell Lung Cancer, Version 1.2020. J Natl Compr Cane Netw. 2019;17(12):1464-1472.

[0135] 8. NIH. Durvalumab vs Placebo With Stereotactic Body Radiation Therapy in Early Stage Unresected Non-small Cell Lung Cancer Patients ( PACIFIC-4 ). 2020; https: / / clinicaltrials.gov / ct2 / show / NCT03833154?term=PACIFIC+4&draw=2&rank=l. Accessed April 12, 2021.

[0136] 9. NIH. Testing the Addition of the Drug Atezolizumab to the UsualRadiation Treatment for Patients With Early Non-small Cell Lung Cancer. 2021; https: / / clinicaltrials.gov / ct2 / show / study / NCT04214262?term=atezolizumab&cond=Lung+Cancer+Stage +ll&phase=2&draw=2&rank=3#contacts. Accessed April 12, 2021.

[0137] 10. NIH. Efficacy and Safety Study of Stereotactic Body Radiotherapy (SBRT)With or Without Pembrolizumab (MK-3475) in Adults With Unresected Stage I or IIA Non-Small Cell Lung Cancer (NSCLC) (MK-3475-867 / KEYNOTE-867 ). 2021; https: / / clinicaltrials.gov / ct2 / show / NCT03924869?term=Keynote-867&draw=2&rank=l. Accessed April12, 2021.

[0138] 11. Hall ET, Singhal S, Dickerson J, et al. Patient-Reported Outcomes forCancer Patients Receiving Checkpoint Inhibitors: Opportunities for Palliative Care-A Systematic Review. J Pain Symptom Manage. 2019;58(l):137-156.el31.

[0139] 12. Steffen McLouth LE, Lycan TW, Levine BJ, et al. Patient-ReportedOutcomes From Patients Receiving Immunotherapy or Chemoimmunotherapy for Metastatic Non-Small- Cell Lung Cancer in Clinical Practice. Clin Lung Cancer. 2020;21(3):255-263.e254.

[0140] 13. Safa H, Tamil M, Spiess PE, et al. Patient Reported Outcomes in ClinicalTrials Leading to Cancer Immunotherapy Drug Approvals from 2011 to 2018: A Systematic Review. J Natl Cancer Inst. 2020.

[0141] 14. Zhong L, Altan M, Shannon VR, Sheshadri A. Immune-Related AdverseEvents: Pneumonitis. Adv Exp Med Biol. 2020;1244:255-269.

[0142] 15. Naidoo J, Nishino M, Patel SP, et al. Immune-Related Pneumonitis AfterChemoradiotherapy and Subsequent Immune Checkpoint Blockade in Unresectable Stage III Non-Small- Cell Lung Cancer. Clin Lung Cancer. 2020;21(5):e435-e444.

[0143] 16. Cho JY, Kim J, Lee JS, et al. Characteristics, incidence, and risk factors of immune checkpoint inhibitor-related pneumonitis in patients with non-small cell lung cancer. Lung Cancer. 2018;125:150-156.

[0144] 17. Suresh K, Voong KR, Shankar B, et al. Pneumonitis in Non-Small CellLung Cancer Patients Receiving Immune Checkpoint Immunotherapy: Incidence and Risk Factors. J Thorac Oncol. 2018;13(12):1930-1939.

[0145] 18. Suresh K, Psoter KJ, Voong KR, et al. Impact of Checkpoint InhibitorPneumonitis on Survival in NSCLC Patients Receiving Immune Checkpoint Immunotherapy. J ThoracOncol. 2019;14(3):494-502.

[0146] 19. Khunger M, Rakshit S, Pasupuleti V, et al. Incidence of Pneumonitis WithUse of Programmed Death 1 and Programmed Death-Ligand 1 Inhibitors in Non-Small Cell Lung Cancer: A Systematic Review and Meta-Analysis of Trials. Chest. 2017;152(2):271-281.

[0147] 20. Nishino M, Giobbie-Hurder A, Hatabu H, Ramaiya NH, Hodi FS. Incidence of Programmed Cell Death 1 Inhibitor-Related Pneumonitis in Patients With Advanced Cancer: A Systematic Review and Meta-analysis. JAMA Oncol. 2016;2(12):1607-1616.

[0148] 21. Shankar B, Zhang J, Naqash AR, et al. Multisystem Immune-RelatedAdverse Events Associated With Immune Checkpoint Inhibitors for Treatment of Non-Small Cell Lung Cancer. JAMA Oncol. 2020;6(12):1952-1956.

[0149] 22. Fukihara J, Sakamoto K, Koyama J, et al. Prognostic Impact and RiskFactors of Immune-Related Pneumonitis in Patients With Non-Small-Cell Lung Cancer Who Received Programmed Death 1 Inhibitors. Clin Lung Cancer. 2019;20(6):442-450.e444.

[0150] 23. Voong KR, Hazell SZ, Fu W, et al. Relationship Between PriorRadiotherapy and Checkpoint-Inhibitor Pneumonitis in Patients With Advanced Non-Small-Cell Lung Cancer. Clin Lung Cancer. 2019;20(4):e470-e479.

[0151] 24. Colen RR, Fujii T, Bilen MA, et al. Radiomics to predict immunotherapy- induced pneumonitis: proof of concept. Invest New Drugs. 2018;36(4):601-607.

[0152] 25. Fagundes CP, Shi Q, Vaporciyan AA, et al. Symptom recovery after thoracic surgery: Measuring patient-reported outcomes with the MD Anderson Symptom Inventory. J Thorac Cardiovasc Surg. 2015;150(3):613-619.e612.

[0153] 26. Bouazza YB, Chiairi I, El Kharbouchi O, et aL Patient-reported outcome measures (PROMs) in the management of lung cancer: A systematic review. Lung Cancer. 2017;113:140-151.

[0154] 27. Berger AM. Patterns of fatigue and activity and rest during adjuvant breast cancer chemotherapy. Oncol Nurs Forum. 1998;25(l):51-62.

[0155] 28. Timmerman JG, Dekker-van Weering MG, Tonis TM, Hermens HJ,Vollenbroek-Hutten MM. Relationship between patterns of daily physical activity and fatigue in cancer survivors. Eur J Oncol Nurs. 2015;19(2):162-168.

[0156] 29. Jim HS, Small B, Faul LA, Franzen J, Apte S, Jacobsen PB. Fatigue, depression, sleep, and activity during chemotherapy: daily and intraday variation and relationships among symptom changes. Ann Behav Med. 2011;42(3):321-333.

[0157] 30. Manz CR, Parikh RB, Evans CN, et al. Integrating machine-generated mortality estimates and behavioral nudges to promote serious illness conversations for cancer patients: Design and methods for a stepped-wedge cluster randomized controlled trial. Contemp Clin Trials. 2020;90:105951.

[0158] 31. Manz CR, Parikh RB, Small DS, et al. Effect of Integrating MachineLearning Mortality Estimates With Behavioral Nudges to Clinicians on Serious Illness Conversations Among Patients With Cancer: A Stepped-Wedge Cluster Randomized Clinical Trial. JAMA Oncol. 2020:e204759.

[0159] 32. ACS. Key statistics for lung cancer. 2021; https: / / www.cancer.org / cancer / lung-cancer / about / key-statistics.html. Accessed February 25, 2021.

[0160] 33. Haslam A, Prasad V. Estimation of the Percentage of US Patients WithCancer Who Are Eligible for and Respond to Checkpoint Inhibitor Immunotherapy Drugs. JAMA Netw Open. 2019;2(5):el92535.

[0161] References - Example 2

[0162] 1. Miller M, Hanna N. Advances in systemic therapy for non-small cell lung cancer. Bmj. 2021;375

[0163] 2. Di Maio M, Gallo C, Leighl NB, et al. Symptomatic toxicities experienced during anticancer treatment: agreement between patient and physician reporting in three randomized trials. Journal of Clinical oncology. 2015;33(8):910-915.

[0164] 3. Basch E. Patient-reported outcomes-harnessing patients' voices to improve clinical care. New England Journal of Medicine. 2017;376(2):105-108.

[0165] 4. Food U, Administration D. Value and use of patient-reported outcomes(PROs) in assessing effects of medical devices. 2020.

[0166] 5. Di Maio M, Basch E, Denis F, et al. The role of patient-reported outcome measures in the continuum of cancer clinical care: ESMO Clinical Practice Guideline. Annals of Oncology. 2022;33(9):878-892.

[0167] 6. Basch E, Deal AM, Dueck AC, et al. Overall Survival Results of a TrialAssessing Patient-Reported Outcomes for Symptom Monitoring During Routine Cancer Treatment. Jama-J Am Med Assoc. Jul 11 2017;318(2):197-198. doi:10.1001 / jama.2017.7156

[0168] 7. Parikh RB, Hasler JS, Zhang Y, et al. Development of machine learning algorithms incorporating electronic health record data, patient-reported outcomes, or both to predict mortality for outpatients with cancer. JCO Clinical Cancer Informatics. 2022;6:e2200073.

[0169] 8. Noel CW, Sutradhar R, Zhao H, et al. Patient-reported symptom burden as a predictor of emergency department use and unplanned hospitalization in head and neck cancer: a longitudinal population-based study. Journal of Clinical Oncology. 2021;39(6):675-684.

[0170] 9. Jim HS, Hoogland Al, Brownstein NC, et al. Innovations in research and clinical care using patient-generated health data. CA: a cancer journal for clinicians. 2020;70(3):182-199.

[0171] 10. Cox SM, Lane A, Volchenboum SL. Use of wearable, mobile, and sensor technology in cancer clinical trials. JCO clinical cancer informatics. 2018;2:1-11.

[0172] 11. Oswald LB, Li X, Carvajal R, et aL Longitudinal collection of patient- reported outcomes and activity data during CAR-T therapy: feasibility, acceptability, and data visualization. Cancers. 2022;14(ll):2742.

[0173] 12. Kos M, Brouwer CG, van Laarhoven HW, Hopman MT, van Oijen MG,Buffart LM. The association between wearable device metrics and clinical outcomes in oncology: A systematic review with evidence synthesis and meta-analysis. Critical Reviews in Oncology / Hematology. 2023:103979.

[0174] 13. Wei L, Niraula D, Gates ED, et al. Artificial intelligence (Al) and machine learning (ML) in precision oncology: a review on enhancing discoverability through multiomics integration. The British Journal of Radiology. 2023;96(1150):20230211.

[0175] 14. Luo Y, Jolly S, Palma D, et al. A situational awareness Bayesian network approach for accurate and credible personalized adaptive radiotherapy outcomes prediction in lung cancer patients. Physica Medica. 2021;87:11-23.

[0176] 15. Luo Y, Cuneo KC, Lawrence TS, et aL A human-in-the-loop basedBayesian network approach to improve imbalanced radiation outcomes prediction for hepatocellular cancer patients with stereotactic body radiotherapy. Frontiers in Oncology. 2022;12:1061024.

[0177] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

WHAT IS CLAIMED:

1. A method for predicting medical risk, the method comprising: receiving multi-modal health data for a subject; inputting the multi-modal health data into a trained machine learning model; and predicting, using the trained machine learning model, a medical risk for the subject, wherein the multi-modal health data for the subject comprises patient-generated health data (PGHD), patient- reported outcome (PRO) data, and medical imaging data.

2. The method of claim 1, wherein the multi-modal health data for the subject further comprises electronic medical record (EMR) data.

3. The method of any one of claim 1 or 2, wherein the multi-modal health data for the subject further comprises clinical data.

4. The method of any one of claims 1-3, wherein the patient-generated health data is recorded by one or more wearable sensors.

5. The method of any one of claims 1-4, wherein the patient-generated health data comprises at least one of sleep data, heart rate data, physical activity data, body temperature, weight, calorie intake data, or macronutrient intake data.

6. The method of any one of claims 1-5, wherein the PRO data comprises a respective metric related to at least one of pain, fatigue, drowsiness, nausea, appetite, shortness of breath, depression, anxiety, constipation, diarrhea, sleep, distress, or suicidal ideation.

7. The method of any one of claims 1-6, wherein the medical imaging data comprises one or more imaging features.

8. The method of any one of claims 1-7, wherein the medical risk for the subject is related to an unplanned medical encounter; a change in medication; an adverse reaction to a drug, pharmaceutical, substance, compound, or treatment; a development of a disease or condition; a relapse or recurrence of a disease or condition; or a death.

9. The method of claim 8, further comprising administering a treatment to the subject based upon the predicted medical risk for the subject.

10. The method of claim 9, wherein administering the treatment to the subject comprises changing a dosage amount or dosage schedule for a drug, pharmaceutical, substance, or compound; changing a type of a drug, pharmaceutical, substance, or compound; or changing an amount, frequency, or type of treatment.

11. The method of any one of claims 1-10, further comprising selecting a plurality of features from the multi-modal health data, wherein the selected features are input into the trained machine learning model.

12. The method of any one of claims 1-11, wherein the trained machine learning model is a supervised learning model.

13. The method of claim 12, wherein the trained machine learning model is a logistic regression model, a logistic classifier model, a support vector machine, or an artificial neural network.

14. A system for predicting medical risk, the system comprising: a computing device comprising a processor and a memory, the memory having computerexecutable instructions stored thereon that, when executed by the processor, cause the processor to: receive multi-modal health data for a subject; input the multi-modal health data into a trained machine learning model; and receive a medical risk for the subject, the medical risk being predicted by the trained machine learning model, wherein the multi-modal health data for the subject comprises patient-generated health data (PGHD), patient-reported outcome (PRO) data, and medical imaging data.

15. The system of claim 14, wherein the multi-modal health data for the subject further comprises electronic medical record (EMR) data.

16. The system of any one of claim 14 or 15, wherein the multi-modal health data for the subject further comprises clinical data.

17. The system of any one of claims 14-16, wherein the patient-generated health data is recorded by one or more wearable sensors.

18. The system of any one of claims 14-17, wherein the patient-generated health data comprises at least one of sleep data, heart rate data, physical activity data, body temperature, weight, calorie intake data, or macronutrient intake data.

19. The system of any one of claims 14-18, wherein the PRO data comprises a respective metric related to at least one of pain, fatigue, drowsiness, nausea, appetite, shortness of breath, depression, anxiety, constipation, diarrhea, sleep, distress, or suicidal ideation.

20. The system of any one of claims 14-19, wherein the medical imaging data comprises one or more imaging features.

21. The system of any one of claims 14-20, wherein the medical risk for the subject is related to an unplanned medical encounter; a change in medication; an adverse reaction to a drug, pharmaceutical, substance, compound, or treatment; a development of a disease or condition; a relapse or recurrence of a disease or condition; or a death.

22. The system of any one of claims 14-21, wherein the trained machine learning model is a supervised learning model.

23. The system of claim 22, wherein the trained machine learning model is a logistic regression model, a logistic classifier model, a support vector machine, or an artificial neural network.

24. A method for training a machine learning model, the method comprising: receiving multi-modal health data for each of a plurality of subjects; processing, using a data integration machine learning model, the multi-modal health data to create a data set;storing the data set in a data repository; and training a machine learning model with the data set, wherein the trained machine learning model is configured to predict a medical risk for a subject, and wherein the multi-modal health data for the subject comprises patient-generated health data (PGHD), patient-reported outcome (PRO) data, and medical imaging data.

25. A method for predicting treatment-related symptoms in a subject, comprising: receiving multi-modal health data for the subject; inputting the multi-modal health data into a trained machine learning model; and predicting, using the trained machine learning model, a risk of immunotherapy-induced lung inflammation in the subject, wherein the multi-modal health data for the subject comprises patientgenerated health data (PGHD), patient-reported outcome (PRO) data, and medical imaging data.

26. The method of claim 25, wherein the patient-generated health data is recorded by one or more wearable sensors.

27. The method of any one of claim 25 or 26, wherein the patient-generated health data comprises at least one of sleep data, heart rate data, or physical activity data.

28. The method of any one of claims 25-27, wherein the PRO data comprises a respective metric related to at least one of wheezing, pain, fatigue, drowsiness, nausea, appetite, shortness of breath, depression, anxiety, constipation, diarrhea, sleep, distress, or suicidal ideation.

29. The method of any one of claims 25-28, wherein the medical imaging data comprises one or more computed tomography (CT) scan, positron emission tomography (PET) scan, or CT / PET scan imaging features.

30. The method of any one of claims 25-29, further comprising predicting, using the trained machine learning model, a severity of the immunotherapy-induced lung inflammation in the subject.

31. The method of any one of claims 25-30, wherein the multi-modal health data for the subject comprises pre-treatment data only.

32. The method of any one of claims 25-31, wherein the multi-modal health data for the subject comprises post-treatment data.

33. The method of claim 32, wherein the multi-modal health data for the subject further comprises pre-treatment data.

34. A method for predicting medical risk for a subject having non-small cell lung cancer (NSCLC), the method comprising: receiving multi-modal health data for the subject having NSCLC, wherein the multi-modal health data for the subject comprises patient-generated health data (PGHD) and patient-reported outcome (PRO) data; inputting the multi-modal health data into a trained machine learning model; and predicting, using the trained machine learning model, a medical risk related to treatment of the subject having NSCLC.

35. The method of claim 34, wherein the multi-modal health data for the subject further comprises electronic medical record (EMR) data.

36. The method of any one of claim 34 or 35, wherein the multi-modal health data for the subject further comprises clinical data.

37. The method of any one of claims 34-36, wherein the PGHD is recorded by one or more wearable sensors.

38. The method of any one of claims 34-37, wherein the PGHD comprises at least one of sleep data, heart rate data, physical activity data, body temperature, weight, calorie intake data, or macronutrient intake data.

39. The method of any one of claims 34-38, wherein the PRO data comprises a respective metric related to at least one of pain, fatigue, drowsiness, nausea, appetite, shortness of breath, depression, anxiety, constipation, diarrhea, sleep, distress, or suicidal ideation.

40. The method of any one of claims 34-39, wherein the medical risk is a risk of healthcare utilization.

41. The method of claim 40, wherein the risk of healthcare utilization is an urgent care or emergency room visit by the subject.

42. The method of any one of claims 34-41, further comprising administering a treatment to the subject based upon the predicted medical risk.

43. The method of claim 42, wherein administering the treatment to the subject comprises changing a dosage amount or dosage schedule for a drug, pharmaceutical, substance, or compound; changing a type of a drug, pharmaceutical, substance, or compound; or changing an amount, frequency, or type of treatment.

44. The method of any one of claims 34-43, wherein the trained machine learning model is a supervised learning model.

45. The method of claim 44, wherein the trained machine learning model is a Bayesian network.

46. A system for predicting medical risk for a subject having non-small cell lung cancer (NSCLC), the system comprising: a computing device comprising a processor and a memory, the memory having computerexecutable instructions stored thereon that, when executed by the processor, cause the processor to: receive multi-modal health data for the subject having NSCLC, wherein the multi-modal health data for the subject comprises patient-generated health data (PGHD) and patient-reported outcome (PRO) data; input the multi-modal health data into a trained machine learning model; and receive a medical risk for the subject, the medical risk being predicted by the trained machine learning model, wherein the medical risk is related to treatment of the subject having NSCLC.

Images