System and method for ensemble-based causal inference in clinical data

Abstract

Disclosed is method and system for identifying treatment subpopulations within a patient population of a clinical study, by applying a causal ensemble model configured to output an ensemble Conditional Average Treatment Effect (eCATE).

Description

[0001] SYSTEM AND METHOD FOR ENSEMBLE-BASED CAUSAL INFERENCE IN CLINICAL DATA

[0002] TECHNOLOGICAL FIELD

[0003] The present disclosure relates generally to computer-implemented methods and systems for data processing and decision support in clinical studies, and more particularly to a technically implemented framework for identifying treatment-responsive subpopulations using ensemble causal inference models. The disclosure addresses technical challenges associated with estimating individualized treatment effects in small- sample, high-dimensional datasets by implementing computational techniques for model integration, uncertainty weighting, and stratified output generation for clinical trial design and treatment optimization.

[0004] BACKGROUND

[0005] Clinical trials aim to estimate the safety and efficacy of a tested treatment, usually in comparison to a control (Standard or Care or Placebo). Efficacy is measured according to clinical outcomes which are commonly referred to as the trial’s endpoints. The main measure of interest in terms of efficacy is the treatment effect, which is the expected difference in outcome under treatment vs outcome under control. However, for any individual patient, one can only observe one of these potential outcomes (either under treatment or under control), but never both - resulting in “the fundamental problem of causal inference”. Since for each patient only one potential outcome is observed, a natural way to infer about treatment effect is to use multiple patients - some given the treatment and some the control and compare their outcomes on average. If the assignment of treatment is independent of the potential outcome, then the difference between average outcome of the treated patients and the average outcome of the control patients equals the Average Treatment Effect (ATE).

[0006] However, often, there is considerable variability in the treatment effect between different subgroups of patients due to an underlying heterogeneity. This variability is one of the key reasons for failures when transitioning from Phase 2 to Phase 3 studies. As a result, medicaments that potentially could be effective for a subgroup of patients, are discarded for use.

[0007] Therefore, the identification of subgroups within the patient population is a paramount part of analyzing clinical data. To this end localized estimates of a treatment effect - also referred to as Conditional Average Treatment Effect (CATE) is conducted.

[0008] However, CATE analyses are frequently unsuccessful as their results are often not replicated in future studies, once again resulting in medicaments, that potentially could be effective for a subgroup of patients, discarded for use.

[0009] Moreover, computing a CATE can be very challenging in particular in exploratory clinical phases, in which significant clinical information is accumulated for the first time, and critical decisions about the target population of a treatment need be made, often based on small sample sizes (especially in Phase II exploratory trials).

[0010] Causal stacking has been suggested as a solution. However, while causal stacking is a good method for larger datasets where the ability to reliably estimate the correct weighting of each model improves performance, the dataset of clinical trials is often too small to obtain reliable results using this method. In fact, stacking methods often overfit, especially when a large number of models is used.

[0011] There therefore remains a need for an improved computation of CATE that can reliably identify subpopulations in clinical studies.

[0012] SUMMARY

[0013] According to some embodiments, there is provided a method for identifying one or more treatment subpopulations within a patient population of a clinical study, by applying on a dataset obtained from the clinical study, a causal ensemble model, the ensemble model integrating at least two different causal predictive models using a consensus based approach, to obtain an improved CATE, also referred to herein as “eCATE”.

[0014] Advantageously, the herein disclosed approach enables identifying subgroups in “wide data”, i.e. data including many covariates relative to the sample size. Specifically, the herein disclosed system and method advantageously enable reliable subgroup estimation in datasets in which the number of covariates is larger than the sample size.

[0015] The disclosed method and system thus provide a novel ensemble -based causal inference framework that reduces overfitting and improves treatment effect estimation stability, in particular in small-sample- wide-covariate datasets. That is, by solving the computer science problem associated with “wide data”, “model agreement”, and computational intractability, the hereindisclosed system and method provides technically superior subgroup detection in computer-aided clinical development.

[0016] Advantageously, the hereindisclosed eCATE model demonstrates improved generalization error and an excess RMSE improvement of 15-30%, as compared to a top- performing individual model.

[0017] Advantageously, the hereindisclosed methods and systems provide a technical improvement in machine learning-based causal inference, in particular as applied to high- dimensional, low-sample datasets for clinical trial subpopulation detection.

[0018] In summary, the present invention provides a computer-implemented method and system for identifying treatment-responsive subpopulations within a patient population using ensemble causal inference techniques. The invention contributes to the technical field of clinical decision support systems and medical trial optimization.

[0019] In particular, the invention produces a technical effect by:

[0020] Improving the accuracy and robustness of treatment effect estimation in clinical datasets characterized by a high-dimensional feature space and limited sample size, a common setting in exploratory Phase II trials and rare disease cohorts;

[0021] Enabling computational integration of heterogeneous predictive models using model- specific confidence estimates, thereby reducing the influence of outlier models and stabilizing the ensemble output;

[0022] Facilitating the automated generation of stratified clinical trial designs by using computational logic to isolate subpopulations most likely to benefit from treatment, supporting objective, algorithmic decision-making; Reducing computational bias and overfitting by means of ensemble averaging and feature-consensus filtering, improving the generalizability of personalized treatment recommendations;

[0023] Executing said operations within a data processing system comprising a processor and memory, thereby achieving effects that are not attainable through mental acts or manual statistical techniques.

[0024] These effects go beyond the mathematical modeling of patient outcomes and instead provide a technical contribution to the operation of clinical computing systems and personalized medicine infrastructure.

[0025] Moreover, the consensus-based approach can advantageously detect even subtle signals, thereby capturing variability at a high resolution. This in turn facilitates discovering personalized treatments, which take individual patient differences into account.

[0026] Furthermore, the herein disclosed method and system enable not only to predict treatment outcomes, but also aids in understanding the mechanisms through which the treatments work, thus further paving the way for more personalized and effective therapies.

[0027] Moreover, by integrating at least two different causal predictive models into a universal consensus-based ensemble algorithm, a methodological synergy between the models is unexpectedly achieved.

[0028] According to some embodiments, there is provided a computer-implemented method for identifying one or more treatment subpopulations within a patient population of a clinical study, the method including: a. obtaining a dataset comprising a measured treatment response for each individual in at least one drug treated patient group and a measured treatment response for each individual in a control treated patient group, wherein each individual belongs either to the drug treated group or to the control group, and wherein each individual in the drug treated group and the control group is characterized by a plurality of features, thereby forming a multidimensional feature space; b. computing an estimated counterfactual treatment response for each individual in the at least one drug treated patient group and for each individual in the control treated patient group c. fitting at least two different causal predictive machine learning models on the dataset, each causal predictive model configured to output a Conditional Average Treatment Effect (CATE) for each point in the multidimensional feature space; d. converting the at least two different causal predictive models into a single causal ensemble model using a consensus based approach , e. fitting the causal ensemble model on the dataset, thereby generating a trained causal ensemble model, f. utilizing the trained causal ensemble model to: i. compute an ensemble Conditional Average Treatment Effect (eCATE) for each point in the multidimensional feature space; ii. identify one or more subpopulations within the patient population, based on the eCATE and their associated features; and iii. optionally, automatically produce a clinical study plan comprising sample size and stratification factors, based on the identified subpopulation or subpopulations.

[0029] According to some embodiments, the method further includes conducting a clinical study using the produced clinical study plan.

[0030] According to some embodiments, the method further includes transmitting the clinical study plan to a clinical decision support system.

[0031] According to some embodiments, at least one of the at least two causal predictive models is a meta-learner. According to some embodiments, the applying of the at least two causal predictive models includes computing a counterfactual control treatment response for each individual in the at least one drug treated patient group and a counterfactual treatment response for each individual in the control drug treated patient group. According to some embodiments, computing the counterfactual treatment response includes, for each individual in the at least one drug treated group, inputting his / her features into a model trained on the control treated group, and for each individual in the control treated group, inputting his / her features into a model trained on the drug treated group. According to some embodiments, the method further includes computing a hypothetical individual treatment effect (ITE) for each individual in the dataset, based on a difference between the measured treatment response and the counterfactual treatment response of each individual in the training set. According to some embodiments, the dataset comprises the treatment response of about 10-10000 individuals.

[0032] According to some embodiments, at least one of the at least two causal predictive models is a causal-forest or a causal tree learner. According to some embodiments, at least one of the at least two causal predictive models is derived using a direct estimation method. According to some embodiments, the causal ensemble model configured output a predicted treatment response for each point in the multi-dimensional space.

[0033] According to some embodiments, the method further includes computing a confidence interval for each of the at least two causal predictive models.

[0034] According to some embodiments, the eCATE is a consensus-based (CBA) eCATE. According to some embodiments, computing the CBA eCATE includes averaging the CATE of the predictive models out of the at least two causal predictive models having a computed confidence interval within a predetermined threshold value only. According to some embodiments, computing the CBA eCATE includes averaging the CATE of the predictive models out of the at least two causal predictive models identifying a same group of features as influencing the CATE only.

[0035] According to some embodiments, the at least two causal predictive ML models are selected from generalized linear model (GLM), Accurate GLM, Causal Forest, Regression Trees, Boosted Regression Trees, Random Forest, Bayesian Additive Regression Trees (BART), Neural Networks deep learning methods (TAR-Net), non-parametric methods such as Gaussian Process regression, Causal Graphical Models or any combination thereof. Each possibility is a separate embodiment.

[0036] According to some embodiments, the at least two causal predictive ML models are selected from Accurate GLM, Causal Forest, Random Forest, Bayesian Additive Regression Trees (BART), or any combination thereof. Each possibility is a separate embodiment. According to some embodiments, the at least two causal predictive ML models comprise at least three, at least four or at least five causal predictive models. Each possibility is a separate embodiment.

[0037] According to some embodiments, the dataset is a clinical trial dataset, an observational dataset, a real-world dataset or any combination thereof. Each possibility is a separate embodiment.

[0038] According to some embodiments, the method further includes outputting, e.g. by displaying on a display, the identified subgroups and their identifying features.

[0039] According to some embodiments, there is provided a system including a memory and a processor coupled to the memory programmed with executable instructions, configuring the processor to: a. obtain a dataset comprising a measured treatment response for each individual in at least one drug treated patient group and a measured treatment response for each individual in a control treated patient group, wherein each individual in the drug treated group and the control group is characterized by a plurality of features, thereby forming a multidimensional feature space; b. apply at least two different causal predictive models on the dataset, each causal predictive model configured to output a Conditional Average Treatment Effect (CATE) for each point in the multidimensional feature space; c. integrate the at least two different causal predictive models into a causal ensemble model configured” to output an ensemble Conditional Average Treatment Effect (eCATE) for each point in the multidimensional feature array, d. identifying one or more subpopulations within the patient population, based on the eCATE and their associated features.

[0040] Advantageously, the hereindisclosed system provides improved processing capabilities to the processor thus allowing it to reliably and robustly identify patient subgroups with patient populations of a clinical study.

[0041] According to some embodiments, the processor is configured to output, e.g. on a display, the identified patient sub-groups and their common features. According to some embodiments, there is provided a system for identifying one or more treatment subpopulations within a patient population of a clinical study, the system comprising:

[0042] (a) a memory for storing a dataset comprising: a measured treatment response for each individual in at least one drug-treated patient group; a measured treatment response for each individual in a control-treated patient group; and a counterfactual treatment response for each individual in the at least one drug- treated patient group; a counterfactual treatment response for each individual in the control-treated patient group; for each individual, a plurality of features defining a multidimensional feature space;

[0043] (b) a processor coupled to the memory and configured to execute instructions that cause the system to:

[0044] (i) fit at least two different causal predictive models to the dataset or to a computed hypothetical individual treatment effect (ITE) for each individual in the dataset, to output a Conditional Average Treatment Effect (CATE) for each point in the multidimensional feature space;

[0045] (ii) for each causal predictive model, compute a model- specific confidence interval associated with the respective CATE output;

[0046] (iii) convert the outputs of the at least two causal predictive models into a causal ensemble model, the converting comprising: averaging the CATE outputs of the models; and weighting the CATE outputs based on the respective confidence intervals, wherein the causal ensemble model is configured to output an ensemble CATE (eCATE) for each point in the multidimensional feature space;

[0047] (iv) identify one or more treatment subpopulations, based on the eCATE values and associated features.

[0048] According to some embodiments, the model- specific confidence interval is computed by the causal predictive model during fitting or wherein the model-specific confidence interval is computed using conformal prediction.

[0049] According to some embodiments, the system further includes a display configured to present the identified subpopulations and their distinguishing features to a user.

[0050] According to some embodiments, the system further comprising a user interface (UI) configured to allow a user to interact and affect changes to one or more subgroups defined by the features defining the multidimensional feature space. According to some embodiments, the changes made by the user are transmitted to or retrieved by the processor. According to some embodiments, the processor is further configured to reidentify one or more treatment subpopulations based on the affected changes. According to some embodiments, the processor is further configured to regenerate the clinical study design based on the affected changes

[0051] According to some embodiments, the processor is configured to fit the ensemble model on a same or different dataset.

[0052] According to some embodiments, the system is configured to operate on clinical datasets having more covariates than samples.

[0053] According to some embodiments, the ensemble model provides improved CATE estimation accuracy over individual causal predictive model applied in isolation.

[0054] According to some embodiments, at least one of the causal predictive models is a meta- learner selected from: S -learner, T-learner, X-learner, or DR-leamer. Each possibility and combination of possibilities is a separate embodiment. According to some embodiments, the causal predictive models are selected from a causal forest model, an accurate generalized linear model (AGLM), a Bayesian additive regression trees (BART) model, causal BART or any combination thereof. Each possibility and combination of possibilities is a separate embodiment

[0055] According to some embodiments, the causal ensemble model comprises at least three different causal predictive models with distinct underlying learning algorithms.

[0056] According to some embodiments, the processor is configured to compute the estimated counterfactual treatment responses for each individual in the at least one drug-treated patient group and for each individual in the control-treated patient group. According to some embodiments, the processor is configured to send / transmit the estimated counterfactual treatment responses to the memory for storage.

[0057] According to some embodiments, the processor is configured to compute the hypothetical individual treatment effect (ITE) for each individual in the dataset, based on a difference between the measured treatment response and the counterfactual treatment response of each individual in the dataset. According to some embodiments, the processor is configured to send / transmit the computed ITEs to the memory for storage.

[0058] According to some embodiments, the dataset is selected from: a clinical trial dataset, an observational study dataset, a real-world evidence dataset, or any combination thereof. Each possibility and combination of possibilities is a separate embodiment.

[0059] According to some embodiments, the identified treatment subpopulation comprises patients expected to exhibit a positive response to a specified treatment arm.

[0060] According to some embodiments, the system is further configured to identify subpopulations exhibiting no treatment effect and flag such subpopulations for exclusion in future trial planning.

[0061] According to some embodiments, there is provided a computer-implemented method for identifying one or more treatment subpopulations within a patient population of a clinical study, the method comprising: obtaining a dataset comprising: a measured treatment response for each individual in at least one drug-treated patient group; a measured treatment response for each individual in a control-treated patient group; and a counterfactual treatment response for each individual in the at least one drug- treated patient group; a counterfactual treatment response for each individual in the control-treated patient group; for each individual, a plurality of features defining a multidimensional feature space; fitting, by a processor, at least two different causal predictive models to the dataset or to a computed hypothetical individual treatment effect (ITE) for each individual in the dataset, to output a Conditional Average Treatment Effect (CATE) for each point in the multidimensional feature space; computing, by the processor, a model- specific confidence interval for each of the at least two causal predictive models; converting, by the processor, the CATE outputs into a causal ensemble model, the converting comprising: averaging the CATE outputs; and weighting the CATE outputs according to the respective computed confidence intervals; wherein the causal ensemble model is configured to output an ensemble CATE (eCATE) for each point in the multidimensional feature space; and identifying, by the processor, one or more treatment subpopulations within the patient population, based on the eCATE values and their associated features. According to some embodiments, the method further includes generating a stratified clinical study design comprising sample size estimates, inclusion or exclusion criteria, or stratification factors for each identified subpopulation.

[0062] According to some embodiments, the computer-implemented method is suitable for use on clinical datasets having more covariates than samples.

[0063] According to some embodiments, at least one of the causal predictive models is a meta- learner selected from: S -learner, T-learner, X-learner, or DR-leamer. Each possibility and combination of possibilities is a separate embodiment

[0064] According to some embodiments, the causal predictive models are selected from a causal forest, an accurate generalized linear model (AGLM), and a Bayesian additive regression trees (BART) model. Each possibility and combination of possibilities is a separate embodiment.

[0065] According to some embodiments, the causal ensemble model comprises at least three causal predictive models trained using distinct machine learning frameworks.

[0066] According to some embodiments, the model- specific confidence interval is computed by the causal predictive model during fitting or wherein the model-specific confidence interval is computed using conformal prediction.

[0067] According to some embodiments, the method further includes presenting the identified subpopulations and their distinguishing features to a user via a display.

[0068] According to some embodiments, the method further includes enabling a user to interact and affect changes to one or more subgroups defined by the features defining the multidimensional feature space, via a user interface.

[0069] According to some embodiments, the method further includes reidentifying one or more treatment subpopulations based on the affected changes.

[0070] According to some embodiments, the method further comprising regenerating the clinical study design based on the affected changes. According to some embodiments, the dataset is selected from: a clinical trial dataset, an observational dataset, a real-world dataset, or any combination thereof. Each possibility and combination of possibilities is a separate embodiment.

[0071] According to some embodiments, the identified treatment subpopulation comprises patients expected to exhibit a statistically significant response to a given treatment arm.

[0072] According to some embodiments, the method further includes identifying one or more subpopulations expected to exhibit no treatment response and excluding them from the generated study design.

[0073] According to some embodiments, the method further includes computing the hypothetical individual treatment effect (ITE) for each individual, based on the difference between the measured treatment response and a counterfactual prediction.

[0074] According to some embodiments, there is provided a computer-implemented method for identifying treatment-responsive subpopulations in a clinical study, comprising: a. receiving, at a computing system, a dataset comprising: a measured treatment response for each individual in at least one drug-treated patient group; a measured treatment response for each individual in a control-treated patient group; and a counterfactual treatment response for each individual in the at least one drug- treated patient group; a counterfactual treatment response for each individual in the control-treated patient group; for each individual, a plurality of features defining a multidimensional feature space b. fitting, by the computing system, at least two different causal inference models to compute respective estimates of the Conditional Average Treatment Effect (CATE) for each subject; c. computing, for each model, a confidence indicator associated with the respective CATE estimate; d. computing an ensemble treatment effect estimate (eCATE) for each subject by combining the CATE outputs based on the confidence indicators; e. identifying, based on the eCATE values and feature profiles, one or more subpopulations predicted to respond to treatment; and f. outputting the identified subpopulations for use in clinical trial stratification, subject recruitment, or treatment planning.

[0075] According to some embodiments, the computing system applies a consensus-based approach in step (d), comprising selecting only those models whose CATE estimates satisfy a predefined agreement threshold.

[0076] According to some embodiments, the at least one of the causal inference models is selected from a meta-learner, a causal forest model, or a Bayesian additive regression tree model. Each possibility and combination of possibilities is a separate embodiment.

[0077] According to some embodiments, the identified subpopulations are used to generate a clinical trial plan specifying a sample size, inclusion criteria, or stratification parameters.

[0078] According to some embodiments, the output is provided via a clinical decision support interface.

[0079] According to some embodiments, the dataset comprises real-world evidence obtained from electronic health records, medical claims, or patient registries.

[0080] According to some embodiments, a counterf actual treatment response is estimated for each subject using a model trained on the opposite treatment group.

[0081] According to some embodiments, there is provided a data processing system comprising a processor and memory storing instructions which, when executed by the processor, cause the system to perform the method of any one of the previous embodiments.

[0082] According to some embodiments, there is provided a data processing system configured to identify treatment-responsive subpopulations in a clinical study, the system comprising:

[0083] (a) a memory configured to store a dataset comprising: a measured treatment response for each subject in at least one treatment group and in at least one control group;

[0084] • and a plurality of features associated with each subject, defining a multidimensional feature space;

[0085] (b) a processor, coupled to the memory, and configured to:

[0086] (i) apply / fit at least two different causal inference models to compute respective Conditional Average Treatment Effect (CATE) estimates for each subject;

[0087] (ii) compute, for each model, a confidence indicator associated with the respective CATE estimate;

[0088] (iii) compute an ensemble treatment effect estimate (eCATE) for each subject, by combining the CATE outputs based on the confidence indicators;

[0089] (iv) identify, based on the eCATE values and the feature vectors, one or more subpopulations expected to respond to treatment; and

[0090] (v) optionally, output the identified subpopulations for use in clinical trial stratification or treatment planning.

[0091] According to some embodiments, the ensemble treatment effect (eCATE) is computed as a weighted average, with weights inversely proportional to the width of the confidence intervals.

[0092] According to some embodiments, the processor is further configured to output a clinical trial design comprising stratification factors or inclusion criteria based on the identified subpopulations.

[0093] According to some embodiments, the dataset comprises real- world patient data extracted from electronic health records or clinical trial databases.

[0094] According to some embodiments, at least one of the causal inference models is a meta- learner selected from the group consisting of: S -learner, T-learner, X-leamer, and DR-leamer. Each possibility and combination of possibilities is a separate embodiment. According to some embodiments, there is provided a method of performing a clinical trial based on clinical trial design parameters identifying using the methods and / or system disclosed herein (all, or some embodiments, thereof).

[0095] According to some embodiments, there is provided a substance or composition for treating a disease where the substance and composition is regulatory approved based on a clinical trial computed using the methods and / or system disclosed herein (all, or some embodiments, thereof).

[0096] Certain embodiments of the present disclosure may include some, all, or none of the above advantages. One or more technical advantages may be readily apparent to those skilled in the art from the figures, descriptions and claims included herein. Moreover, while specific advantages have been enumerated above, various embodiments may include all, some or none of the enumerated advantages.

[0097] In addition to the exemplary aspects and embodiments described above, further aspects and embodiments will become apparent by reference to the figures and by study of the following detailed descriptions.

[0098] BRIEF DESCRIPTION OF THE FIGURES

[0099] Some embodiments of the disclosure are described herein with reference to the accompanying figures. The description, together with the figures, makes apparent to a person having ordinary skill in the art how some embodiments may be practiced. The figures are for the purpose of illustrative description and no attempt is made to show structural details of an embodiment in more detail than is necessary for a fundamental understanding of the disclosure. For the sake of clarity, some objects depicted in the figures are not drawn to scale. Moreover, two different objects in the same figure may be drawn to different scales. In particular, the scale of some objects may be greatly exaggerated as compared to other objects in the same figure.

[0100] In block diagrams and flowcharts, certain steps may be conducted in the indicated order only, while others may be conducted before a previous step, after a subsequent step or simultaneously with another step. Such changes to the orders of the step will be evident for the skilled artisan.

[0101] FIG. 1 is an exemplary flowchart of the hereindisclosed method for identifying one or more treatment subpopulations within a patient population of a clinical study;

[0102] FIG. 2 illustratively depicts the hereindisclosed system for identifying one or more treatment subpopulations within a patient population of a clinical study;

[0103] FIG. 3a-3c show exemplary screen shots of the hereindisclosed interactive user interface.

[0104] FIG. 4 is a bar graph showing the excess scaled (Root Mean Square Error) RMSE (vs. best) of various CATE estimators as compared to the herein disclosed CATE ensemble model.

[0105] DETAILED DESCRIPTION

[0106] In the following description, various aspects of the disclosure will be described. For the purpose of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the different aspects of the disclosure. However, it will also be apparent to one skilled in the art that the disclosure may be practiced without specific details being presented herein. Furthermore, well-known features may be omitted or simplified in order not to obscure the disclosure.

[0107] According to some embodiments, disclosed are method and systems for identifying one or more treatment subpopulations within a patient population of a clinical study.

[0108] As used herein the term “clinical study” refers to research studies that test how well medical approaches work in people.

[0109] According to some embodiments, the clinical study may be a clinical trial. As used herein, the term “clinical trial” refers to prospective biomedical (or behavioral) research studies on human participants designed to answer specific questions about new treatments. They generate data on dosage, safety and efficacy and typically include four phases. According to some embodiments, the clinical trial may be an exploratory phase II clinical trial. According to some embodiments, the clinical study may be an observational study. As used herein, the term “observational study” refers to a study in which events, behaviors, or phenomena are recorded as they naturally occur without interference or manipulation.

[0110] According to some embodiments, the clinical study may be a real-world study. As used herein the term, “real world study” refers to the collection of Real -world data (RWD) i.e. data relating to patient health status routinely collected from a variety of sources. RWD can be generated from: Electronic health records, medical claims, billing data, insurance data, data from product and disease registries, patient-generated data, data gathered from mobile devices etc.

[0111] According to some embodiments, the method includes obtaining a dataset including a measured treatment response for each individual in at least one treated patient group of the clinical study and a measured treatment response for each individual in a control patient group of the clinical study. According to some embodiments, each individual in the drug treated group and in the control group is characterized by a plurality of features, which form a multidimensional feature space.

[0112] According to some embodiments, the system includes a memory configured to store the dataset.

[0113] According to some embodiments, the term “control patient group” may refer to a group of patients receiving no treatment. According to some embodiments, the term “control patient group” may refer to a group of patients receiving a placebo treatment. According to some embodiments, the term “control patient group” may refer to a group of patients receiving standard care (SoC). According to some embodiments, the term “control patient group” may refer to a hypothetical group of patients derived from real world data (RWD), e.g. using matching algorithms. For example, in single arm studies 9e.g. single arm oncology studies), a randomized clinical trial may be achieved by deriving matching SoC data from RWD or from other clinical trials.

[0114] As used herein the term “measured treatment response” refers to the actual treatment response measured for an individual. According to some embodiments, the measured treatment response is a function of a vector of the patient’s features / covariates and the treatment allocation (e.g. treatment I, treatment II or control). According to some embodiments, the measured treatment response can be mirrored by a “hypothetical treatment response”, also referred to herein as a “counterfactual treatment response”. According to some embodiments, the method includes computing an estimated counterfactual treatment response for each individual in the treatment patient group and the control patient group. According to some embodiments, the processor is configured to compute the estimated counterfactual treatment response and subsequently store it in a memory functionally connected to the processor.

[0115] For example, if a patient is allocated to a treatment group (for which he / she has a measured treatment response) a hypothetical response may be computed for the same individual, as further elaborated herein.

[0116] According to some embodiments, the term “at least one”, with respect to treatment groups (also referred to as “arms”) of a clinical study, may refer to a clinical study including a single treatment group, two treatment groups (e.g. first medicament and second medicament, first dose and second dose etc.), three treatment groups, four treatment groups, five treatment groups or more. Each possibility is a separate embodiment. According to some embodiments, a clinical study including more than one treatment arm may include a single control group (e.g. standard care or placebo). According to some embodiments, a clinical study including more than one treatment arm may include a control group for each arm.

[0117] According to some embodiments, the term “plurality” with respect to the features refers to at least 3 features, at least 5 features, at least 10 features, at least 15 features or more. Each possibility is a separate embodiment. According to some embodiments, the plurality of features may include three or more of: age, sex, weight, height, ethnicity, medical background, marital status, geographic location, socio-economic status, heart rate at rest, saturation, diet, number of children, number of pregnancies, number of unforced abortions, genomic signatures such as PD1 levels, metabolic signatures, previous medications and the like. Each possibility and combination of possibilities is a separate embodiment.

[0118] As used herein, the term “multidimensional feature space” refers to the space generated by the multiple combination of features (also referred to as “covariate”) characterizing each individual in the clinical study (e.g. female, age 50, weight 70kg, married with 5 children etc.) as well as hypothetical combinations (i.e., combinations of features that are possible but not represented by any of the individuals in the clinical study. According to some embodiments, the method includes training or fitting at least two different causal predictive models on the dataset. According to some embodiments, each causal predictive model configured to output a predicted Conditional Average Treatment Effect (CATE) for each point in the multidimensional feature space (real and / or hypothetic). According to some embodiments, the processor is configured to train or fit the at least two different causal predictive models on the dataset.

[0119] According to some embodiments, the one or more causal predictive models and / or the causal ensemble model are trained on the dataset. As used herein, the term “training” refers to the process by which a machine learning algorithm learns from data to make predictions or decisions without being explicitly programmed to perform the task.

[0120] According to some embodiments, the one or more causal predictive models and / or the causal ensemble model are fitted to the dataset. As used herein, the term “fitting” refers to a machine learning approach where the model learns a desired output directly from the input data (also referred to as “direct learning”) without training on a training dataset. Direct learning may be particularly advantageous when the dataset is smaller than the number of variables. According to some embodiments, the direct learning comprises using the entire dataset for the learning process.

[0121] It is understood, that a person of ordinary skill in the art would understand that CATE estimation is technically infeasible without computational models, especially in wide-data contexts.

[0122] As used herein, the terms “Average Treatment Effect” and “ATE” refer to the difference between an average outcome of treated patients and an average outcome of control patients. Assuming that the assignment of treatment is independent of the potential outcomes, ATE can be expressed as:

[0123] E[y (a=l) - y (a=0) ] if: all{y (a=0) ,y (a=l) } = E[y|a = 1] - E[y|a = 0]

[0124] Where a is the treatment assignment (0 for control, 1 for treatment), and y (a=i) is the outcome under treatment assignment i.

[0125] As used herein, the terms “Conditional Average Treatment Effect” and “CATE” refer to a difference between the expected outcomes of the two treatments conditioned on covariates, i.e. the average effect of a treatment on a sub-group, wherein the validity of the estimate is conditional on being part of this subgroup. CATE is distinct from ATE, which is the average treatment effect on an entire study population.

[0126] CATE can be expressed as: r (Xi ) = E[y (a=l) - y (a=0) |X = Xi ] based on the patients features; by definition r (X) = E[ri |X]

[0127] As used herein, the term “causal predictive models” and “predictive causal models” may be used interchangeably and refer to machine learning (ML) models that relate independent variables (i.e. variables which can be manipulated) to dependent variables (variables that can be measured), generating predictions for the values of dependent variables given a set of values for the independent variables. According to some embodiments, the at least two predictive models may be causal forests and / or meta learners.

[0128] Causal forest (CF) is an adaptation of random forests, in which the base trees composing the forest are causal trees, aimed at estimating local differences between average potential outcomes. According to some embodiments, the CF is a CF with double / debiased machine learning. Double / debiased machine learning (DML) is a method developed to use regularized regression techniques for variable selection in a high-dimensional causal inference setting. It seeks variables that are highly correlated with both treatment and outcome, thereby reducing small approximation errors that arise when selecting among a large set of covariates.

[0129] Meta-Leamers are an estimation framework that enables using any ML model as a “base learner” for learning various nuisance functions and composing an estimator for CATE using a transformation of the learned functions. There are several common meta-leamer structures, including, but not limited to:

[0130] • S learner: A single (hence “S”) model is trained to regress the outcomes on the features and the treatment assignment (the treatment is treated as an additional binary variable attached to X). y'(x, a) = E [y(X, a) IX = x, a = a] CATE is the estimated by contrasting this model’s predictions for both potential outcomes:

[0131] • T learner: This approach uses base-learners to estimate the conditional expectations of the two (hence “T”) potential outcomes - {(Xi , yi ) ; ai = 0} that are used to train pO(X), an estimate for E[y|a = 0] and {(Xi , yi ) ; ai = 1 } to train pl(X). Finally, an estimate for CATE is obtained by subtracting them:

[0132] • X learner: This approach builds on the foundations of the T Learner and starts similarly by estimating p 0(X) and 'pl(X). it then uses these estimates to impute the missing potential outcomes and generate “pseudo individual effects”:

[0133] Next, {D 0 i} and {D 1 i} are used to train two separate estimates for CATE - VI (X) using {D 1 i } and A0(X) using {D 0 i }. Finally, a weighted average of the two estimates is used to estimate CATE.

[0134] • DR learner: This approach constructs a doubly-robust pseudo-outcome for CATE using a sub-sample of the training data, and uses the rest of the train-set to regress this pseudo outcome on X. First, using the first subset SI to train V(X), p 0(X), p'l(X) - estimates for 7t(X), E[y|a = 0, X], E[y|a = 1, X], respectively. Then, the following pseudo -outcome is constructed:

[0135] Finally, the rest of the train-sample, S2 = S \ SI, is used to regress:

[0136] Each of the aforementioned types of causal predictor model can use any ML model for any of the nuisance functions it estimates. The choice of base estimators should account for the assumed complexity of the underlying data-generating process, the size of available training data etc. Clinical trials often test complex mechanisms, which favor more flexible ML models. However, the typical sample sizes in this setting are usually quite limited, favoring simpler more tightly regulated models.

[0137] According to some embodiments, the at least two different causal predictive models (MLs) may be selected from:

[0138] • GLMs: Regularized (Lasso and Elastic-Net) linear regression is used to estimate outcome and pseudo-outcomes, and logistic regression to estimate the propensity.

[0139] • Accurate GLM (AGLM): This method utilizes Lasso regression to fit a piece-wise constant function, by first encoding each variable into nested bins.

[0140] • Boosted Regression Trees: A sum of trees, where each tree is fitted on the residuals of the previous one.

[0141] • Random Forests (RF): An average of trees, constructed with stochastic sampling of features and sample to induce variability in the trees, which acts as a form of regularization.

[0142] • Bayesian Additive Regression Trees (BART): inspired by ensemble methods, with boosting in particular, BART also trains a sum-of-trees model, with the addition of a regularization prior which controls the parameters of that model.

[0143] • Regression trees: decision tree algorithms where the target variable can take continuous values.

[0144] • Non-parametric methods such as gaussian process regression, or spline regression.

[0145] According to some embodiments, the at least two causal predictive models are converted into a causal ensemble model configured to output an “ensemble Conditional Average Treatment Effect” (also referred to herein as “eCATE”).

[0146] According to some embodiments, the converting comprises stacking (e.g. x-stacking) of base learners rather than of the causal predictive models themselves.

[0147] Advantageously, the ensemble model enables dynamic adaptation to model- specific uncertainty and improves the accuracy and generalizability of predictions over single models. According to some embodiments, the eCATE is computed by averaging the CATE of each the at least two different causal predictive models, also referred to herein as “a simple average eCATE”.

[0148] According to some embodiments, the at least two causal predictive models are converted into a causal ensemble model using a consensus approach. As used herein, the term “consensus approach refers to combining predictions from multiple models by reaching an "agreement" among them, rather than simply averaging their outputs or weighting them statistically.

[0149] According to some embodiments, the eCATE is computed by averaging the CATE of each the at least two different causal predictive models while weighing according to a computed confidence interval of each model, such that models that output a CATE with a higher confidence interval get at larger weight - in which case the eCATE is also referred to herein as “a weighted average eCATE”. According to some embodiments, computing the confidence interval is inherent to the causal predictive model. According to some embodiments computing the confidence interval comprises conformal prediction. It is understood by the skilled in the art that such computed model-specific confidence intervals which cannot be derived manually or by conventional statistical tools.

[0150] According to some embodiments, the consensus approach includes using some form of majority vote or agreement to decide the final output. According to some embodiments, the eCATE is a consensus-based (CBA) eCATE. According to some embodiments, the CBA- eCATE may be computed by only averaging the CATE of the predictive models having a confidence interval within a predetermined threshold value, while ignoring outlier models.

[0151] According to some embodiments, the CBA-eCATE may be computed by only averaging the CATE of the predictive models that identify the same features as influencing the CATE (e.g. averaging only models that identify gender, age and weight is influencers of CATE while ignoring models that identify other features (different from those of other models) e.g. height.

[0152] According to some embodiments, the hereindisclosed system and method exhibit reduced estimation bias and overfitting. According to some embodiments, the reduced bias is achieved by a method for bias estimation and valid inference in subgroup analysis, particularly when subgroups are selected based on their observed Conditional Average Treatment Effects (CATE). The approach leverages the bootstrap to estimate the selection-induced bias arising from evaluating multiple candidate partitions. Let G = {g1, ... , gm} denote the set of all candidate partitions, and let t(gl, X, Y, W) be the estimated CATE for subgroup gi, derived from covariates X, outcomes Y, and treatments W. A bootstrap procedure is employed: for each bootstrap replicate j = 1, . . ., K, a new sample {Xi, Yi, Wi} is drawn with replacement, and the CATEs are re-estimated. A conservative estimate of the selection bias is then computed as Bj = max; [f(gi, X, Y, W) - f(gi, Xi, Yi, Wi)], which allows for constructing bias-adjusted confidence intervals. While conservative, this method ensures robustness by accounting for the dependence across subgroups and a wide array of selection strategies.

[0153] To address the conservativeness and improve power, the principle of Simultaneous Over the Selected inference was integrated. In this approach, a portion of the significance level, a, is allocated for the selection of the most promising subgroups. Specifically, a set F c G serves as a confidence set, with coverage 1 - v, for the subgroups that are truly most promising according to the underlying selection criterion. Bias is then re-estimated as Bj = X, Y, W) — f gi, X’, Y’, ] / Y’)], reducing over-conservatism, while maintaining validity. The resulting confidence intervals, adjusted to reflect the selection step, are given by B(a-v . Note, that while the error is restricted to F, the inference is only valid to the most promising subgroups selected using the original data.

[0154] By combining bootstrap inference with local simultaneous inference, the method yields valid and less conservative confidence intervals and debiased estimates for the selected subgroups.

[0155] According to some embodiments, the method further includes identifying one or more drug subpopulations within the patient population, based on the eCATE and their associated features and optionally planning future clinical studies, based on the stratification into sub- populations.

[0156] Although some embodiments of the invention are not limited in this regard, discussions utilizing terms such as, for example, “processing,” “computing.” “calculating,” “determining,” “establishing”, “analyzing”, “checking”, “detecting”, “identifying”, “characterizing”, or the like, may refer to operation(s) and / or process(es) of a computer, a computing platform, a computing system, or other electronic computing device, that manipulates and / or transforms data represented as physical (e.g., electronic) quantities within the computer's registers and / or memories into other data similarly represented as physical quantities within the computer's registers and / or memories or other information non-transitory storage medium that may store instructions to perform operations and / or processes. Although embodiments of the invention are not limited in this regard, the terms “plurality” and “a plurality” as used herein may include, for example, “multiple” or “two or more”. The terms “plurality” or “a plurality” may be used throughout the specification to describe two or more components, devices, elements, units, parameters, or the like. The term set when used herein may include one or more items. Unless explicitly stated, the method embodiments described herein are not constrained to a particular order or sequence. Additionally, some of the described method embodiments or elements thereof can occur or be performed simultaneously, at the same point in time, or concurrently.

[0157] As used herein, the terms “approximately”, “essentially” and “about” in reference to a number are generally taken to include numbers that fall within a range of 5%, 2.5% or in the range of 1% in either direction (greater than or less than) the number unless otherwise stated or otherwise evident from the context (except where such number would exceed 100% of a possible value). Where ranges are stated, the endpoints are included within the range unless otherwise stated or otherwise evident from the context.

[0158] As used herein, the singular forms "a," "an" and "the" include plural referents unless the context clearly dictates otherwise.

[0159] As used herein, "optional" or "optionally" means that the subsequently described event or circumstance does or does not occur, and that the description includes instances where said event or circumstance occurs and instances where it does not.

[0160] It is appreciated that certain features of the disclosure, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the disclosure, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable sub- combination or as suitable in any other described embodiment of the disclosure. No feature described in the context of an embodiment is to be considered an essential feature of that embodiment, unless explicitly specified as such. Although stages of methods, according to some embodiments, may be described in a specific sequence, the methods of the disclosure may include some or all of the described stages carried out in a different order. In particular, it is to be understood that the order of stages and sub-stages of any of the described methods may be reordered unless the context clearly dictates otherwise, for example, when a latter stage requires as input an output of a former stage or when a latter stage requires a product of a former stage. A method of the disclosure may include a few of the stages described or all of the stages described. No particular stage in a disclosed method is to be considered an essential stage of that method, unless explicitly specified as such.

[0161] Although the disclosure is described in conjunction with specific embodiments thereof, it is evident that numerous alternatives, modifications, and variations that are apparent to those skilled in the art may exist. Accordingly, the disclosure embraces all such alternatives, modifications, and variations that fall within the scope of the appended claims. It is to be understood that the disclosure is not necessarily limited in its application to the details of construction and the arrangement of the components and / or methods set forth herein. Other embodiments may be practiced, and an embodiment may be carried out in various ways.

[0162] Reference is now made to FIG. 1, which is an exemplary flowchart of the herein disclosed method 100 for identifying treatment subpopulation(s) within a patient population of a clinical study.

[0163] In step 110, a clinical dataset is obtained, the dataset including the measured treatment response for each individual in one or more drug treated patient groups and the measured treatment response for each individual in one or more control treated patient group, wherein each individual in each of the groups is characterized by a plurality of features, forming a multidimensional feature space. According to some embodiments, the dataset also includes the counterfactual treatment response for each individual in the at least one drug-treated patient group and the counterfactual treatment response for each individual in the control-treated patient group.

[0164] In step 120, at least two different causal predictive (ML) models are each fitted to the dataset or to a computed hypothetical individual treatment effect (ITE) of individuals in the dataset, to output a Conditional Average Treatment Effect (CATE) for each point in the multidimensional feature space, whether the point reflects an actual or hypothetical patient. As a non-limiting example, the at least two different causal predictive (ML) models may be XLearner (BART or causal BART), Xlearner (RF), CF and AGLM. According to some embodiments, this step may also include computing or retrieving a model -specific confidence interval for the at least two causal predictive models

[0165] In step 130, the at least two different causal predictive models are converted into a causal ensemble model configured to output an ensemble Conditional Average Treatment Effect (eCATE) for each point in the multidimensional feature array. According to some embodiments, the integration comprises a simple averaging of the CATE of each the at least two different causal predictive models. According to some embodiments, the integration comprises computing a weighted average of the CATE of each of the at least two different causal predictive models, wherein the weighting is carried out, for example, based on the confidence interval of each model and / or the predictive strength of each model. According to some embodiments, the integration comprises consensus-based averaging, e.g. averaging the CATE only of models having a confidence interval within a predetermined threshold value and / or averaging the CATE only of models that identify the same dominant covariates.

[0166] In step 140, one or more subpopulations are identified within the patient population, based on the eCATE and their associated features. According to some embodiments, the subpopulation may be a subpopulation or subpopulations that are sensitive to a specific treatment arm. According to some embodiments, the subpopulation may be a subpopulation or subpopulations that do not respond to (any) treatment and as such should be excluded.

[0167] Optionally, method 100 may include an additional step 150 of planning a future clinical study (e.g. a phase III clinical trial) based on the identified subpopulation or subpopulations. According to some embodiments, the planning may include choosing an optimal number of individuals belonging to a specific subpopulation. According to some embodiments, the planning may include excluding subpopulations from the planned study etc.

[0168] Reference is now made to FIG. 2, which schematically shows a system 200 for identifying one or more treatment subpopulations within a patient population of a clinical study, according to some embodiments. It is noted that while the elements of system 200 are depicted as stand-alone modules, some or all of the modules may all be integrated into a single unit.

[0169] A first module (no hierarchy intended) of system 200 includes a memory module 210 (also referred to herein as “memory”) configured to inter alia store: a measured treatment response for each individual in at least one drug-treated patient group;

[0170] • a measured treatment response for each individual in a control-treated patient group; and

[0171] • a counterfactual treatment response for each individual in the at least one drug- treated patient group;

[0172] • a counterfactual treatment response for each individual in the control-treated patient group;

[0173] • for each individual, a plurality of features defining a multidimensional feature space;

[0174] According to some embodiments, memory module may also be configured to store a computed hypothetical individual treatment effect (ITE).

[0175] Functionally coupled to memory module 210 is a processing module 220 (also referred to herein as “processor”). Processing module 220 is configured to fit at least two different causal predictive models to the dataset or to the computed hypothetical ITE, to output a Conditional Average Treatment Effect (CATE) for each point in the multidimensional feature space. Optional causal predictive models include: a causal forest model, an accurate generalized linear model (AGLM), a Bayesian additive regression trees (BART) model, causal BART or any combination thereof. Processing module 220 is further configured to compute / retrieve a model-specific confidence interval associated with the respective CATE output. According to some embodiments, the confidence interval may for some models be computed integrally, while other causal predictive model require additional computation e.g. by conformal prediction. Processing module 220 is further configured to convert the outputs of the at least two causal predictive models into a causal ensemble model, by averaging the CATE outputs of the models; and weighting the CATE outputs, based on the respective computed / retrieved confidence intervals, The ensemble model can then output an ensemble CATE (eCATE) for each point in the multidimensional feature space. According to some embodiments, processing module 220 may be configured to fit the ensemble model on the same or a different dataset. Based on this, processing module 220 may identify one or more treatment subpopulations, based on the eCATE values and associated features and optionally generate a stratified clinical study design comprising sample size estimates, inclusion / exclusion criteria, and stratification factors for each identified subpopulation. System 200 further includes a display module 230 (also referred to as “display”) Display module 230 is configured to present the identified subpopulations and their distinguishing features to a user further comprising a user interface configured to allow a user to interact and affect changes to one or more subgroups defined by the features defining the multidimensional feature space. Display 240 may also feature a user interface module 240 (also referred to simply as user interface or UI). UI 240 is configured to allow a user to interact and affect changes to one or more subgroups defined by the features defining the multidimensional feature space. It is understood that since all modules are functionally interconnected information may be passed back and forth between the different modules. For example, if a user makes changes (e.g. to the sample size or addition of an endpoint) processing module 220 may reidentify one or more treatment subpopulations and / or regenerate the clinical study design, based on the affected changes.

[0176] Reference is now made to FIG. 3a-3c, which show exemplary user interfaces (310- 330) displayed on a display. As seen from the display, a user can via the UI select covariates to explore effect on subgroups (FIG. 3a), Select model to review influence on treatment effect (FIG. 3b) and select stratification covariates to identify correlations (FIG. 3c). [no sure how relevant this is to this application, or rather PSV 001]

[0177] The following examples are presented in order to more fully illustrate some embodiments of the invention. They should in no way be construed, however, as limiting the broad scope of the invention. One skilled in the art can readily devise many variations and modifications of the principles disclosed herein without departing from the scope of the invention.

[0178] EXAMPLES

[0179] Example 1 - comparative study of CATE calculation

[0180] The focus of this study is a comprehensive analysis of the performance of different CATE estimation models in different scenarios. Since CATE is not directly observable in real patients, simulated data is required in order to test the performance in a valid and reliable manner. A simulation study on synthetic data, and a complementary analysis of real-world clinical trials data, were thus conducted. 1.1 Scenario simulation

[0181] In order to evaluate the robustness of the models, two data generating mechanisms, based on a linear model or a linear model with addition of nonlinear transformations and / or second order interactions between features were tested, representing three type of scenarios, namely: Linear scenarios, slightly nonlinear scenarios and highly nonlinear scenarios.

[0182] In addition, a second data-generating mechanism utilizing a mechanistic disease model of a main pathway or mechanism of action for a selected disease. Here, a simplified mechanistic model for the PD-L1 pathway in urothelial cancer was utilized. This model describes the connections between a few key factors related to the progression of urothelial tumors and specifically to their treatment using PD-L1 inhibitors.

[0183] 1.2 Estimators

[0184] Two types of estimators were applied, namely causal forests and meta-learners and base-learners spanning a wide range in complexity were compared:

[0185] • Boosted Regression Trees: A sum of trees, where each tree is fitted on the residuals of the previous one.

[0186] • Accurate GLM (AGLM): This method utilizes Lasso regression to fit a piece-wise constant function, by first encoding each variable into nested bins.

[0187] • Random Forests (RF): An average of trees, constructed with stochastic sampling of features and sample to induce variability in the trees, which acts as a form of regularization.

[0188] • Bayesian Additive Regression Trees (BART): with boosting in particular, BART also trains a sum-of-trees model, with the addition of a regularization prior which controls the parameters of that model.

[0189] • Non-parametric methods such as gaussian process regression, or spline regression.

[0190] • Neural Networks algorithms, such as TAR-net

[0191] These estimators were compared to interaction testing, which is the traditional approach used to analyze heterogeneity and identify subgroups in clinical data. The abovementioned estimators, as well as the interaction testing were further compared to the herein disclosed ensemble model, also referred to as a CATE ensemble model, integrating the causal forest, AGLM and BART. Two types of integrations were tested namely, integration based on a simple mean of the predictions (results not shown), and a Consensus Based Averaging (CBA) approach, which identifies a cluster of “most concordant” models, and averages only them.

[0192] 1.3 Model preparation

[0193] In contrast to outcome prediction, where it is possible to compare the predicted outcome for an observation to its measured outcome, in the case of CATE estimation, this is impossible due to the fact that the treatment effect cannot be directly measured for any patient, since only one potential outcome can be observed for each patient. This was solved inter alia by using synthetic datasets to train / fit and test models, which allows us to compare CATE estimation to the real (known) CATE for any individual of interest

[0194] In short, the models were either trained on a training data set or, as in this case, fitted to the dataset, and its performance evaluated on a held-out test set generated using the same synthetic data generating process (DGP) by Root Mean Squared Error RMSE. RMSE measures the distance between predictions and actual CATE. The RMSE was scaled by the standard deviation of the CATE.

[0195] 1.4 Results

[0196] As seen from FIG. 4, none of the single estimators had good performance (compared to others) in all scenarios, and every single estimator failed significantly in at least one of the scenario types. This, as opposed to the CATE ensemble model, which performed remarkably well in all scenarios, in particular in highly non-linear scenarios and biological models.

[0197] These results clearly demonstrate the superiority of the herein disclosed model in deriving reliable and robust CATEs from clinical studies and thus for providing a reliable prediction of treatment efficacy. Hence, the herein disclosed CATE ensemble model leverages the strengths of multiple causal inference techniques and ensures superior performance and capability of handling complex, heterogeneous datasets effectively.

[0198] The ability of the herein disclosed ensemble algorithm to account for patient heterogeneity significantly improves the analysis of clinical trials and thus bolsters the chances of drug approval. Moreover, by providing a more accurate assessment of how different patient groups will respond to treatments, the herein disclosed ensemble approach enables personalized medicine strategies that are more effective and efficient. This not only optimizes resource use but also improves patient care by tailoring treatments to individual needs, thus advancing the field of personalized medicine.

[0199] While certain embodiments of the invention have been illustrated and described, it will be clear that the invention is not limited to the embodiments described herein. Numerous modifications, changes, variations, substitutions and equivalents will be apparent to those skilled in the art without departing from the spirit and scope of the present invention as described by the claims, which follow.

Claims

CLAIMS1. A system for identifying one or more treatment subpopulations within a patient population of a clinical study, the system comprising:(a) a memory for storing a dataset comprising:• a measured treatment response for each individual in at least one drug-treated patient group;• a measured treatment response for each individual in a control-treated patient group; and• a counterfactual treatment response for each individual in the at least one drug- treated patient group;• a counterfactual treatment response for each individual in the control-treated patient group;• for each individual, a plurality of features defining a multidimensional feature space;(b) a processor coupled to the memory and configured to execute instructions that cause the system to:(i) fit at least two different causal predictive models to the dataset or to a computed hypothetical individual treatment effect (ITE) for each individual in the dataset, to output a Conditional Average Treatment Effect (CATE) for each point in the multidimensional feature space;(ii) for each causal predictive model, compute a model- specific confidence interval associated with the respective CATE output;(iii) convert the outputs of the at least two causal predictive models into a causal ensemble model, the converting comprising:• averaging the CATE outputs of the models; and• weighting the CATE outputs based on the respective confidence intervals, wherein the causal ensemble model is configured to output an ensemble CATE (eCATE) for each point in the multidimensional feature space;(iv) identify one or more treatment subpopulations, based on the eCATE values and associated features.

2. The system of claim 1, wherein the model-specific confidence interval is computed by the causal predictive model during fitting or wherein the model- specific confidence interval is computed using conformal prediction.

3. The system of claim 1 or 2, further comprising a display configured to present the identified subpopulations and their distinguishing features to a user.

4. The system of any one of claims 1-3, further comprising a user interface configured to allow a user to interact and affect changes to one or more subgroups defined by the features defining the multidimensional feature space.

5. The system of claim 4, wherein the processor is further configured to reidentify one or more treatment subpopulations based on the affected changes.

6. The system of claim 4 or 5, wherein the processor is further configured to regenerate the clinical study design based on the affected changes.

7. The system of any one of claims 1-6, wherein the processor is configured to fit the ensemble model on a same or different dataset.

8. The system of any one of claims 1-7, wherein the system is configured to operate on clinical datasets having more covariates than samples.

9. The system of any one of claims 1-8, wherein the ensemble model provides improved CATE estimation accuracy over individual causal predictive model applied in isolation.

10. The system of any one of claims 1-9, wherein at least one of the causal predictive models is a meta-leamer selected from: S -learner, T-learner, X-learner, or DR-learner.

11. The system of any one of claims 1-10, wherein the causal predictive models are selected from a causal forest model, an accurate generalized linear model (AGLM), a Bayesian additive regression trees (BART) model, causal BART or any combination thereof.

12. The system of any one of claims 1-11, wherein the causal ensemble model comprises at least three different causal predictive models with distinct underlying learning algorithms.

13. The system of any one of claims 1-12, wherein the processor is configured to compute the estimated counterfactual treatment responses for each individual in the at least one drug-treated patient group and for each individual in the control-treated patient group and to send them to the memory for storage.

14. The system of any one of claims 1-13, wherein the processor is configured to compute the hypothetical individual treatment effect (ITE) for each individual in the dataset, based on a difference between the measured treatment response and the counterfactual treatment response of each individual in the dataset and to store in the memory.

15. The system of any one of claims 1-14, wherein the dataset is selected from: a clinical trial dataset, an observational study dataset, a real-world evidence dataset, or any combination thereof.

16. The system of any one of claims 1-15, wherein the identified treatment subpopulation comprises patients expected to exhibit a positive response to a specified treatment arm.

17. The system of any one of claims 1-16, further configured to identify subpopulations exhibiting no treatment effect and flag such subpopulations for exclusion in future trial planning.

18. A computer-implemented method for identifying one or more treatment subpopulations within a patient population of a clinical study, the method comprising: a) obtaining a dataset comprising: o a measured treatment response for each individual in at least one drug- treated patient group; o a measured treatment response for each individual in a control-treated patient group; and o a counterfactual treatment response for each individual in the at least one drug-treated patient group;o a counterfactual treatment response for each individual in the control- treated patient group; o for each individual, a plurality of features defining a multidimensional feature space; b) fitting, by a processor, at least two different causal predictive models to the dataset or to a computed hypothetical individual treatment effect (ITE) for each individual in the dataset, to output a Conditional Average Treatment Effect (CATE) for each point in the multidimensional feature space; c) computing, by the processor, a model- specific confidence interval for each of the at least two causal predictive models; d) converting, by the processor, the CATE outputs into a causal ensemble model, the converting comprising:• averaging the CATE outputs; and• weighting the CATE outputs according to the respective computed confidence intervals; wherein the causal ensemble model is configured to output an ensemble CATE (eCATE) for each point in the multidimensional feature space; and e) identifying, by the processor, one or more treatment subpopulations within the patient population, based on the eCATE values and their associated features.

19. The method of claim 18, further comprising generating a stratified clinical study design comprising sample size estimates, inclusion or exclusion criteria, or stratification factors for each identified subpopulation.

20. The method of claim 18 or 19, wherein the computer- implemented method is suitable for use on clinical datasets having more covariates than samples.

21. The method of any one of claims 18-20, wherein at least one of the causal predictive models is a meta-leamer selected from: S -learner, T-learner, X-learner, or DR-learner.

22. The method of any one of claims 18-21, wherein the causal predictive models are selected from a causal forest, an accurate generalized linear model (AGLM), and a Bayesian additive regression trees (BART) model.

23. The method of any one of claims 18-22, wherein the causal ensemble model comprises at least three causal predictive models trained using distinct machine learning frameworks.

24. The method of any one of claims 18-23, wherein the model- specific confidence interval is computed by the causal predictive model during fitting or wherein the model- specific confidence interval is computed using conformal prediction.

25. The method of any one of claims 18-24, further comprising presenting the identified subpopulations and their distinguishing features to a user via a display.

26. The method of any one of claims 18-25, further comprising enabling a user to interact and affect changes to one or more subgroups defined by the features defining the multidimensional feature space via a user interface.

27. The method of claim 26, further comprising reidentifying one or more treatment subpopulations based on the affected changes.

28. The method of claim 26 or 27, wherein the further comprising regenerating the clinical study design based on the affected changes.

29. The method of any one of claims 18-28, wherein the dataset is selected from: a clinical trial dataset, an observational dataset, a real- world dataset, or any combination thereof.

30. The method of any one of claims 18-29, wherein the identified treatment subpopulation comprises patients expected to exhibit a statistically significant response to a given treatment arm.

31. The method of any one of claims 18-30, further comprising identifying one or more subpopulations expected to exhibit no treatment response and excluding them from the generated study design.

32. The method of any one of claims 18-31, further comprising computing the hypothetical individual treatment effect (ITE) for each individual, based on the difference between the measured treatment response and a counterfactual prediction.

33. A computer- implemented method for identifying treatment-responsive subpopulations in a clinical study, comprising: a) receiving, at a computing system, a dataset comprising:• a measured treatment response for each individual in at least one drug-treated patient group;• a measured treatment response for each individual in a control-treated patient group; and• a counterfactual treatment response for each individual in the at least one drug- treated patient group;• a counterfactual treatment response for each individual in the control-treated patient group; b) for each individual, a plurality of features defining a multidimensional feature space c) fitting, by the computing system, at least two different causal inference models to compute respective estimates of the Conditional Average Treatment Effect (CATE) for each subject; d) computing, for each model, a confidence indicator associated with the respective CATE estimate; e) computing an ensemble treatment effect estimate (eCATE) for each subject by combining the CATE outputs based on the confidence indicators; f) identifying, based on the eCATE values and feature profiles, one or more subpopulations predicted to respond to treatment; and g) outputting the identified subpopulations for use in clinical trial stratification, subject recruitment, or treatment planning.

Images