Predicting therapeutic clinical trial enrollment for adult patients with low- and high-grade glioma using supervised machine learning

A supervised machine learning method using ensemble models analyzes patient-level variables to optimize clinical trial enrollment, addressing enrollment disparities for glioma patients, enhancing research generalizability and equitable healthcare resource allocation.

WO2026236385A1PCT designated stage Publication Date: 2026-11-12RGT UNIV OF CALIFORNIA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/US2026/026934
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-09
Filing Date
2026-05-06
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Clinical trial enrollment rates for low- and high-grade glioma disproportionately lag behind incidence and mortality patterns, particularly among women and minority patients, due to nuanced interactions between demographic, socioeconomic, and prognostic variables that are not adequately understood, leading to enrollment disparities.

Method used

A supervised machine learning approach using ensemble models, such as boosted neural networks, to analyze patient-level variables and determine their relative influence on clinical trial enrollment, enabling targeted recruitment strategies to address enrollment barriers for underrepresented groups.

Benefits of technology

Enhances the generalizability of medical research by optimizing clinical trial participant demographics, ensuring equitable healthcare resource allocation, and facilitating the discovery of therapeutic targets for precision medicine.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure US2026026934_12112026_PF_FP_ABST
    Figure US2026026934_12112026_PF_FP_ABST
Patent Text Reader

Abstract

Methods and systems for optimizing clinical trial participant demographics are disclosed herein. Novel modeling approaches, capturing the complex interactions between demographic, socioeconomic, and prognostic variables in a manner reflecting how such variables influence decisions made at the individual patient level, are provided in order to determine the relative influence of each variable on clinical trial enrollment for various distinct patient subgroups. Further, methods for translating model-derived insights into targeted recruitment strategies are also disclosed. Thus, the methods and systems of the invention, e.g., as described in greater detail below, find use in a variety of applications where it is desirable to enhance medical research generalizability, support equitable access to healthcare resources, and facilitate the discovery of therapeutic targets and interventions useful for precision medicine.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] PREDICTING THERAPEUTIC CLINICAL TRIAL ENROLLMENT FOR ADULT PATIENTS WITH LOW- AND HIGH-GRADE GLIOMA USING SUPERVISED MACHINE LEARNING CROSS-REFERENCE TO RELATED APPLICATION

[0002] Pursuant to 35 U.S.C. § 119(e), this application claims priority to the filing dates of United States Provisional Patent Application Serial No. 63 / 802,971 filed May 9, 2025, the disclosure of which application is herein incorporated by reference in its entirety.

[0003] INTRODUCTION

[0004] Clinical trials offer high quality medical care and help preserve hope for patients with hard to treat medical conditions, such as cancer. Yet, despite disease-specific accrual benchmarks introduced by the NIH Revitalization Act of 1993 to improve representation, national clinical trial enrollment rates have yet to match disease incidence and mortality across demographic groups. For example, recent studies have revealed alarming disparities in trial participation among women and minority patients with low- and high-grade glioma — with enrollment rates significantly lagging behind incidence and mortality patterns. Recently, researchers have begun to ascertain various independent factors that may potentially influence clinical trial participation. Unfortunately, understanding of the nuanced interactions between these factors has remained limited — impeding efforts to identify and address enrollment obstacles for underrepresented patient populations.

[0005] SUMMARY

[0006] Thus, there is a critical need for improved and useful methods and systems for identifying the various factors driving clinical trial participation across patient populations. Particularly, methods accounting for the subtle and complex interactions between these variables are needed in order to determine how each variable differentially affects clinical trial participation among distinct patient subgroups.

[0007] This invention provides such new and useful methods and systems, addressing the limitations mentioned above. To accomplish this, the invention utilizes a unique blend of neuroscientific and machine learning expertise — enabling the creation of novel models capturing the nuanced interactions between demographic, socioeconomic, and prognostic variablesinfluencing clinical trial enrollment in a manner that mirrors how such factors influence complex clinical trial enrollment decisions for an individual patient. Further, methods for translating model-derived insights into targeted recruitment strategies are provided — allowing for enrollment barriers unique to specific patient subgroups to be both precisely identified and effectively addressed. Thus, the methods and systems of the invention find use in a variety of applications where it is desirable to optimize clinical trial participant demographics for the purpose of enhancing the generalizability of medical research, supporting the equitable allocation of healthcare resources, and facilitating the scientific discovery of therapeutic targets and interventions useful for precision medicine.

[0008] In one aspect, methods (such as, e.g., computer-implemented methods) of optimizing clinical trial participant demographics are provided. Aspects of the methods include: obtaining values for multiple patient-level variables for each of a plurality of individuals associated with the clinical trial, the patient-level variables including social identifier variables and prognostic variables, wherein the plurality of individuals includes at least a first subpopulation and a second subpopulation of individuals having different values of a primary social identifier variable; determining the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals using a first ensemble machine learning (ML) model, wherein the first ensemble ML model is trained using a dataset including individuals from each subpopulation of the plurality of individuals; determining the relative influence of the patientlevel variables on clinical trial enrollment for individuals of the second subpopulation defined by the primary social identifier variable using a second ensemble ML model, wherein the second ensemble ML model is trained using a dataset including individuals of the second subpopulation; and optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first and second ensemble ML models. In some embodiments, the multiple social identifier variables include demographic variables and socioeconomic variables. In some embodiments, the second ensemble ML model is trained using a dataset consisting of individuals of the second subpopulation. In some embodiments, the ensemble ML models are boosted neural networks (BNNs).

[0009] In certain embodiments, the primary social identifier variable includes a biologically based social identifier variable. For example, the primary social identifier variable may include age, race, or biological sex. In some embodiments, the plurality of clinical trial associatedindividuals have been diagnosed with a health condition addressed by the clinical trial. In these instances, the clinical trial may be a therapeutic clinical trial. In some embodiments, the plurality of clinical trial associated individuals have a known status of clinical trial participation, wherein at least a portion of the individuals have participated in a clinical trial addressing the health condition (e.g., addressed by the clinical trial for which participant demographics are optimized). In some embodiments, the health condition includes cancer, wherein the prognostic variables include oncologic variables. For example, the health condition may be a glioma. In some embodiments, the oncologic variables include one or more of Kamofsky Performance Status (KPS), tumor grade, tumor type, tumor hemisphere and / or lobe, receipt of chemotherapy and / or radiation, extent of resection (EOR), and seizure presentation.

[0010] In some embodiments, the multiple social identifier variables include a plurality of socioeconomic variables, the socioeconomic variables including one or more of insurance status, employment status, occupation classification, household income, distance from clinical trial facility, and interpreter use. In some embodiments, the multiple social identifier variables include a plurality of demographic variables, the demographic variables including one or more of biological sex, race, ethnicity, age, and preferred language.

[0011] In certain embodiments, one or more of the patient-level variables are derived from electronic health record (EHR) data obtained for each of the plurality of individuals. In these instances, the method may further include determining one or more patient-level variable values based on proxy indicators of the variables found in the EHR data. In some embodiments, the primary social identifier variable includes race, the first subpopulation of individuals defined by the primary social identifier variable includes individuals of a majority racial group, and the second subpopulation of individuals defined by the primary social identifier variable includes individuals of one or more racial groups other than the majority racial group. In these instances, the second subpopulation of individuals defined by the primary social identifier may include one or more racial groups underrepresented in clinical trial enrollment.

[0012] In certain embodiments, the plurality of individuals includes at least a first subpopulation and a second subpopulation of individuals having different values of a secondary social identifier variable. In these instances, the method may further include: determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of the first subpopulation defined by the secondary social identifier variable using a third ensemble ML model, wherein thethird ensemble ML model is trained using a dataset including individuals of the first subpopulation defined by the secondary social identifier variable; and optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first, second, and third ensemble ML models. In some embodiments, the secondary social identifier variable includes age, race, or biological sex. In some embodiments, the secondary social identifier variable includes biological sex and the first subpopulation of individuals defined by the secondary social identifier variable includes biological females.

[0013] In certain embodiments, the method further includes: identifying a plurality of target subpopulations of interest, each defined by a distinct set of social identifier variable values; determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of each target subpopulation using a corresponding ensemble ML model, wherein each ensemble ML model is trained using a dataset including individuals of the respective target subpopulation; and optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by the ensemble ML models for each target subpopulation. In these instances, the plurality of target subpopulations may be identified based on disparities between the incidence of the health condition among subpopulations defined by social identifier variable values and the corresponding enrollment patterns observed for those subpopulations in clinical trials addressing the health condition. In some embodiments, the method further includes identifying the plurality of target subpopulations from the multiple patient-level variables obtained for each of the plurality of clinical trial associated individuals.

[0014] In certain embodiments, the patient-level variables include a screening status indicating whether an individual has been screened for a clinical trial addressing the health condition. In these instances, the patient-level variables may further include the number of clinical trials discussed with an individual. In some instances, for each ensemble ML model used to determine the relative influence of patient-level variables on clinical trial enrollment, the method further includes training a corresponding ensemble ML model to determine the relative influence of the patient-level variables on clinical trial screening for the same subpopulation. In some embodiments, the method further includes: determining the relative influence of the patient-level variables on clinical trial screening for individuals of each subpopulation using each screening ensemble ML model; and optimizing recruitment of individuals for the clinical trial based on therelative influence of the patient-level variables determined by each screening and enrollment ensemble ML model.

[0015] In certain embodiments, method further includes preprocessing the patient-level variables. In these instances, the patient-level variables may be preprocessed by: one-hot encoding each categorical variable into a set of binary variables; and transforming each continuous variable to reduce the influence of outliers or skewed distributions. In some instances, each continuous variable is transformed to approximate a normal distribution using a Johnson Su or a Johnson Sb distribution. In some embodiments, the preprocessing includes, for each ensemble ML model, filtering the patient-level variables to remove variables that are highly correlated with other variables in predicting enrollment outcomes. In these cases, the highly correlated variables may be determined using correlation matrix analysis.

[0016] In certain embodiments, wherein each ensemble ML model includes a BNN, the BNN including five or more component models trained using boosting techniques such as, e.g., ten or more component models. In some embodiments, the first layer of the hidden layer structure of each BNN includes a plurality of different types of nodes, wherein the plurality of different types of nodes are configured to capture a plurality of different types of effects of patient-level variables. In some instances, the first layer of each component model of each BNN includes a plurality of different types of nodes.

[0017] In certain embodiments, the method further includes training each ensemble ML model. In some embodiments, for each enrollment ensemble ML model, the training includes: sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial enrollment based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; and minimizing a loss function including a squared penalty function during training of each component model, the squared penalty function including a squared error between predicted and actual enrollment outcomes. In other embodiments, for each screening ensemble ML model, the training includes: sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial screening based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; and minimizing a loss function including a squaredpenalty function during training of each component model, the squared penalty function including a squared error between predicted and actual screening outcomes.

[0018] In certain embodiments, the training is configured to preserve the predictive influence of multiple patient-level variables during training. In some embodiments, the plurality of individuals associated with the clinical trial includes individuals whose distribution of social identifier variable values approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial. In some embodiments, the training includes performing k-fold cross-validation. In some embodiments, the training includes removing patient-level variables having an estimated influence below a predetermined threshold from consideration in the final model. For example, the removing may include performing backward elimination. In some instances, the dataset used to train each ensemble ML model consists only of individuals from the subpopulation for which the model is configured to determine the relative influence of patient-level variables on clinical trial screening or enrollment.

[0019] In certain embodiments, determining the relative influence of each patient-level variable for an ensemble ML model includes calculating a quantitative influence metric for each patientlevel variable. In these instances, the quantitative influence metric may include a Shapley-based value representing the average contribution of each patient-level variable to model predictions. In some embodiments, the method further includes using the trained ensemble ML models to determine an individual patient’s probability of clinical trial enrollment based on the patientlevel variables of the patient.

[0020] In certain embodiments, optimizing recruitment of individuals for the clinical trial includes allocating recruitment resources based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models. In some embodiments, optimizing recruitment of individuals for the clinical trial includes generating a recruitment plan based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models. In some embodiments, the method further includes generating a recruitment report based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models. In these instances, the recruitment report may include one or more of a summary of differences in the relative influence of patient-level variables between the overall population cohort determined by the first ensemble ML model andeach of the one or more subpopulations, a recommended allocation of recruitment resources, and a proposed recruitment plan. In some instances, the recruitment report is generated using a natural language processing (NLP) model such as, e.g., a large language model (LLM).

[0021] In certain embodiments, the participant demographics of the clinical trial are optimized such that the distribution of social identifier variable values among recruited individuals better approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial. In some embodiments, the participant demographics of the clinical trial are optimized in order to better represent one or more subpopulations underrepresented in prior clinical trials addressing the health condition. In some instances, the participant demographics of the clinical trial are optimized to comply with one or more regulatory or institutional demographic guidelines for clinical trial enrollment. In some embodiments, the participant demographics of the clinical trial are optimized to increase biological heterogeneity among trial participants.

[0022] Aspects of the present disclosure also include systems (such as, e.g., computer systems) for performing the methods of optimizing clinical trial participant demographics, e.g., as described above and herein. Systems of interest include: a processor including memory operably coupled to the processor, wherein the memory includes instructions stored thereon, which, when executed by the processor, cause the processor to: obtain values for multiple patient-level variables for each of a plurality of individuals associated with the clinical trial, the patient-level variables including social identifier variables and prognostic variables, wherein the plurality of individuals includes at least a first subpopulation and a second subpopulation of individuals having different values of a primary social identifier variable; determine the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals using a first ensemble machine learning (ML) model, wherein the first ensemble ML model is trained using a dataset including individuals from each subpopulation of the plurality of individuals; determine the relative influence of the patient-level variables on clinical trial enrollment for individuals of the second subpopulation defined by the primary social identifier variable using a second ensemble ML model, wherein the second ensemble ML model is trained using a dataset including individuals of the second subpopulation; and optimize recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first and second ensemble ML models.In certain embodiments, the instructions, when executed by the processor, further cause the processor to train each ensemble ML model using an initiation component. In some embodiments, the instructions, when executed by the processor, further cause the processor to generate a recruitment report based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models using an integration component. In these instances, the integration component may include an NLP model such as, e.g., and LLM.

[0023] In another aspect, computer program products, including instructions (e.g., in the form of lines of computer code) which, when executed by a computer, cause the computer to implement the methods of optimizing clinical trial participant demographics, and non-transitory computer readable storage media including the instructions of the memory of the systems and / or the computer products, are also provided.

[0024] BRIEF DESCRIPTION OF THE FIGURES FIGS. 1A to II depict receiver operator curves and ranked variables of importance for boosted neural network (BNN) models predicting clinical trial enrollment for a general cohort, a women only cohort, and a minority only cohort, in accordance with embodiments of the disclosure.

[0025] FIG. 2 illustrates Shapley values for the influential predictors of a whole cohort BNN model, in accordance with embodiments of the disclosure.

[0026] FIG. 3 provides a schematic for BNN development and validation for a whole cohort, women only cohort, and minority only cohort of patients associated with a clinical trial or, i.e., patients having a health condition pertinent to the condition being studied in the clinical trial, in accordance with embodiments of the disclosure.

[0027] FIGS. 4A to 41 depict receiver operator curves and ranked variables of importance for BNN models predicting clinical trial screening for a general cohort, a women only cohort, and a minority only cohort, in accordance with embodiments of the disclosure.

[0028] FIGS. 5A to 5B depict the receiver operator curve and the ranked variables of importance for a combined minority BNN model, developed from the combined minority patients of a development and a validation cohort of individuals, in accordance with embodiments of the disclosure.FTG. 6 illustrates Shapley values for the influential predictors of a women only cohort BNN model, in accordance with embodiments of the disclosure.

[0029] FIG. 7 illustrates Shapley values for the influential predictors for a minority only cohort BNN model, in accordance with embodiments of the disclosure.

[0030] FIG. 8 provides a table of demographic, socioeconomic, and oncologic characteristics of a development cohort and a validation cohort of individuals used to generate training and external validation datasets for a number of ensemble machine learning (ML) models, in accordance with embodiments of the disclosure.

[0031] FIG. 9 shows the results of univariate and multivariate logistic regression for factors associated with therapeutic clinical trial enrollment for a general cohort of relevant clinical trial associated patients, in accordance with embodiments of the disclosure.

[0032] FIG. 10 provides a table of demographic, socioeconomic, and oncologic characteristics of various subsets of a validation cohort of individuals used to generate external validation datasets for a number of ensemble machine learning models, in accordance with embodiments of the disclosure.

[0033] FIG. 11 provides a table of demographic, socioeconomic, and oncologic characteristics of women and minority subsets of a validation cohort of individuals used to generate external validation datasets for ensemble machine learning models, in accordance with embodiments of the disclosure.

[0034] FIG. 12 shows the results of univariate and multivariate logistic regression analysis for factors associated with clinical trial enrollment among those screened for a clinical trial, in accordance with embodiments of the disclosure.

[0035] FIG. 13 demonstrates an association between sex, minority status, and preferred language and the number of clinical trials discussed with patients during trial screening found using methods according to embodiments of the disclosure.

[0036] FIG. 14 shows the results of univariate and multivariate logistic regression for factors associated with screening for a clinical trial for a general cohort of relevant clinical trial associated patients, in accordance with embodiments of the disclosure.

[0037] FIG. 15 shows the results of univariate and multivariate logistic regression for factors associated with clinical trial enrollment for women only minority only subsets of a development cohort, in accordance with embodiments of the disclosure.FTG. 16 provides a comparison of nine machine learning techniques, including ensemble ML techniques, for prediction of clinical trial enrollment, according to embodiments of the disclosure.

[0038] FIG. 17 provides a flow chart depicting a method of optimizing clinical trial participant demographics, according to embodiments of the disclosure.

[0039] FIG. 18 illustrates a system of BNN models for optimizing clinical trial participant demographics, in accordance with embodiments of the disclosure.

[0040] FIG. 19 shows a tool for estimating patient enrollment probabilities for a given therapeutic clinical trial based on patient-level variables input by a user, in accordance with embodiments of the disclosure.

[0041] FIGS. 20A to 20B demonstrates use of a database to obtain socioeconomic variables for an individual, according to embodiments of the disclosure.

[0042] FIGS. 21A to 21D demonstrates use of a tool for estimating patient enrollment probabilities for a given therapeutic clinical trial based on patient-level variables input by a user, according to embodiments of the disclosure.

[0043] DETAILED DESCRIPTION

[0044] Methods and systems for optimizing clinical trial participant demographics are disclosed herein. Novel modeling approaches, capturing the complex interactions between demographic, socioeconomic, and prognostic variables in a manner reflecting how such variables influence decisions made at the individual patient level, are provided in order to determine the relative influence of each variable on clinical trial enrollment for various distinct patient subgroups. Further, methods for translating model-derived insights into targeted recruitment strategies are also disclosed. Thus, the methods and systems of the invention, e.g., as described in greater detail below, find use in a variety of applications where it is desirable to enhance medical research generalizability, support equitable access to healthcare resources, and facilitate the discovery of therapeutic targets and interventions useful for precision medicine.

[0045] Before the present invention is described in greater detail, it is to be understood that this invention is not limited to particular embodiments described, as such may, of course, vary. It is also to be understood that the terminology used herein is for the purpose of describing particularembodiments only, and is not intended to be limiting, since the scope of the present invention will be limited only by the appended claims.

[0046] Where a range of values is provided, it is understood that each intervening value, to the tenth of the unit of the lower limit unless the context clearly dictates otherwise, between the upper and lower limit of that range and any other stated or intervening value in that stated range, is encompassed within the invention. The upper and lower limits of these smaller ranges may independently be included in the smaller ranges and are also encompassed within the invention, subject to any specifically excluded limit in the stated range. Where the stated range includes one or both of the limits, ranges excluding either or both of those included limits are also included in the invention.

[0047] Certain ranges are presented herein with numerical values being preceded by the term “about.” The term “about” is used herein to provide literal support for the exact number that it precedes, as well as a number that is near to or approximately the number that the term precedes. In determining whether a number is near to or approximately a specifically recited number, the near or approximating unrecited number may be a number which, in the context in which it is presented, provides the substantial equivalent of the specifically recited number.

[0048] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention, representative illustrative methods and materials are now described.

[0049] All publications and patents cited in this specification are herein incorporated by reference as if each individual publication or patent were specifically and individually indicated to be incorporated by reference and are incorporated herein by reference to disclose and describe the methods and / or materials in connection with which the publications are cited. The citation of any publication is for its disclosure prior to the filing date and should not be construed as an admission that the present invention is not entitled to antedate such publication by virtue of prior invention. Further, the dates of publication provided may be different from the actual publication dates, which may need to be independently confirmed.

[0050] It is noted that, as used herein and in the appended claims, the singular forms “a”, “an”, and “the” include plural referents unless the context clearly dictates otherwise. It is further notedthat the claims may be drafted to exclude any optional element. As such, this statement is intended to serve as antecedent basis for use of such exclusive terminology as “solely,” “only” and the like in connection with the recitation of claim elements, or use of a “negative” limitation.

[0051] As will be apparent to those of skill in the art upon reading this disclosure, each of the individual embodiments described and illustrated herein has discrete components and features which may be readily separated from or combined with the features of any of the other several embodiments without departing from the scope or spirit of the present invention. Any recited method can be carried out in the order of events recited or in any other order which is logically possible.

[0052] While the apparatus and method has or will be described for the sake of grammatical fluidity with functional explanations, it is to be expressly understood that the claims, unless expressly formulated under 35 U.S.C. §112, are not to be construed as necessarily limited in any way by the construction of “means” or “steps” limitations, but are to be accorded the full scope of the meaning and equivalents of the definition provided by the claims under the judicial doctrine of equivalents, and in the case where the claims are expressly formulated under 35 U.S.C. §112 are to be accorded full statutory equivalents under 35 U.S.C. §112.

[0053] METHODS

[0054] As summarized above, (such as, e.g., computer-implemented methods) for optimizing clinical trial participant demographics are provided. Aspects of the methods include: obtaining values for multiple patient-level variables for each of a plurality of individuals associated with the clinical trial, the patient-level variables including social identifier variables and prognostic variables, wherein the plurality of individuals includes at least a first subpopulation and a second subpopulation of individuals having different values of a primary social identifier variable; determining the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals using a first ensemble machine learning (ML) model, wherein the first ensemble ML model is trained using a dataset including individuals from each subpopulation of the plurality of individuals; determining the relative influence of the patientlevel variables on clinical trial enrollment for individuals of the second subpopulation defined by the primary social identifier variable using a second ensemble ML model, wherein the second ensemble ML model is trained using a dataset including individuals of the second subpopulation;and optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first and second ensemble ML models. In some embodiments, the multiple social identifier variables include demographic variables and socioeconomic variables. In some embodiments, the second ensemble ML model is trained using a dataset consisting of individuals of the second subpopulation. In some embodiments, the ensemble ML models are boosted neural networks (BNNs).

[0055] The terms “subject”, “individual”, “patient”, and “participant” are used interchangeably herein and may refer, e.g., to a person associated with the clinical trial. Depending on context, these terms may refer, e.g., to: a person for whom patient-level variable values are obtained — such as to train or apply the ensemble ML models described herein; a person diagnosed with the health condition addressed by the clinical trial and thus eligible for inclusion in the plurality of individuals modeled; a member of a target subpopulation whom a generated recruitment strategy (i.e., for optimizing clinical trial participant demographics) aims to reach for clinical trial enrollment; or a person for whom the likelihood of clinical trial screening or enrollment is predicted. The subject is preferably human, e.g., a child, an adolescent, or an adult (such as a young, middle-aged, or elderly adult) human. In some cases, the subject is sixty years of age or older. In other cases, the subject is younger than sixty years of age.

[0056] As set forth above, embodiments of the methods include obtaining values for multiple patient-level variables for each of a plurality of individuals associated with the clinical trial, the patient-level variables including social identifier variables and prognostic variables. By “obtain” is meant to make the multiple patient-level variable values (referred to interchangeably herein as, e.g., the multiple patient-level variables, or simply the patient-level variables) accessible or available for the subsequent steps of the methods (e.g., available for use in training the ensemble ML models, etc.). The multiple patient-level variables may be obtained through any available means, and from any available source. Accordingly, “obtaining” the patient-level variable values (e.g., for each of the plurality of individuals) is intended to broadly encompass any manner of receiving, accessing, retrieving, determining, or otherwise acquiring such patient-level variables whether through, e.g., communication from an external source or device retrieval from memory, storage, or a data structure, generation or calculation by one or more components of a computer system (e.g., via a model, lookup table, or sensor data), user input, automated sensing, or any combination thereof. In certain embodiments, one or more of the patient-level variables arederived from electronic health record (EHR) data obtained for each of the plurality of individuals. In these instances, the method may further include determining one or more patientlevel variable values based on proxy indicators of the variables found in the EHR data. For example, patient zip codes may be used to determine income and poverty levels.

[0057] As used herein, a “patient-level variable” refers to any variable that describes a characteristic, attribute, or data point associated with an individual patient (e.g., associated with the clinical trial) or clinical trial candidate. Patient-level variables include both social identifier variables and prognostic variables, as well as any other variables that may influence or be used to predict clinical trial screening or enrollment. Patient-level variables may be categorical, ordinal, or continuous, and may be obtained from structured or unstructured data sources, including but not limited to electronic health records (EHRs), patient-reported outcomes, patient registries (e g., disease-specific clinical registries or institutionally maintained structured data sources), clinical trial management systems, or public health datasets.

[0058] As used herein, a “social identifier variable” refers to any variable that characterizes or categorizes an individual based on one or more socially recognized attributes (i.e., characteristics or classifications that are commonly understood, acknowledged, or used within a given society or cultural context to categorize individuals, whether formally or informally), including, but not limited to, demographic or socioeconomic factors. Social identifier variables may include, e.g., race, ethnicity, biological sex, gender identity, income level, education level, occupation, insurance status, housing stability, geographic region (e.g., ZIP code), primary language, immigration status, or other variables that serve as proxies or indicators of an individual’s social identity or position within a broader sociocultural, economic, or political context. In some embodiments, social identifier variables may be explicitly self-reported, derived from institutional records (e.g., EHR data), or inferred or approximated using statistical, rule-based, or machine learning classification or imputation techniques.

[0059] In some embodiments, the multiple social identifier variables include a plurality of socioeconomic variables, the socioeconomic variables including one or more of insurance status, employment status, occupation classification, household income, distance from clinical trial facility, and interpreter use. In some embodiments, the multiple social identifier variables include a plurality of demographic variables, the demographic variables including one or more of biological sex, race, ethnicity, age, and preferred language.As used herein, a “prognostic variable” refers to any variable that reflects or characterizes a biological, clinical, or health condition (e.g., disease) related condition of an individual that may influence, or be associated with, clinical trial enrollment, screening, or outcomes.

[0060] Prognostic variables may include, but are not limited to, diagnostic classifications (e.g., tumor grade, disease stage, histological subtype), clinical metrics (e.g., Karnofsky Performance Status [KPS], ECOG scores), biomarker levels, comorbid conditions, prior treatment history, performance test results, imaging-derived features, or any other health-related indicators. In some embodiments, prognostic variables may be obtained directly from clinical data sources such as EHRs, registries, or diagnostic systems, or may be derived through analysis (e.g., imaging, laboratory data, etc.).

[0061] In some embodiments, the plurality of clinical trial associated individuals have been diagnosed with a health condition addressed by the clinical trial. The term “health condition,” as used herein, refers to any disease or condition that impairs or affects the normal functioning of the body. In some embodiments, the health condition may be, e.g., an infectious disease, physiological disease, hereditary disease, autoimmune disease, psychiatric or mental health disorder, congenital disorder, traumatic injury, or a condition associated with aging. The health condition may arise from a variety of causes including, e.g., genetic predispositions, environmental exposures, lifestyle factors, age-related processes, or a combination thereof.

[0062] The health condition may vary in its level of specificity, encompassing broad categories that include multiple distinct subtypes (e.g., an umbrella diagnosis), or being highly specific with no further subclassifications. For example, the health condition may refer to a general disease classification, such as a metabolic disorder, or a narrowly defined condition, such as a specific monogenic disorder with a well-characterized molecular basis. Accordingly, the scope of the health condition may range from encompassing a diverse spectrum of related conditions to describing a singular, precisely defined medical entity. In some embodiments, one or more individuals of the plurality of individuals associated with the clinical trial (i.e., for whom multiple social identifier variables are obtained) may have been diagnosed with different forms of a broader category of the health condition addressed by the clinical trial. For example, if the clinical trial addresses a specific condition such as diffuse midline glioma (DMG), the associated individuals may include patients diagnosed with various glioma subtypes.In some embodiments, the health condition is a neurological disorder such as, e.g., a neurodegenerative disease (NDD). For example, the health condition might be, or include, one or more of: Alzheimer’s disease (AD), frontotemporal dementia (FTD), dementia with Lewy bodies (DLB), and the like. In some embodiments, the health condition is a form of cancer. In some cases, the health condition is a brain tumor such as, e.g., a malignant or a benign brain tumor. Malignant brain tumors may include, but are not limited to, gliomas, choroid plexus tumors, embryonal tumors, germ cell tumors, pineal tumors, meningiomas, nerve tumors, and pituitary tumors. In some embodiments, the health condition is a glioma such as, e.g., an astrocytoma, glioblastoma, oligodendroglioma, ependymoma, diffuse midline gliomas (DMG), diffuse intrinsic pontine gliomas (DIPG), etc.

[0063] As discussed above, in some embodiments, the health condition may include a form of cancer such as, e.g., a glioma. In these instances, the prognostic variables may include oncologic variables. In some embodiments, the oncologic variables include one or more of Karnofsky Performance Status (KPS), tumor grade, tumor type, tumor hemisphere and / or lobe, receipt of chemotherapy and / or radiation, extent of resection (EOR), and seizure presentation.

[0064] In some embodiments, the clinical trial may be a therapeutic clinical trial. As used herein, a “therapeutic clinical trial” refers to a clinical study in which enrolled patients receive one or more medical interventions intended to treat, manage, or modify the course of a diagnosed health condition. Such interventions may include pharmaceutical agents, biologies, medical devices, surgical procedures, or behavioral therapies administered with the goal of improving clinical outcomes or altering disease progression.

[0065] In certain embodiments, the primary social identifier variable includes a biologically based social identifier variable. By “biologically based” social identifier variable is meant a type of social identifier variable that is at least partially rooted in genetic, anatomical, or physiological characteristics. For example, the primary social identifier variable may include age, race, or biological sex. In some embodiments, the plurality of clinical trial associated individuals have a known status of clinical trial participation, wherein at least a portion of the individuals have participated in a clinical trial addressing the health condition (e.g., addressed by the clinical trial for which participant demographics are optimized).

[0066] In some embodiments, the primary social identifier variable includes race, the first subpopulation of individuals defined by the primary social identifier variable includesindividuals of a majority racial group, and the second subpopulation of individuals defined by the primary social identifier variable includes individuals of one or more racial groups other than the majority racial group. In these instances, the second subpopulation of individuals defined by the primary social identifier may include one or more racial groups underrepresented in clinical trial enrollment. In some embodiments, the second subpopulation of individuals includes individuals of all racial groups besides the majority racial group.

[0067] As discussed above, embodiments of the methods includes determining the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals using a first ensemble machine learning (ML) model, and for individuals of the second subpopulation defined by the primary social identifier variable using a second ensemble ML model. As used herein, “relative influence” refers to a quantitative or qualitative measure of the degree to which a given patient-level variable contributes to the prediction of a clinical outcome or behavior (e.g., screening or enrollment in a clinical trial) as determined by an ensemble ML model of the disclosure. Relative influence may be expressed in terms of model-derived metrics such as feature importance scores, Shapley values, partial dependence, or gradient-based sensitivity, and may be normalized across variables (or, e.g., across models) to facilitate comparison. In some embodiments, the relative influence of a patient-level variable is determined separately for a full population cohort and one or more target subpopulations to identify differences in how the variable affects model predictions across groups. In some embodiments, “relative influence” refers to a metric used to characterize how much impact a variable has on an ensemble ML model’s output (e.g., a screening or enrollment probability) relative to other input variables.

[0068] In certain embodiments, the plurality of individuals includes at least a first subpopulation and a second subpopulation of individuals having different values of a secondary social identifier variable. In these instances, the method may further include: determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of the first subpopulation defined by the secondary social identifier variable using a third ensemble ML model, wherein the third ensemble ML model is trained using a dataset including individuals of the first subpopulation defined by the secondary social identifier variable; and optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first, second, and third ensemble ML models. In someembodiments, the secondary social identifier variable includes age, race, or biological sex. In some embodiments, the secondary social identifier variable includes biological sex and the first subpopulation of individuals defined by the secondary social identifier variable includes biological females.

[0069] In some embodiments, the first ensemble ML model is used to determine the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals (i.e., across a general patient-wide population, or whole cohort, of clinical trial associated individuals), the second ensemble ML model is used to determine the relative influence of the patient-level variables on clinical trial enrollment for individuals of all racial groups besides the majority racial group (i.e., minorities), and the third ensemble ML model is used to determine the relative influence of the patient-level variables on clinical trial enrollment for biological females. In some embodiments, differences in the relative influence of the patientlevel variables determined by each of the ensemble ML models (e g., BNN models) may be used to guide targeted recruitment strategies for subpopulations represented by one or more of the models. For example, if insurance status is found to play a larger role in predicting enrollment among racial minority patients than in the general population, the system may recommend implementing outreach programs focused on, e.g., educating Medicare recipients in minority communities about clinical trial funding options.

[0070] In certain embodiments, the method further includes: identifying a plurality of target subpopulations of interest, each defined by a distinct set of social identifier variable values; determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of each target subpopulation using a corresponding ensemble ML model, wherein each ensemble ML model is trained using a dataset including individuals of the respective target subpopulation; and optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by the ensemble ML models for each target subpopulation. In these instances, the plurality of target subpopulations may be identified based on disparities between the incidence of the health condition among subpopulations defined by social identifier variable values and the corresponding enrollment patterns observed for those subpopulations in clinical trials addressing the health condition. In some embodiments, the method further includes identifying the plurality of target subpopulations from the multiple patient-level variables obtained for each of the plurality of clinical trial associated individuals.In certain embodiments, the patient-level variables include a screening status indicating whether an individual has been screened for a clinical trial addressing the health condition. In these instances, the patient-level variables may further include the number of clinical trials discussed with an individual. In some instances, for each ensemble ML model used to determine the relative influence of patient-level variables on clinical trial enrollment, the method further includes training a corresponding ensemble ML model to determine the relative influence of the patient-level variables on clinical trial screening for the same subpopulation. In other embodiments, a corresponding ML model for determining the relative influence of the patientlevel variables on clinical trial screening is trained only for a subpopulation in which screening is found to have a greater than threshold-level influence on clinical trial enrollment. In some embodiments, for subpopulations for which a screening ensemble model is trained, the method further includes: determining the relative influence of the patient-level variables on clinical trial screening for individuals of each subpopulation using each screening ensemble ML model; and optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by each screening and enrollment ensemble ML model.

[0071] In certain embodiments, method further includes preprocessing the patient-level variables. In these instances, the patient-level variables may be preprocessed by: one-hot encoding each categorical variable into a set of binary variables; and transforming each continuous variable to reduce the influence of outliers or skewed distributions. In some instances, each continuous variable is transformed to approximate a normal distribution using a Johnson Su or a Johnson Sb distribution. In some embodiments, the preprocessing includes, for each ensemble ML model, filtering the patient-level variables to remove variables that are highly correlated with other variables in predicting enrollment outcomes. In these cases, the highly correlated variables may be determined using correlation matrix analysis.

[0072] In certain embodiments, each ensemble ML model includes a boosted neural network (BNN), each BNN comprising a plurality of component models, configured as a neural networks (NNs), trained in a sequential manner such that each successive component model is trained to reduce residual error or prediction loss from the combination of one or more (or, e.g., all) of the preceding component models — wherein outputs of the component models are aggregated to generate a final model output (e.g., enrollment or screening probability). In some embodiments, each ensemble ML model BNN includes five or more component models trained using boostingtechniques (e.g., as described above) such as, e.g., ten or more component models, or fifteen or more component models, or twenty or more component models, etc.

[0073] In some embodiments, the first layer of the hidden layer structure of each BNN includes a plurality of different types of nodes, wherein the plurality of different types of nodes are configured to capture a plurality of different types of effects of patient-level variables. In some instances, the first layer of each component model of each BNN includes a plurality of different types of nodes. In some embodiments, the node types include one or more of: smooth nonlinear activation function nodes (i.e., nodes for capturing smooth nonlinear trends such as, e.g., hyperbolic tangent function [TanH], Sigmoid, softsign, Swish, or GELU nodes), linear or identity activation function nodes, and localized activation function nodes (i.e., nodes for capturing localized or region-specific effects such as, e g., Gaussian, RBF, multiquadric, or inverse multiquadric nodes). In some embodiments, the first layer of each component model of each BNN includes a plurality of TanH, linear, and Gaussian nodes.

[0074] In certain embodiments, the method further includes training each ensemble ML model. In some embodiments, for each enrollment ensemble ML model, the training includes: sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial enrollment based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; and minimizing a loss function including a squared penalty function during training of each component model, the squared penalty function including a squared error between predicted and actual enrollment outcomes. In other embodiments, for each screening ensemble ML model, the training includes: sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial screening based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; and minimizing a loss function including a squared penalty function during training of each component model, the squared penalty function including a squared error between predicted and actual screening outcomes.

[0075] In certain embodiments, the training is configured to preserve the predictive influence of multiple patient-level variables during training. In some embodiments, the plurality of individuals associated with the clinical trial includes individuals whose distribution of socialidentifier variable values approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial. In some embodiments, the training includes performing k-fold cross-validation such as, e.g., a 10-fold cross-validation. In some embodiments, the training includes removing patient-level variables having an estimated influence below a predetermined threshold from consideration in the final model. For example, the removing may include performing backward elimination. In some instances, the dataset used to train each ensemble ML model consists only of individuals from the subpopulation for which the model is configured to determine the relative influence of patient-level variables on clinical trial screening or enrollment.

[0076] In certain embodiments, determining the relative influence of each patient-level variable for an ensemble ML model includes calculating a quantitative influence metric for each patientlevel variable. In these instances, the quantitative influence metric may include a Shapley-based value representing the average contribution of each patient-level variable to model predictions. In some embodiments, the method further includes using the trained ensemble ML models to determine an individual patient’s probability of clinical trial enrollment based on the patientlevel variables of the patient.

[0077] In certain embodiments, optimizing recruitment of individuals for the clinical trial includes allocating recruitment resources based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models. Recruitment resources may include, e.g., funding allocations, personnel assignments, outreach materials, clinical site placement decisions, advertising budgets, community engagement efforts, or patient navigation services used to support clinical trial enrollment. In some embodiments, recruitment resources are allocated to increase enrollment for a target subpopulation based on patient-level variables determined to play a larger role in predicting enrollment among individuals of the target subpopulation than in the general patient population of clinical trial associated individuals (i.e., using whole cohort and target subpopulation ensemble ML models).

[0078] In some embodiments, optimizing recruitment of individuals for the clinical trial includes generating a recruitment plan based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models. In some embodiments, the method further includes generating a recruitment report based on a comparisonof the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models. In these instances, the recruitment report may include one or more of a summary of differences in the relative influence of patient-level variables between the overall population cohort determined by the first ensemble ML model and each of the one or more subpopulations, a recommended allocation of recruitment resources, and a proposed recruitment plan. In some instances, the recruitment report is generated using a natural language processing (NLP) model such as, e.g., a large language model (LLM).

[0079] In certain embodiments, the participant demographics of the clinical trial are optimized such that the distribution of social identifier variable values among recruited individuals better approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial. In some embodiments, the participant demographics of the clinical trial are optimized in order to better represent one or more subpopulations underrepresented in prior clinical trials addressing the health condition. In some instances, the participant demographics of the clinical trial are optimized to comply with one or more regulatory or institutional demographic guidelines for clinical trial enrollment. In some embodiments, the participant demographics of the clinical trial are optimized to increase biological heterogeneity among trial participants.

[0080] In some embodiments, the methods of the present disclosure are implemented via a computer such as, e.g., using lines of computer code. Embodiments of the present disclosure also include computer systems configured to perform the method as described above and herein, e.g., as described in greater detail below.

[0081] FIG. 17 provides a flow chart depicting a method for optimizing clinical trial participant demographics (e.g., as described above), in accordance with embodiments of the disclosure. In step 1701, values are obtained for multiple patient-level variables for each of a plurality of individuals associated with the clinical trial, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a primary social identifier variable. The patient-level variables include social identifier variables (such as, e.g., demographic and socioeconomic variables) and prognostic variables. In step 1702, the relative influence of the patient-level variables on clinical trial enrollment are determined across the plurality of individuals using a first ensemble ML model, wherein the first ensemble ML model is trained using a dataset comprising individuals from each subpopulation of theplurality of individuals. In step 1703, the relative influence of the patient-level variables on clinical trial enrollment are determined for individuals of the second subpopulation defined by the primary social identifier variable using a second ensemble ML model, wherein the second ensemble ML model is trained using a dataset comprising individuals of the second subpopulation. In step 1704, recruitment of individuals for the clinical trial is optimized based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first and second ensemble ML models.

[0082] FIG. 18 illustrates a system of BNN models for performing the methods of optimizing clinical trial participant demographics, in accordance with embodiments of the disclosure. The system includes a first BNN model trained using a dataset comprising individuals from each subpopulation of a plurality of individuals to determine the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals or, i.e., across the whole cohort. The system also includes any number of additional BNN models (e.g., BNN Model 2 through BNN Model «), each trained using a dataset including individuals of an identified target subpopulation to determine the relative influence of the patient-level variables on clinical trial enrollment for individuals of the respective target subpopulation. In some embodiments, the plurality of target subpopulations may be identified based on disparities between the incidence of the health condition among subpopulations defined by social identifier variable values and the corresponding enrollment patterns observed for those subpopulations in clinical trials addressing the health condition. Training data such as, e g., EHR data (i.e., as discussed above) is received by an initiation component. The initiation component may then: extract multiple patient-level variables for each of a plurality of individuals associated with the clinical trial from the EHR data, preprocess the patient-level variables, generate training datasets for each of the BNN models, and train each BNN model using each generated dataset. An integration component may calculate a quantitative influence metric for each patient-level variable (e.g., mean Shapley value) and each BNN model, compare the quantitative influence metrics calculated for each BNN model, and generate a recruitment report based on the comparison of the relative influence of the patient-level variables on clinical trial enrollment. In some embodiments, the recruitment report includes one or more of: a summary of differences in the relative influence of patient-level variables between the overall population cohort determined by the first ensemble ML model and each of the one or more target subpopulations, a recommended allocation of recruitmentresources, and a proposed recruitment plan. The integration component may also include an NLP model such as, e g., an LLM.

[0083] COMPUTER SYSTEMS

[0084] Aspects of the present disclosure further include systems, such as computer-controlled systems, for practicing embodiments of the above methods. As discussed above, the subject methods may be adapted as computer-implemented methods as, e.g., the methods are particularly suited for automatic implementation via computer. For example, the steps of the subject methods may be represented or written as lines of computer code, wherein the subject systems may automatically implement the subject via a computer executing or running the lines of code.

[0085] In one aspect, systems for performing the methods of optimizing clinical trial participant demographics, as described above, are provided. Embodiments of the systems include: a processor including memory operably coupled to the processor, wherein the memory includes instructions stored thereon, which, when executed by the processor, cause the processor to: obtain values for multiple patient-level variables for each of a plurality of individuals associated with a clinical trial, the patient-level variables including social identifier variables and prognostic variables, wherein the plurality of individuals includes at least a first subpopulation and a second subpopulation of individuals having different values of a primary social identifier variable; determine the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals using a first ensemble ML model, wherein the first ensemble ML model is trained using a dataset including individuals from each subpopulation of the plurality of individuals; determine the relative influence of the patient-level variables on clinical trial enrollment for individuals of the second subpopulation defined by the primary social identifier variable using a second ensemble ML model, wherein the second ensemble ML model is trained using a dataset including individuals of the second subpopulation; and optimize recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first and second ensemble ML models.

[0086] In certain embodiments, the instructions, when executed by the processor, further cause the processor to train each ensemble ML model using an initiation component. In some embodiments, the instructions, when executed by the processor, further cause the processor to generate a recruitment report based on a comparison of the relative influence of the patient-levelvariables on clinical trial enrollment determined by each of the ensemble ML models using an integration component. In these instances, the integration component may include an NLP model such as, e.g., and LLM.

[0087] In some embodiments, the instructions, when executed by the processor, further cause the processor to perform two or more of the subject methods described above (including, e.g., two or more of the same subject method for two or more different individuals or datasets), e.g., simultaneously, in parallel, or serially. In some embodiments, the instructions are represented as lines of computer code.

[0088] In some embodiments, the systems further include one or more computers for complete automation, or partial automation, of the methods described herein. For example, a system of the disclosure may further include one or more computers for automatically or semi-automatically running the lines of computer code that make up the instructions (e g., as described above, wherein the memory of the instructions may take the form of a computer readable storage medium). In some embodiments, the subject systems include a computer having a computer readable storage medium with a computer program stored thereon, the computer program being configured to perform the methods of the disclosure, e.g., as described above and herein.

[0089] The methods described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, a data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or any combination thereof.

[0090] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code).A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0091] The instructions may be any set of instructions to be executed directly (such as, e.g., machine code) or indirectly (such as, e.g., scripts) by the processor. In that regard, the terms “instruction(s),” “step(s),” and “program(s)” may be used interchangeably herein. The instructions may be stored in object code form for direct processing by the processor, or in any other computer language including scripts or collections of independent source code modules that are interpreted on demand or compiled in advance.

[0092] Data may be retrieved, stored or modified by the processor in accordance with the instructions. For instance, although the system is not limited by any particular data structure, the data may be stored in computer registers, in a relational database as a table having a plurality of different fields and records, XML documents, or flat files. The data may also be formatted in any computer-readable format such as, but not limited to, binary values, ASCII or Unicode.

[0093] Moreover, the data may include any information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations) or information which is used by a function to calculate the relevant data.

[0094] In certain embodiments, the processor and storage component may include multiple processors and storage components that may or may not be stored within the same physical housing. For example, some of the instructions and data may be stored on removable CD-ROM and others within a read-only computer chip. Some or all of the instructions and data may be stored in a location physically remote from, yet still accessible by, the processor. Similarly, the processor may include a collection of processors which may or may not operate in parallel.

[0095] In some embodiments, the subject systems include an input module, a processing module, and an output module. In some embodiments, the systems are configured to perform the computer implemented methods of the disclosure (e.g., as described above and herein) and may include a computer containing a processor, a storage component (i.e., memory), a display component, and any other components typically present in general purpose computers. The subject systems may include both hardware and software components, where the hardware components may take the form of one or more platforms, e.g., in the form of servers, such thatthe functional elements, i.e., those elements of the system that carry out specific tasks (such as managing input and output of information, processing information, etc.), of the system may be carried out by the execution of software applications on and across the one or more computer platforms represented by the system.

[0096] Systems of the disclosure may include a display and operator input device. Operator input devices may, for example, be a keyboard, mouse, or the like. The processing module includes a processor which has access to a memory having instructions stored thereon for performing the steps of the subject methods. The processing module may include an operating system, a graphical user interface (GUI) controller, a system memory, memory storage devices, and inputoutput controllers, cache memory, a data backup unit, and many other devices. The processor may be a commercially available processor, or it may be one of other processors that are or will become available. The processor executes the operating system and the operating system interfaces with firmware and hardware in a well-known manner, and facilitates the processor in coordinating and executing the functions of various computer programs that may be written in a variety of programming languages, such as Java, Perl, C++, Python, other high-level or low-level languages, as well as combinations thereof, as is known in the art. The operating system, typically in cooperation with the processor, coordinates and executes functions of the other components of the computer. The operating system also provides scheduling, input-output control, file and data management, memory management, and communication control and related services, all in accordance with known techniques. The processor may be any suitable analog or digital system. In some embodiments, the processor includes analog electronics which provide feedback control, such as for example negative feedback control.

[0097] The system memory may be any of a variety of known or future memory storage devices. Examples include any commonly available random access memory (RAM), magnetic medium such as a resident hard disk or tape, an optical medium such as a read and write compact disc, flash memory devices, or other memory storage device. The memory storage device may be any of a variety of known or future devices, including a compact disk drive, a tape drive, a removable hard disk drive, or a diskette drive. Such types of memory storage devices typically read from, and / or write to, a program storage medium (not shown) such as, respectively, a compact disk, magnetic tape, removable hard disk, or floppy diskette. Any of these program storage media, or others now in use or that may later be developed, may be considered a computer programproduct. As will be appreciated, these program storage media typically store a computer software program and / or data. Computer software programs, also called computer control logic, typically are stored in system memory and / or the program storage device used in conjunction with the memory storage device.

[0098] In some embodiments, a computer program product is described including a computer usable medium having control logic (computer software program, including program code) stored therein. The control logic, when executed by the processor the computer, causes the processor to perform functions described herein. In other embodiments, some functions are implemented primarily in hardware using, for example, a hardware state machine.

[0099] Implementation of the hardware state machine so as to perform the functions described herein will be apparent to those skilled in the relevant arts.

[0100] Memory may be any suitable device in which the processor can retrieve data and / or store data (i.e., any device capable of storing information accessible by the processor), such as magnetic, optical, or solid-state storage devices (including magnetic or optical disks or tape or RAM, or any other suitable device, either fixed or portable). The processor may be any well-known processor, such as processors from Intel Corporation. Alternatively, the processor may be a dedicated controller such as an ASIC or an FPGA. The processor may include a general-purpose digital microprocessor suitably programmed from a computer readable medium carrying necessary program code. Programming can be provided remotely to processor through a communication channel, or previously saved in a computer program product such as memory or some other portable or fixed computer readable storage medium using any of those devices in connection with memory. For example, a magnetic or optical disk may carry the programming, and can be read by a disk writer / reader. Systems of the invention also include programming, e.g., in the form of computer program products, algorithms for use in practicing the methods as described above. Programming according to the present invention can be recorded on computer readable media, e.g., any medium that can be read and accessed directly by a computer. Such media include, but are not limited to: magnetic storage media, such as floppy discs, hard disc storage medium, and magnetic tape; optical storage media such as CD-ROM; electrical storage media such as RAM and ROM; portable flash drive; and hybrids of these categories such as magnetic / optical storage media.The processor may also have access to a communication channel to communicate with a user at a remote location. By remote location is meant the user is not directly in contact with the system and relays input information to an input manager from an external device, such as a computer connected to a Wide Area Network (“WAN”), telephone network, satellite network, or any other suitable communication channel, including a mobile telephone (i.e., smartphone).

[0101] In some embodiments, systems of the present disclosure may be configured to include a communication interface. In some embodiments, the communication interface includes a receiver and / or transmitter for communicating with a network and / or another device. The communication interface can be configured for wired or wireless communication, including, but not limited to, radio frequency (RF) communication (e.g., Radio-Frequency Identification (RFID), Zigbee communication protocols, WiFi, infrared, wireless Universal Serial Bus (USB), Ultra-Wide Band (UWB), Bluetooth® communication protocols, and cellular communication, such as code division multiple access (CDMA) or Global System for Mobile communications (GSM).

[0102] In one embodiment, the communication interface is configured to include one or more communication ports, e.g., physical ports or interfaces such as a USB port, an RS-232 port, or any other suitable electrical connection port to allow data communication between the subject systems and other external devices such as a computer terminal (e.g., at a physician’s office or in hospital environment) that is configured for similar complementary data communication.

[0103] In one embodiment, the communication interface is configured for infrared communication, Bluetooth® communication, or any other suitable wireless communication protocol to enable the subject systems to communicate with other devices such as computer terminals and / or networks, communication enabled mobile telephones, personal digital assistants, or any other communication devices which the user may use in conjunction.

[0104] In one embodiment, the communication interface is configured to provide a connection for data transfer utilizing Internet Protocol (IP) through a cell phone network, Short Message Service (SMS), wireless connection to a personal computer (PC) on a Local Area Network (LAN) which is connected to the internet, or WiFi connection to the internet at a WiFi hotspot.

[0105] In one embodiment, the subject systems are configured to wirelessly communicate with a server device via the communication interface, e.g., using a common standard such as 802.11 or Bluetooth® RF protocol, or an IrDA infrared protocol. The server device may be another portable device, such as a smart phone, Personal Digital Assistant (PDA) or notebook computer;or a larger device such as a desktop computer, appliance, etc. In some embodiments, the server device has a display, such as a liquid crystal display (LCD), as well as an input device, such as buttons, a keyboard, mouse or touch-screen.

[0106] In some embodiments, the communication interface is configured to automatically or semi-automatically communicate data stored in the subject systems, e.g., in an optional data storage unit, with a network or server device using one or more of the communication protocols and / or mechanisms described above.

[0107] Output controllers may include controllers for any of a variety of known display devices for presenting information to a user, whether a human or a machine, whether local or remote. If one of the display devices provides visual information, this information typically may be logically and / or physically organized as an array of picture elements. A graphical user interface (GUI) controller may include any of a variety of known or future software programs for providing graphical input and output interfaces between the system and a user, and for processing user inputs. The functional elements of the computer may communicate with each other via system bus. Some of these communications may be accomplished in alternative embodiments using network or other types of remote communications. The output manager may also provide information generated by the processing module to a user at a remote location, e.g., over the Internet, phone or satellite network, in accordance with known techniques. The presentation of data by the output manager may be implemented in accordance with a variety of known techniques. As some examples, data may include SQL, HTML or XML documents, email or other files, or data in other forms. The data may include Internet URL addresses so that a user may retrieve additional SQL, HTML, XML, or other documents or data from remote sources.

[0108] The one or more platforms present in the subject systems may be any type of known computer platform or a type to be developed in the future, although they typically will be of a class of computer commonly referred to as servers. However, they may also be a main-frame computer, a workstation, or other computer type. They may be connected via any known or future type of cabling or other communication system including wireless systems, either networked or otherwise. They may be co-located, or they may be physically separated. Various operating systems may be employed on any of the computer platforms, possibly depending on the type and / or make of computer platform chosen. Appropriate operating systems include Windows, iOS, Oracle Solaris, Linux, IBM, Unix, and others.NON- TRANSITORY COMPUTER READABLE STORAGE MEDIA

[0109] Aspects of the present disclosure further include non-transitory computer readable storage mediums having instructions for practicing the subject methods described herein.

[0110] Computer readable storage mediums may be employed on one or more computers for complete automation or partial automation of a system for practicing methods described herein. In certain embodiments, instructions in accordance with the methods described herein can be coded onto a computer-readable medium in the form of “programming”, where the term “computer readable medium,” as used herein, refers to any non-transitory storage medium that participates in providing instructions and data to a computer for execution and processing. Examples of suitable non-transitory storage media include a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, non-volatile memory card, ROM, DVD-ROM, Blue-ray disk, solid state disk, and network attached storage (NAS), whether or not such devices are internal or external to the computer. A file containing information can be “stored” on computer readable medium, where “storing” means recording information such that it is accessible and retrievable at a later date by a computer. The computer-implemented methods described herein can be executed using programming that can be written in one or more of any number of computer programming languages. Such languages include, for example, Python, Java, Java Script, C, C#, C++, Go, R, Swift, PHP, as well as many others.

[0111] The non-transitory computer readable storage medium may be employed on one or more computer systems having a processing module, display, and operator input device. Operator input devices may, for example, be a keyboard, mouse, or the like. The processing module includes a processor which has access to a memory having instructions stored thereon for performing the steps of the subject methods. The processing module may include an operating system, a graphical user interface (GUI) controller, a system memory, memory storage devices, input-output controllers, cache memory, a data backup unit, and many other devices. The processor may be a commercially available processor, or it may be one of other processors that are, or will become, available. The processor executes the operating system and the operating system interfaces with firmware and hardware in a well-known manner, and facilitates the processor in coordinating and executing the functions of various computer programs that may be written in a variety of programming languages such as, e.g., those mentioned above, other high level or low-level languages, as well as combinations thereof, as is known in the art. Theoperating system, typically in cooperation with the processor, coordinates and executes functions of the other components of the computer. The operating system may also provide scheduling, input-output control, file and data management, memory management, and communication control and related services, all in accordance with known techniques.

[0112] KITS

[0113] Aspects of the present disclosure further include kits for carrying out the subject methods described above and herein. In some embodiments, the kit includes software and / or memory for carrying out the methods of the disclosure via a computer (e.g., the kit includes software, access to a computer program product, and / or memory for carrying out the computer-implemented methods of the disclosure). In some embodiments, the kit includes one or more components of a system for carrying out the methods of the disclosure, e.g., as described above.

[0114] In some embodiments, the kit includes one or more components of a system for optimizing clinical trial participant demographics. Such a system may include: a processor; memory operably coupled to the processor wherein the memory includes instructions stored thereon, which when executed by the processor, cause the processor to perform the subject methods; a visual display; and / or one or more operator input devices. In some embodiments, the kit includes a non-transitory computer readable storage medium including the instructions of the memory of the system and / or the instructions of a computer program product of the disclosure e.g., as described above and herein.

[0115] In addition, the subject kits may further include, in certain embodiments, instructions for practicing the subject methods. These instructions may be present in the subject kits in a variety of forms, one or more of which may be present in the kit. For example, instructions may be present as printed information on a suitable medium or substrate such as, e.g., a piece or pieces of paper on which the information is printed, in the packaging of the kit, in a package insert, and the like. Another form of these instructions is a computer-readable medium such as, e.g., a diskette, compact disk (CD), flash drive, and the like, on which the information has been recorded. Yet another form of these instructions that may be present is a website address which may be used via the internet to access the information at a removed site.UTILITY

[0116] The methods and systems of the present disclosure find use in a variety of applications where it is desirable to optimize the demographic composition of clinical trial participants in order to improve the reliability, fairness, and / or applicability of clinical research. In particular, embodiments of the invention utilize novel ensemble machine learning (ML) models such as, e.g., boosted neural networks (BNNs), that are specifically adapted to capture how nuanced interactions between heterogeneous types of social identifier variables differentially influence clinical trial enrollment across distinct subpopulations of individuals. In some embodiments, the disclosure further includes methods for generating or adjusting recruitment strategies based on the outputs of the disclosed models, thereby enabling targeted recruitment of participants from subpopulations selected by a user including, e.g., historically underrepresented subpopulations.

[0117] In certain embodiments, the disclosed methods and systems find use in improving the generalizability of clinical trial results. For example, by optimizing the enrollment of participants from a wider range of demographic and biological backgrounds, the disclosed methods may improve the extent to which clinical trial findings are applicable to a broader patient population. In some embodiments, the disclosed methods are configured to identify subpopulations that may be underrepresented if an initial recruitment strategy is implemented and, subsequently, adjust the recruitment strategy to prevent such underrepresentation — thereby facilitating the evaluation of investigational interventions across a broader range of biological characteristics (e.g., genetic, age-related, or sex-linked factors). As such, the present invention may find use in supporting the generation of clinical trial data that more robustly reflects real-world patient variation, improving the safety and efficacy of therapeutic interventions prior to approval or commercialization. In certain embodiments, the disclosed methods and systems find use in supporting the equitable allocation of healthcare resources. For example, by enabling targeted recruitment of individuals from historically underrepresented subpopulations (e.g., based on race, ethnicity, or language preference), the methods and systems of the invention may contribute to reducing disparities in both clinical trial access and downstream therapeutic benefits. In some cases, the present invention facilitates the inclusion of individuals whose responses to medical interventions may otherwise go uncharacterized — thereby improving the clinical and regulatory evaluation of therapeutic targets and interventions for a wider array of population subgroups. Insome embodiments, the methods and systems of the disclosure may find use in addressing institutional disparities in biomedical research and treatment.

[0118] In certain embodiments, the methods and systems described herein find use in enabling or accelerating the discovery of new therapeutic targets and interventions for precision medicine. For example, by promoting the enrollment of biologically diverse patient populations, the disclosed methods and systems may support the collection and analysis of more comprehensive datasets capturing a wider array of unique genetic, proteomic, and metabolic characteristics. In some embodiments, the methods of the present disclosure may find use in identifying population-specific biomarkers affecting medical intervention outcomes that would otherwise remain unidentified, e.g., for clinical trials comprised of a relatively homogenous group of participants. Accordingly, embodiments of the disclosed methods and systems may find use in advancing the development of personalized diagnostics and treatments, improving patient outcomes, and contributing to the broader development of precision medicine strategies that adapt medical interventions to the unique biological profiles of individual patients within a diverse population.

[0119] EXEMPLARY NON-LIMITING ASPECTS OF THE DISCLOSURE

[0120] Aspects, including embodiments, of the present subject matter described above may be beneficial alone or in combination with one or more other aspects or embodiments. Without limiting the foregoing description, certain non-limiting aspects of the disclosure are provided below. As will be apparent to those of ordinary skill in the art upon reading this disclosure, each of the individually numbered aspects may be used or combined with any one of the preceding or following individually numbered aspects. This is intended to provide support for all such combinations of aspects and is not limited to combinations of aspects explicitly provided below. It will be apparent to one of ordinary skill in the art that various changes and modifications can be made without departing from the spirit or scope of the invention.

[0121] 1. A method for optimizing participant demographics of a clinical trial, the method comprising:

[0122] obtaining values for multiple patient-level variables for each of a plurality of individuals associated with the clinical trial, the patient-level variables comprising social identifier variablesand prognostic variables, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a primary social identifier variable;

[0123] determining the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals using a first ensemble machine learning (ML) model, wherein the first ensemble ML model is trained using a dataset comprising individuals from each subpopulation of the plurality of individuals;

[0124] determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of the second subpopulation defined by the primary social identifier variable using a second ensemble ML model, wherein the second ensemble ML model is trained using a dataset comprising individuals of the second subpopulation; and

[0125] optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first and second ensemble ML models.

[0126] 2. The method according to aspect 1, wherein the multiple social identifier variables comprise demographic variables and socioeconomic variables.

[0127] 3. The method according to aspect 1 or 2, wherein the second ensemble ML model is trained using a dataset consisting of individuals of the second subpopulation.

[0128] 4. The method according to any one of the preceding aspects, wherein the ensemble ML models are boosted neural networks (BNNs).

[0129] 5. The method according to any one of the preceding aspects, wherein the primary social identifier variable comprises a biologically based social identifier variable.

[0130] 6. The method according to aspect 5, wherein the primary social identifier variable comprises age, race, or biological sex.

[0131] 7. The method according to any one of the preceding aspects, wherein the plurality of clinical trial associated individuals have been diagnosed with a health condition addressed by the clinical trial.

[0132] 8. The method according to aspect 8, wherein the clinical trial is a therapeutic clinical trial.

[0133] 9. The method according to aspect 7 or 8, wherein the plurality of clinical trial associated individuals have a known status of clinical trial participation, and wherein at least a portion of the individuals have participated in a clinical trial addressing the health condition.10. The method according to any one of aspects 7-9, wherein the health condition comprises cancer, and wherein the prognostic variables comprise oncologic variables.

[0134] 11. The method according to aspect 10, wherein the cancer is a glioma.

[0135] 12. The method according to aspect 11, wherein the oncologic variables comprise one or more of Karnofsky Performance Status (KPS), tumor grade, tumor type, tumor hemisphere and / or lobe, receipt of chemotherapy and / or radiation, extent of resection (EOR), and seizure presentation.

[0136] 13. The method according to any one of the preceding aspects, wherein the multiple social identifier variables comprise a plurality of socioeconomic variables, the socioeconomic variables including one or more of insurance status, employment status, occupation classification, household income, distance from clinical trial facility, and interpreter use.

[0137] 14. The method according to any one of the preceding aspects, wherein the multiple social identifier variables comprise a plurality of demographic variables, the demographic variables including one or more of biological sex, race, ethnicity, age, and preferred language.

[0138] 15. The method according to any one of the preceding aspects, wherein one or more of the patient-level variables are derived from electronic health record (EHR) data obtained for each of the plurality of individuals.

[0139] 16. The method according to aspect 15, wherein the method further comprises determining one or more patient-level variable values based on proxy indicators of the variables found in the EHR data.

[0140] 17. The method according to any one of the preceding aspects, wherein the primary social identifier variable comprises race, the first subpopulation of individuals defined by the primary social identifier variable comprises individuals of a majority racial group, and the second subpopulation of individuals defined by the primary social identifier variable comprises individuals of one or more racial groups other than the majority racial group.

[0141] 18. The method according to aspect 17, wherein the second subpopulation of individuals defined by the primary social identifier comprises one or more racial groups underrepresented in clinical trial enrollment.

[0142] 19. The method according to any one of the preceding aspects, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a secondary social identifier variable.20. The method according to aspect 19, wherein the method further comprises: determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of the first subpopulation defined by the secondary social identifier variable using a third ensemble ML model, wherein the third ensemble ML model is trained using a dataset comprising individuals of the first subpopulation defined by the secondary social identifier variable; and

[0143] optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first, second, and third ensemble ML models.

[0144] 21. The method according to aspect 20, wherein the secondary social identifier variable comprises age, race, or biological sex.

[0145] 22. The method according to aspect 21, wherein the secondary social identifier variable comprises biological sex and the first subpopulation of individuals defined by the secondary social identifier variable comprises biological females.

[0146] 23. The method according to any one of the preceding aspects, wherein the method further comprises:

[0147] identifying a plurality of target subpopulations of interest, each defined by a distinct set of social identifier variable values;

[0148] determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of each target subpopulation using a corresponding ensemble ML model, wherein each ensemble ML model is trained using a dataset comprising individuals of the respective target subpopulation; and

[0149] optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by the ensemble ML models for each target subpopulation.

[0150] 24. The method according to aspect 23, wherein the plurality of target subpopulations are identified based on disparities between the incidence of the health condition among subpopulations defined by social identifier variable values and the corresponding enrollment patterns observed forthose subpopulations in clinical trials addressing the health condition.25. The method according to aspect 24, wherein the method further comprises identifying the plurality of target subpopulations from the multiple patient-level variables obtained for each of the plurality of clinical trial associated individuals.

[0151] 26. The method according to any one of the preceding aspects, wherein the patient-level variables comprise a screening status indicating whether an individual has been screened for a clinical trial addressing the health condition.

[0152] 27. The method according to aspect 26, wherein the patient-level variables comprise the number of clinical trials discussed with an individual.

[0153] 28. The method according to aspect 26 or 27, wherein, for each ensemble ML model used to determine the relative influence of patient-level variables on clinical trial enrollment, the method further comprises training a corresponding ensemble ML model to determine the relative influence of the patient-level variables on clinical trial screening for the same subpopulation. 29. The method according to aspect 26, wherein the method further comprises:

[0154] determining the relative influence of the patient-level variables on clinical trial screening for individuals of each subpopulation using each screening ensemble ML model; and optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by each screening and enrollment ensemble ML model.

[0155] 30. The method according to any one of the preceding aspects, wherein the method further comprises preprocessing the patient-level variables.

[0156] 31. The method according to aspect 30, the patient-level variables are preprocessed by: one-hot encoding each categorical variable into a set of binary variables; and transforming each continuous variable to reduce the influence of outliers or skewed distributions.

[0157] 32. The method according to aspect 31, wherein each continuous variable is transformed to approximate a normal distribution using a Johnson Su or a Johnson Sb distribution.

[0158] 33. The method according to any one of aspects 30 to 32, wherein the preprocessing comprises, for each ensemble ML model, filtering the patient-level variables to remove variables that are highly correlated with other variables in predicting enrollment outcomes.

[0159] 34. The method according to aspect 33, wherein the highly correlated variables are determined using correlation matrix analysis.35. The method according to any one of the preceding aspects, wherein each ensemble ML model comprises a BNN, the BNN comprising five or more component models trained using boosting techniques.

[0160] 36. The method according to aspect 35, wherein each BNN comprises ten or more component models.

[0161] 37. The method according to aspect 35 or 36, wherein the first layer of the hidden layer structure of each BNN comprises a plurality of different types of nodes, wherein the plurality of different types of nodes are configured to capture a plurality of different types of effects of patient-level variables.

[0162] 38. The method according to aspect 37, wherein the first layer of each component model of each BNN comprises a plurality of different types of nodes.

[0163] 39. The method according to any one of aspects 35 to 38, wherein the method further comprises training each ensemble ML model.

[0164] 40. The method according to aspect 39, wherein for each enrollment ensemble ML model the training comprises:

[0165] sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial enrollment based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; and

[0166] minimizing a loss function comprising a squared penalty function during training of each component model, the squared penalty function comprising a squared error between predicted and actual enrollment outcomes.

[0167] 41. The method according to aspect 39, wherein for each screening ensemble ML model the training comprises:

[0168] sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial screening based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; and

[0169] minimizing a loss function comprising a squared penalty function during training of each component model, the squared penalty function comprising a squared error between predicted and actual screening outcomes.42. The method according to aspect 40 or 41, wherein the training is configured to preserve the predictive influence of multiple patient-level variables during training.

[0170] 43. The method according to any one of aspects 39 to 42, wherein the plurality of individuals associated with the clinical trial comprises individuals whose distribution of social identifier variable values approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial.

[0171] 44. The method according to any one of aspects 39 to 43, wherein the training comprises performing k-fold cross-validation.

[0172] 45. The method according to any one of aspects 39 to 44, wherein the training comprises removing patient-level variables having an estimated influence below a predetermined threshold from consideration in the final model.

[0173] 46. The method according to aspect 45, wherein the removing comprises performing backward elimination.

[0174] 47. The method according to any one of aspects 39 to 46, wherein the dataset used to train each ensemble ML model consists only of individuals from the subpopulation for which the model is configured to determine the relative influence of patient-level variables on clinical trial screening or enrollment.

[0175] 48. The method according to any one of the preceding aspects, wherein determining the relative influence of each patient-level variable for an ensemble ML model comprises calculating a quantitative influence metric for each patient-level variable.

[0176] 49. The method according to aspect 45, wherein the quantitative influence metric comprises a Shapley-based value representing the average contribution of each patient-level variable to model predictions.

[0177] 50. The method according to any one of the preceding aspects, wherein the method further comprises using the trained ensemble ML models to determine an individual patient’s probability of clinical trial enrollment based on the patient-level variables of the patient.

[0178] 51. The method according to any one of the preceding aspects, wherein optimizing recruitment of individuals for the clinical trial comprises allocating recruitment resources based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.52. The method according to any one of the preceding aspects, wherein optimizing recruitment of individuals for the clinical trial comprises generating a recruitment plan based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

[0179] 53. The method according to any one of the preceding aspects, wherein the method further comprises generating a recruitment report based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

[0180] 54. The method according to aspect 53, wherein the recruitment report comprises one or more of a summary of differences in the relative influence of patient-level variables between the overall population cohort determined by the first ensemble ML model and each of the one or more subpopulations, a recommended allocation of recruitment resources, and a proposed recruitment plan.

[0181] 55. The method according to aspect 54, wherein the recruitment report is generated using a natural language processing (NLP) model.

[0182] 56. The method according to aspect 55, wherein the NLP model is a large language model (LLM).

[0183] 57. The method according to any one of the preceding aspects, wherein the participant demographics of the clinical trial are optimized such that the distribution of social identifier variable values among recruited individuals better approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial.

[0184] 58. The method according to any one of the preceding aspects, wherein the participant demographics of the clinical trial are optimized in order to better represent one or more subpopulations underrepresented in prior clinical trials addressing the health condition.

[0185] 59. The method according to any one of the preceding aspects, wherein the participant demographics of the clinical trial are optimized to comply with one or more regulatory or institutional demographic guidelines for clinical trial enrollment.

[0186] 60. The method according to any one of the preceding aspects, wherein the participant demographics of the clinical trial are optimized to increase biological heterogeneity among trial participants.61. The method according to any one of the preceding aspects, wherein the method is implemented via a computer.

[0187] 62. The method according to aspect 54, wherein the method is implemented automatically using lines of computer code.

[0188] 63. A computer system configured to perform the method according to any one of aspects 1 to 56.

[0189] 64. A system for optimizing participant demographics of a clinical trial, the system comprising:

[0190] a processor comprising memory operably coupled to the processor, wherein the memory comprises instructions stored thereon, which, when executed by the processor, cause the processor to:

[0191] obtain values for multiple patient-level variables for each of a plurality of individuals associated with the clinical trial, the patient-level variables comprising social identifier variables and prognostic variables, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a primary social identifier variable;

[0192] determine the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals using a first ensemble machine learning (ML) model, wherein the first ensemble ML model is trained using a dataset comprising individuals from each subpopulation of the plurality of individuals;

[0193] determine the relative influence of the patient-level variables on clinical trial enrollment for individuals of the second subpopulation defined by the primary social identifier variable using a second ensemble ML model, wherein the second ensemble ML model is trained using a dataset comprising individuals of the second subpopulation; and optimize recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first and second ensemble ML models.

[0194] 65. The system according to aspect 64, wherein the multiple social identifier variables comprise demographic variables and socioeconomic variables.

[0195] 66. The system according to aspect 64 or 65, wherein the second ensemble ML model is trained using a dataset consisting of individuals of the second subpopulation.67. The system according to any one of aspects 64 to 66, wherein the ensemble ML models are boosted neural networks (BNNs).

[0196] 68. The system according to any one of aspects 64 to 67, wherein the primary social identifier variable comprises a biologically based social identifier variable.

[0197] 69. The system according to aspect 68, wherein the primary social identifier variable comprises age, race, or biological sex.

[0198] 70. The system according to any one of aspects 64 to 69, wherein the plurality of clinical trial associated individuals have been diagnosed with a health condition addressed by the clinical trial.

[0199] 71. The system according to aspect 70, wherein the clinical trial is a therapeutic clinical trial.

[0200] 72. The system according to aspect 70 or 71, wherein the plurality of clinical trial associated individuals have a known status of clinical trial participation, and wherein at least a portion of the individuals have participated in a clinical trial addressing the health condition.

[0201] 73. The system according to any one of aspects 70-72, wherein the health condition comprises cancer, and wherein the prognostic variables comprise oncologic variables.

[0202] 74. The system according to aspect 73, wherein the cancer is a glioma.

[0203] 75. The system according to aspect 74, wherein the oncologic variables comprise one or more of Karnofsky Performance Status (KPS), tumor grade, tumor type, tumor hemisphere and / or lobe, receipt of chemotherapy and / or radiation, extent of resection (EOR), and seizure presentation.

[0204] 76. The system according to any one of aspects 64 to 75, wherein the multiple social identifier variables comprise a plurality of socioeconomic variables, the socioeconomic variables including one or more of insurance status, employment status, occupation classification, household income, distance from clinical trial facility, and interpreter use.

[0205] 77. The system according to any one of aspects 64 to 76, wherein the multiple social identifier variables comprise a plurality of demographic variables, the demographic variables including one or more of biological sex, race, ethnicity, age, and preferred language.

[0206] 78. The system according to any one of aspects 64 to 77, wherein one or more of the patientlevel variables are derived from electronic health record (EHR) data obtained for each of the plurality of individuals.79. The system according to aspect 78, wherein the instructions, when executed by the processor, further cause the processor to determine one or more patient-level variable values based on proxy indicators of the variables found in the EHR data.

[0207] 80. The system according to any one of aspects 64 to 79, wherein the primary social identifier variable comprises race, the first subpopulation of individuals defined by the primary social identifier variable comprises individuals of a majority racial group, and the second subpopulation of individuals defined by the primary social identifier variable comprises individuals of one or more racial groups other than the majority racial group.

[0208] 81. The system according to aspect 80, wherein the second subpopulation of individuals defined by the primary social identifier comprises one or more racial groups underrepresented in clinical trial enrollment.

[0209] 82. The system according to any one of aspects 64 to 81, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a secondary social identifier variable.

[0210] 83. The system according to aspect 82, wherein the instructions, when executed by the processor, further cause the processor to:

[0211] determine the relative influence of the patient-level variables on clinical trial enrollment for individuals of the first subpopulation defined by the secondary social identifier variable using a third ensemble ML model, wherein the third ensemble ML model is trained using a dataset comprising individuals of the first subpopulation defined by the secondary social identifier variable; and

[0212] optimize recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first, second, and third ensemble ML models.

[0213] 84. The system according to aspect 83, wherein the secondary social identifier variable comprises age, race, or biological sex.

[0214] 85. The system according to aspect 84, wherein the secondary social identifier variable comprises biological sex and the first subpopulation of individuals defined by the secondary social identifier variable comprises biological females.

[0215] 86. The system according to any one of aspects 64 to 85, wherein the instructions, when executed by the processor, further cause the processor to:identify a plurality of target subpopulations of interest, each defined by a distinct set of social identifier variable values;

[0216] determine the relative influence of the patient-level variables on clinical trial enrollment for individuals of each target subpopulation using a corresponding ensemble ML model, wherein each ensemble ML model is trained using a dataset comprising individuals of the respective target subpopulation; and

[0217] optimize recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by the ensemble ML models for each target subpopulation.

[0218] 87. The system according to aspect 86, wherein the plurality of target subpopulations are identified based on disparities between the incidence of the health condition among subpopulations defined by social identifier variable values and the corresponding enrollment patterns observed forthose subpopulations in clinical trials addressing the health condition. 88. The system according to aspect 87, wherein the instructions, when executed by the processor, further cause the processor to identify the plurality of target subpopulations from the multiple patient-level variables obtained for each of the plurality of clinical trial associated individuals.

[0219] 89. The system according to any one of aspects 64 to 88, wherein the patient-level variables comprise a screening status indicating whether an individual has been screened for a clinical trial addressing the health condition.

[0220] 90. The system according to aspect 89, wherein the patient-level variables comprise the number of clinical trials discussed with an individual.

[0221] 91. The system according to aspect 89 or 90, wherein, for each ensemble ML model used to determine the relative influence of patient-level variables on clinical trial enrollment, the method further comprises training a corresponding ensemble ML model to determine the relative influence of the patient-level variables on clinical trial screening for the same subpopulation. 92. The system according to aspect 91, wherein the instructions, when executed by the processor, further cause the processor to:

[0222] determine the relative influence of the patient-level variables on clinical trial screening for individuals of each subpopulation using each screening ensemble ML model; and optimize recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by each screening and enrollment ensemble ML model.93. The system according to any one of aspects 64 to 92, wherein the instructions, when executed by the processor, further cause the processor to preprocess the patient-level variables.

[0223] 94. The system according to aspect 93, the patient-level variables are preprocessed by:

[0224] one-hot encoding each categorical variable into a set of binary variables; and transforming each continuous variable to reduce the influence of outliers or skewed distributions.

[0225] 95. The system according to aspect 94, wherein each continuous variable is transformed to approximate a normal distribution using a Johnson Su or a Johnson Sb distribution.

[0226] 96. The system according to any one of aspects 93 to 95, wherein the preprocessing comprises, for each ensemble ML model, filtering the patient-level variables to remove variables that are highly correlated with other variables in predicting enrollment outcomes.

[0227] 97. The system according to aspect 96, wherein the highly correlated variables are determined using correlation matrix analysis.

[0228] 98. The system according to any one of aspects 64 to 97, wherein each ensemble ML model comprises a BNN, the BNN comprising five or more component models trained using boosting techniques.

[0229] 99. The system according to aspect 98, wherein each BNN comprises ten or more component models.

[0230] 100. The system according to aspect 98 or 99, wherein the first layer of the hidden layer structure of each BNN comprises a plurality of different types of nodes, wherein the plurality of different types of nodes are configured to capture a plurality of different types of effects of patient-level variables.

[0231] 101. The system according to aspect 100, wherein the first layer of each component model of each BNN comprises a plurality of different types of nodes.

[0232] 102. The system according to any one of aspects 98 to 100, wherein the instructions, when executed by the processor, further cause the processor to train each ensemble ML model using an initiation component.

[0233] 103. The system according to aspect 102, wherein for each enrollment ensemble ML model the training comprises:

[0234] sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial enrollment based on the patient-level variables,wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; and

[0235] minimizing a loss function comprising a squared penalty function during training of each component model, the squared penalty function comprising a squared error between predicted and actual enrollment outcomes.

[0236] 104. The system according to aspect 102, wherein for each screening ensemble ML model the training comprises:

[0237] sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial screening based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; and

[0238] minimizing a loss function comprising a squared penalty function during training of each component model, the squared penalty function comprising a squared error between predicted and actual screening outcomes.

[0239] 105. The system according to aspect 103 or 104, wherein the training is configured to preserve the predictive influence of multiple patient-level variables during training.

[0240] 106. The system according to any one of aspects 102 to 105, wherein the plurality of individuals associated with the clinical trial comprises individuals whose distribution of social identifier variable values approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial.

[0241] 107. The system according to any one of aspects 102 to 106, wherein the training comprises performing k-fold cross-validation.

[0242] 108. The system according to any one of aspects 102 to 107, wherein the training comprises removing patient-level variables having an estimated influence below a predetermined threshold from consideration in the final model.

[0243] 109. The system according to aspect 108, wherein the removing comprises performing backward elimination.

[0244] 110. The system according to any one of aspects 102 to 109, wherein the dataset used to train each ensemble ML model consists only of individuals from the subpopulation for which the model is configured to determine the relative influence of patient-level variables on clinical trial screening or enrollment.111. The system according to any one of aspects 64 to 110, wherein determining the relative influence of each patient-level variable for an ensemble ML model comprises calculating a quantitative influence metric for each patient-level variable.

[0245] 112. The system according to aspect 111, wherein the quantitative influence metric comprises a Shapley-based value representing the average contribution of each patient-level variable to model predictions.

[0246] 113. The system according to any one of aspects 64 to 112, wherein the instructions, when executed by the processor, further cause the processor to use the trained ensemble ML models to determine an individual patient’s probability of clinical trial enrollment based on the patientlevel variables of the patient.

[0247] 114. The system according to any one of aspects 64 to 113, wherein optimizing recruitment of individuals for the clinical trial comprises allocating recruitment resources based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

[0248] 115. The system according to any one of aspects 64 to 114, wherein optimizing recruitment of individuals for the clinical trial comprises generating a recruitment plan based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

[0249] 116. The system according to any one of aspects 64 to 115, wherein the instructions, when executed by the processor, further cause the processor to generate a recruitment report based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

[0250] 117. The system according to aspect 116, wherein the recruitment report comprises one or more of a summary of differences in the relative influence of patient-level variables between the overall population cohort determined by the first ensemble ML model and each of the one or more subpopulations, a recommended allocation of recruitment resources, and a proposed recruitment plan.

[0251] 118. The system according to aspect 117, wherein the recruitment report is generated using an integration component comprising a natural language processing (NLP) model.

[0252] 119. The system according to aspect 118, wherein the NLP model is a large language model (LLM).120. The system according to any one of aspects 64 to 119, wherein the participant demographics of the clinical trial are optimized such that the distribution of social identifier variable values among recruited individuals better approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial.

[0253] 121. The system according to any one of aspects 64 to 120, wherein the participant demographics of the clinical trial are optimized in order to better represent one or more subpopulations underrepresented in prior clinical trials addressing the health condition.

[0254] 122. The system according to any one of aspects 64 to 121, wherein the participant demographics of the clinical trial are optimized to comply with one or more regulatory or institutional demographic guidelines for clinical trial enrollment.

[0255] 123. The system according to any one of aspects 64 to 122, wherein the participant demographics of the clinical trial are optimized to increase biological heterogeneity among trial participants.

[0256] 124. The system according to any one of aspects 64 to 123, wherein the system is a computer system.

[0257] 125. A computer program product comprising instructions which, when executed by a computer, cause the computer to perform operations comprising:

[0258] obtaining values for multiple patient-level variables for each of a plurality of individuals associated with the clinical trial, the patient-level variables comprising social identifier variables and prognostic variables, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a primary social identifier variable;

[0259] determining the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals using a first ensemble machine learning (ML) model, wherein the first ensemble ML model is trained using a dataset comprising individuals from each subpopulation of the plurality of individuals;

[0260] determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of the second subpopulation defined by the primary social identifier variable using a second ensemble ML model, wherein the second ensemble ML model is trained using a dataset comprising individuals of the second subpopulation; andoptimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first and second ensemble ML models.

[0261] 126. The computer program product according to aspect 125, wherein the multiple social identifier variables comprise demographic variables and socioeconomic variables.

[0262] 127. The computer program product according to aspect 125 or 126, wherein the second ensemble ML model is trained using a dataset consisting of individuals of the second subpopulation.

[0263] 128. The computer program product according to any one of aspects 125 to 127, wherein the ensemble ML models are boosted neural networks (BNNs).

[0264] 129. The computer program product according to any one of aspects 125 to 128, wherein the primary social identifier variable comprises a biologically based social identifier variable.

[0265] 130. The computer program product according to aspect 129, wherein the primary social identifier variable comprises age, race, or biological sex.

[0266] 131. The computer program product according to any one of aspects 125 to 130, wherein the plurality of clinical trial associated individuals have been diagnosed with a health condition addressed by the clinical trial.

[0267] 132. The computer program product according to aspect 131, wherein the clinical trial is a therapeutic clinical trial.

[0268] 133. The computer program product according to aspect 131 or 132, wherein the plurality of clinical trial associated individuals have a known status of clinical trial participation, and wherein at least a portion of the individuals have participated in a clinical trial addressing the health condition.

[0269] 134. The computer program product according to any one of aspects 131-133, wherein the health condition comprises cancer, and wherein the prognostic variables comprise oncologic variables.

[0270] 135. The computer program product according to aspect 134, wherein the cancer is a glioma.

[0271] 136. The computer program product according to aspect 135, wherein the oncologic variables comprise one or more of Karnofsky Performance Status (KPS), tumor grade, tumor type, tumor hemisphere and / or lobe, receipt of chemotherapy and / or radiation, extent of resection (EOR), and seizure presentation.137. The computer program product according to any one of aspects 125 to 136, wherein the multiple social identifier variables comprise a plurality of socioeconomic variables, the socioeconomic variables including one or more of insurance status, employment status, occupation classification, household income, distance from clinical trial facility, and interpreter use.

[0272] 138. The computer program product according to any one of aspects 125 to 137, wherein the multiple social identifier variables comprise a plurality of demographic variables, the demographic variables including one or more of biological sex, race, ethnicity, age, and preferred language.

[0273] 139. The computer program product according to any one of aspects 125 to 138, wherein one or more of the patient-level variables are derived from electronic health record (EHR) data obtained for each of the plurality of individuals.

[0274] 140. The computer program product according to aspect 139, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of determining one or more patient-level variable values based on proxy indicators of the variables found in the EHR data.

[0275] 141. The computer program product according to any one of aspects 125 to 140, wherein the primary social identifier variable comprises race, the first subpopulation of individuals defined by the primary social identifier variable comprises individuals of a majority racial group, and the second subpopulation of individuals defined by the primary social identifier variable comprises individuals of one or more racial groups other than the majority racial group.

[0276] 142. The computer program product according to aspect 141, wherein the second subpopulation of individuals defined by the primary social identifier comprises one or more racial groups underrepresented in clinical trial enrollment.

[0277] 143. The computer program product according to any one of aspects 125 to 142, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a secondary social identifier variable.

[0278] 144. The computer program product according to aspect 143, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of:

[0279] determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of the first subpopulation defined by the secondary social identifiervariable using a third ensemble ML model, wherein the third ensemble ML model is trained using a dataset comprising individuals of the first subpopulation defined by the secondary social identifier variable; and

[0280] optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first, second, and third ensemble ML models.

[0281] 145. The computer program product according to aspect 144, wherein the secondary social identifier variable comprises age, race, or biological sex.

[0282] 146. The computer program product according to aspect 145, wherein the secondary social identifier variable comprises biological sex and the first subpopulation of individuals defined by the secondary social identifier variable comprises biological females.

[0283] 147. The computer program product according to any one of aspects 125 to 146, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of:

[0284] identifying a plurality of target subpopulations of interest, each defined by a distinct set of social identifier variable values;

[0285] determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of each target subpopulation using a corresponding ensemble ML model, wherein each ensemble ML model is trained using a dataset comprising individuals of the respective target subpopulation; and

[0286] optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by the ensemble ML models for each target subpopulation.

[0287] 148. The computer program product according to aspect 147, wherein the plurality of target subpopulations are identified based on disparities between the incidence of the health condition among subpopulations defined by social identifier variable values and the corresponding enrollment patterns observed for those subpopulations in clinical trials addressing the health condition.

[0288] 149. The computer program product according to aspect 148, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of identifyingthe plurality of target subpopulations from the multiple patient-level variables obtained for each of the plurality of clinical trial associated individuals.

[0289] 150. The computer program product according to any one of aspects 125 to 149, wherein the patient-level variables comprise a screening status indicating whether an individual has been screened for a clinical trial addressing the health condition.

[0290] 151. The computer program product according to aspect 150, wherein the patient-level variables comprise the number of clinical trials discussed with an individual.

[0291] 152. The computer program product according to aspect 150 or 151, wherein, for each ensemble ML model used to determine the relative influence of patient-level variables on clinical trial enrollment, the method further comprises training a corresponding ensemble ML model to determine the relative influence of the patient-level variables on clinical trial screening for the same subpopulation.

[0292] 153. The computer program product according to aspect 152, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of:

[0293] determining the relative influence of the patient-level variables on clinical trial screening for individuals of each subpopulation using each screening ensemble ML model; and optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by each screening and enrollment ensemble ML model.

[0294] 154. The computer program product according to any one of aspects 125 to 153, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of preprocessing the patient-level variables.

[0295] 155. The computer program product according to aspect 154, the patient-level variables are preprocessed by:

[0296] one-hot encoding each categorical variable into a set of binary variables; and transforming each continuous variable to reduce the influence of outliers or skewed distributions.

[0297] 156. The computer program product according to aspect 155, wherein each continuous variable is transformed to approximate a normal distribution using a Johnson Su or a Johnson Sb distribution.

[0298] 157. The computer program product according to any one of aspects 154 to 156, wherein the preprocessing comprises, for each ensemble ML model, filtering the patient-level variables toremove variables that are highly correlated with other variables in predicting enrollment outcomes.

[0299] 158. The computer program product according to aspect 157, wherein the highly correlated variables are determined using correlation matrix analysis.

[0300] 159. The computer program product according to any one of aspects 125 to 158, wherein each ensemble ML model comprises a BNN, the BNN comprising five or more component models trained using boosting techniques.

[0301] 160. The computer program product according to aspect 159, wherein each BNN comprises ten or more component models.

[0302] 161. The computer program product according to aspect 159 or 160, wherein the first layer of the hidden layer structure of each BNN comprises a plurality of different types of nodes, wherein the plurality of different types of nodes are configured to capture a plurality of different types of effects of patient-level variables.

[0303] 162. The computer program product according to aspect 161, wherein the first layer of each component model of each BNN comprises a plurality of different types of nodes.

[0304] 163. The computer program product according to any one of aspects 159 to 162, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of training each ensemble ML model.

[0305] 164. The computer program product according to aspect 163, wherein for each enrollment ensemble ML model the training comprises:

[0306] sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial enrollment based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; and

[0307] minimizing a loss function comprising a squared penalty function during training of each component model, the squared penalty function comprising a squared error between predicted and actual enrollment outcomes.

[0308] 165. The computer program product according to aspect 163, wherein for each screening ensemble ML model the training comprises:

[0309] sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial screening based on the patient-level variables,wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; and

[0310] minimizing a loss function comprising a squared penalty function during training of each component model, the squared penalty function comprising a squared error between predicted and actual screening outcomes.

[0311] 166. The computer program product according to aspect 164 or 165, wherein the training is configured to preserve the predictive influence of multiple patient-level variables during training.

[0312] 167. The computer program product according to any one of aspects 163 to 166, wherein the plurality of individuals associated with the clinical trial comprises individuals whose distribution of social identifier variable values approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial.

[0313] 168. The computer program product according to any one of aspects 163 to 167, wherein the training comprises performing k-fold cross-validation.

[0314] 169. The computer program product according to any one of aspects 163 to 168, wherein the training comprises removing patient-level variables having an estimated influence below a predetermined threshold from consideration in the final model.

[0315] 170. The computer program product according to aspect 169, wherein the removing comprises performing backward elimination.

[0316] 171. The computer program product according to any one of aspects 163 to 170, wherein the dataset used to train each ensemble ML model consists only of individuals from the subpopulation for which the model is configured to determine the relative influence of patientlevel variables on clinical trial screening or enrollment.

[0317] 172. The computer program product according to any one of aspects 125 to 171, wherein determining the relative influence of each patient-level variable for an ensemble ML model comprises calculating a quantitative influence metric for each patient-level variable.

[0318] 173. The computer program product according to aspect 172, wherein the quantitative influence metric comprises a Shapley-based value representing the average contribution of each patient-level variable to model predictions.

[0319] 174. The computer program product according to any one of aspects 125 to 173, wherein the instructions, when executed by the processor, further cause the processor to perform theoperations of using the trained ensemble ML models to determine an individual patient’s probability of clinical trial enrollment based on the patient-level variables of the patient.

[0320] 175. The computer program product according to any one of aspects 125 to 174, wherein optimizing recruitment of individuals for the clinical trial comprises allocating recruitment resources based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

[0321] 176. The computer program product according to any one of aspects 125 to 175, wherein optimizing recruitment of individuals for the clinical trial comprises generating a recruitment plan based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

[0322] 177. The computer program product according to any one of aspects 125 to 176, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of generating a recruitment report based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

[0323] 178. The computer program product according to aspect 177, wherein the recruitment report comprises one or more of a summary of differences in the relative influence of patient-level variables between the overall population cohort determined by the first ensemble ML model and each of the one or more subpopulations, a recommended allocation of recruitment resources, and a proposed recruitment plan.

[0324] 179. The computer program product according to aspect 178, wherein the recruitment report is generated using a natural language processing (NLP) model.

[0325] 180. The computer program product according to aspect 179, wherein the NLP model is a large language model (LLM).

[0326] 181. The computer program product according to any one of aspects 125 to 180, wherein the participant demographics of the clinical trial are optimized such that the distribution of social identifier variable values among recruited individuals better approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial.

[0327] 182. The computer program product according to any one of aspects 125 to 181, wherein the participant demographics of the clinical trial are optimized in order to better represent one or more subpopulations underrepresented in prior clinical trials addressing the health condition.183. The computer program product according to any one of aspects 125 to 182, wherein the participant demographics of the clinical trial are optimized to comply with one or more regulatory or institutional demographic guidelines for clinical trial enrollment.

[0328] 184. The computer program product according to any one of aspects 125 to 183, wherein the participant demographics of the clinical trial are optimized to increase biological heterogeneity among trial participants.

[0329] 185. The computer program product according to any one of aspects 125 to 184, wherein the operations are implemented automatically using lines of computer code.

[0330] 186. The computer program product according to aspect 185, wherein the instructions are embodied on a non-transitory computer-readable medium.EXPERIMENTAL

[0331] As demonstrated in the above disclosure, the present invention has a wide variety of applications. The following examples are put forth so as to provide those of ordinary skill in the art with a complete disclosure and description of how to make and use the present invention and are not intended to limit the scope of what the inventors regard as their invention nor are they intended to represent that the experiments below are all or the only experiments performed. Those of skill in the art will readily recognize a variety of noncritical parameters that could be changed or modified to yield essentially similar results. Efforts have been made to ensure accuracy with respect to numbers used (e.g. amounts, percentages, etc.) but some experimental errors and deviations should be accounted for.

[0332] Example 1; Predicting Therapeutic Clinical Trial Enrollment for Adult Patients with Low- and High-Grade Glioma Using Supervised Machine Learning

[0333] 1.1. Overview

[0334] Therapeutic clinical trial enrollment does not match glioma incidence and mortality across demographics. Traditional statistical methods have identified independent predictors of trial enrollment; however, understanding of the interactions between these factors remains limited. To test the interactive effects of demographic, socioeconomic, and oncologic variables on therapeutic trial enrollment, boosted neural networks (BNN) for the general glioma patient population (n=1042), women (n=445, 42.7%), and minorities (n=151, 14.5%) were designed and externally validated (whole cohort: n=230, women: n=89 (38.7%), minority: n=66 (28.7%)). For the whole cohort BNN, the most influential variables on enrollment were oncologic variables, including Karnofsky Performance Status (KPS) (Total effect (TE): 0.327), chemotherapy (TE: 0.326), tumor location (TE: 0.322), and seizures (TE: 0.239). The women only BNN exhibited a similar trend. Conversely, for the minority only BNN, the most influential predictors were socioeconomic variables (insurance status (TE: 0.213), occupation classification (TE: 0.204), and employment status (TE: 0.150)). These results may help prioritize personalized patient-specific initiatives to address accrual barriers.1.2. Introduction

[0335] Clinical trials offer high quality medical care and help preserve hope for patients with cancer [1, 2], However, national enrollment rates have yet to match disease incidence and mortality across demographics [3-5], Previous studies have used traditional statistical methods to identify independent factors associated with clinical trial enrollment [6-8]; however, understanding of the nuanced interactions between these factors has remained limited.

[0336] Harnessing machine learning techniques may serve as an innovative approach to assess the influence of prognostic variables and their complex relationships on trial accrual, improving understanding of how enrollment obstacles can be overcome for select groups of patients [9-11], The NIH Revitalization Act of 1993 established disease-specific accrual benchmarks to increase representation in clinical trials. Despite these benchmarks, recent inquiry demonstrates alarming incidence-specific and mortality-specific under-accrual of women and minority patients with low- and high-grade glioma [3], Specifically, women represent 44.1% of incident cases and 44.2% of disease mortality, but only 37.7% of trial accrual [3], Similarly, minority patients represent 16.9% of incident cases and 20.1% of disease mortality, but only 5.9% of trial accrual [3], Additionally, many of the landmark clinical research studies supporting modern-day cancer-directed therapies for patients with diffuse gliomas are built on datasets with suboptimal representation of women and minorities [12, 13], Therefore, increasing the enrollment of women and minority patients in glioma therapeutic clinical trials is paramount for ensuring generalizability of clinical research.

[0337] Recent investigations have demonstrated the potential of supervised machine learning to improve various aspects of participant-based study design [14, 15], including operational efficiency

[0016] , eligibility screening [17-19], study matching

[0020] , electronic health record data extraction [21, 22], and dynamic adjustment of outreach sites and durations [23, 24], However, these studies have not focused on increasing equitable clinical trial enrollment for patients and, more specifically, equitable enrollment for patients with brain cancer. Thus, the present disclosure strives to close this critical gap in the literature. The primary aim is to design and externally validate machine learning models to test the interactive effects of demographic, socioeconomic, and oncologic variables on clinical trial enrollment for the general glioma patient population, women, and minorities. The secondary aim is to evaluate factors influencing trial screening and assess unique enrollment drivers across subgroups.1.3. Results

[0338] Patient characteristics for the development cohort (University of California San Francisco (UCSF)) and the external validation cohort (Duke University, University of Michigan, and Dana Farber Cancer Institute) are presented in FIG. 8. The development cohort included 1042 patients, 445 of whom were women (42.7%) and 151 of whom were minorities (14.5%). The validation cohort included 230 patients, 89 of whom were women (38.7%) and 66 of whom were minorities (28.7%). The development and validation cohorts did not differ significantly by sex, but the validation cohort had a significantly higher percentage of Black / African American patients (11.7% vs. 1.6%) and Hispanic / Latino patients (8.7% vs. 5.5%) compared to the development cohort. The percentage of minority patients in the development cohort matched the percentage of adult glioma patients who are minorities in the United States (expected: 16.9%, observed 14.5%), but several demographics differed (American Indian / Alaska Native - expected: 0.5%, observed: 0.4%; Asian - expected: 1.9%, observed: 4.6%; Hispanic / Latino - expected: 8.6%, observed: 5.5%, Black / African American - expected: 6.0 >, observed: 1.6%) [3], The overall percentage of minority patients in the validation cohort was greater than expected based on glioma incidence rates (expected: 16.9%, observed 28.7%), but approximated a mid-point between the percentage of adult minority glioma patients and minority individuals in the United States (-42.6%) [3, 25], Demographics of the validation cohort matched or exceeded expected rates (American Indian / Alaska Native - expected: 0.5%, observed: 0.9%; Asian - expected: 1.9%, observed: 5.2%; Hispanic / Latino - expected: 8.6%, observed: 8.7%, Black / African American - expected: 6.0%, observed: 11.7%) [3], For the development cohort, 995 patients (95.5%) preferred English for medical communication. Six-hundred-and-forty -three patients (62.0%) had private insurance, 83 patients (8.0%) had Medi-Cal / Medicaid, and 182 patients (17.5%) had Medicare. Six-hundred-and-fifty-eight patients (63.2%) had WHO grade 4 gliomas. Additional details on the characteristics of the Duke University, University of Michigan, Dana Farber Cancer Institute subsets of the external validation cohort are provided in FIG. 10.

[0339] Of the 1042 patients with a median follow-up time of 28.8 months (IQR: 12.1 - 106.1), 628 (60.3%) were screened for and 350 (33.6%) enrolled in a glioma therapeutic clinical trial (FIG. 10). One-hundred-and-forty-four of the 445 women (32.4%) and 41 of the 151 minority patients (27.2%) enrolled in a trial (FIGS. 11-12). Clinical trial screening rates (validation vsdevelopment: 80.9% vs. 60.3%) and enrollment rates (validation vs development: 58.7% vs. 33.6%) were higher for the validation cohort than for the development cohort. Patient characteristics for the women and minority subsets of the development and validation cohorts are presented in FIG. 11.

[0340] FIG. 8 provides a table of demographic, socioeconomic, and oncologic characteristics of the UCSF development cohort by therapeutic clinical trial enrollment. Patient characteristics for both the development cohort and the external validation cohort are presented. Abbreviations: SD = standard deviation, NIH = National Institute of Health, ISCO = International Standard Classification of Occupations, IQR = interquartile range, UCSF = University of California, San Francisco, WHO = World Health Organization, KPS = Kamofsky Performance Scale; indicators * = Difference calculated using a two-tailed t-test, ** = Difference calculated using a Wilcoxon rank-sum test).

[0341] FIG. 10 provides a table of demographic, socioeconomic, and oncologic characteristics of the Duke University, University of Michigan, and Dana Farber Cancer Institute subsets of the combined validation cohort. Abbreviations: UCSF = University of California, San Francisco, SD = standard deviation, IQR = interquartile range, WHO = World Health Organization, KPS = Kamofsky Performance Scale; indicators: * = Difference calculated using a two-tailed t-test, ** = Difference calculated using a Wilcoxon rank-sum test.

[0342] FIG. 11 provides a table of demographic, socioeconomic, and oncologic characteristics of women and minority subsets of the development and validation cohorts. Abbreviations: UCSF = University of California, San Francisco, SD = standard deviation, IQR = interquartile range, WHO = World Health Organization, KPS = Kamofsky Performance Scale, IDH = isocitrate dehydrogenase; indicators: * = Difference calculated using a two-tailed t-test, ** = Difference calculated using a Wilcoxon rank-sum test

[0343] FIG. 12 shows the results of univariate and multivariate logistic regression analysis for factors associated with clinical trial enrollment among those screened for a clinical trial.

[0344] Abbreviations: OR = odds ratio, CI = confidence interval, ISCO = International Standard Classification of Occupations, UCSF = University of California, San Francisco, WHO = World Health Organization, KPS = Kamofsky Performance Scale, IDH = isocitrate dehydrogenase.1.3.1. Machine learning prediction of clinical trial enrollment and screening

[0345] Model screening with internal validation: On internal cross-validation, the boosted neural network (BNN) model achieved the best predictive performance for enrollment (mean AUC across 10 folds: 0.7840, accuracy: 0.7249, misclassification rate: 0.2743, MCC: 0.3849) and was thus selected for external validation (FIG. 16).

[0346] FIG. 16 provides a comparison of nine machine learning techniques for prediction of clinical trial enrollment, screened using 10-fold internal cross-validation. Abbreviations: Sum Freq = Sum of Frequency Column, RASE = Root Average Squared Error, AUC = Area Under the Curve, MR = Misclassification Rate, MCC = Mathews Correlation Coefficient

[0347] Model performance on validation cohort for trial screening: Development models were computed using patients at UCSF, and validated against glioma patients treated at Duke University, University of Michigan, and the Dana-Farber Cancer Institute. The whole cohort BNN model achieved good predictive performance for trial screening in both the development [AUC: 0.8677, accuracy: 79.9%, MR: 0.201, MCC: 0.575] and validation [AUC: 0.8372, accuracy: 84.4%, MR: 0.157, MCC: 0.4274] cohorts (FIG. 4). The women only BNN model (Development: AUC: 0.8915, accuracy: 79.8%, MR: 0.202, MCC: 0.569; Validation: AUC: 0.8575, accuracy: 83.2%, MR: 0.169, MCC: 0.4971) and the minority only BNN model (Development: AUC: 0.930, accuracy: 87.4%, MR: 0.126, MCC: 0.734; Validation: AUC:

[0348] 0.8202, accuracy: 83.6%, MR: 0.164, MCC: 0.603) also accurately predicted trial screening (FIG. 4).

[0349] For all cohorts, the most influential variables predicting trial screening were oncologic variables, including treatment with chemotherapy (whole cohort - TE: 0.344; women - TE:

[0350] 0.354; minority - TE: 0.140), KPS (whole cohort - TE: 0.308; women - TE: 0.295; minority - TE: 0.216), and tumor location (whole cohort - TE: 0.219; women - TE: 0.289; minority - TE: 0.198), followed by a blend of socioeconomic, demographic, and other oncologic variables (FIG. 4).

[0351] FIG. 4 depicts ROC curves and variables of importance for predicting therapeutic clinical trial screening using BNN models for the whole, women only, and minority only cohorts. In all ROC curves, the red lines represent clinical trial screening while the blue lines represent lack of screening. The area under the curve (AUC) values for each ROC curve are listed. Each variable’s main and total effects are listed in variables of importance output for each model. Adjacent to the variables of importance outputs are color-coded heatmaps indicating whether the ranked factorsare demographic, socioeconomic, or oncologic variables. A) The ROC curve for the whole development cohort; B) The ROC curve for the whole validation cohort; C) The variables of importance for the whole cohort model in descending order of importance; D) The ROC curve for the women development cohort; E) The ROC curve for the women validation cohort; F) The ranked variables of importance for the women cohort model in descending order of importance; G) The ROC curve for the minority development cohort; H) The ROC curve for the minority validation cohort; I) The ranked variables of importance for the minority cohort model in descending order of importance.

[0352] Model performance on validation cohort for trial enrollment: The whole cohort BNN model achieved good predictive performance in both the development [AUC: 0.8314, accuracy: 76.6%, MR: 0.234, MCC: 0.461] and validation [AUC: 0.8205, accuracy: 76.1%, MR: 0.239, MCC: 0.523] cohorts (FIG. 1). The women only BNN model (Development: AUC: 0.8520, accuracy: 78.7%, MR: 0.213, MCC: 0.502; Validation: AUC: 0.8280, accuracy: 71.3%, MR: 0.281, MCC: 0.469) and the minority only BNN model (Development: AUC: .8756, accuracy: 84.1%, MR: 0.159, MCC: 0.598; Validation: AUC: 0.8262, accuracy: 77.6%, MR: 0.224, MCC: 0.583) also accurately predicted enrollment (FIG. 1).

[0353] The most influential variables in predicting trial enrollment varied by cohort (FIG. 1). For the whole cohort BNN model, oncologic variables, including KPS (TE: 0.327), chemotherapy (TE: 0.326), tumor location (TE: 0.322), and seizures (TE: 0.239), were most influential on enrollment, followed by socioeconomic (employment status (TE: 0.216), insurance status (TE: 0.175)) and demographic (age (TE: 0.128), race (TE: 0.093)) variables (FIG. 1C). Similarly, for the women only BNN model, oncologic variables (chemotherapy (TE: 0.363), KPS (TE: 0.176), tumor location (TE: 0.146)) were most influential; however, a blend of socioeconomic (employment status (TE: 0.119), occupation classification (TE: 0.085), insurance status (TE: 0.063), household income (TE: 0.063)), and other oncologic variables (seizures (TE: 0.081), WHO grade (TE: 0.069)) were also highly influential (FIG. IF).

[0354] In contrast, for the minority only BNN model, the most influential predictors were socioeconomic variables (insurance status (TE: 0.213), occupation classification (TE: 0.204), employment status (TE: 0.150)), followed by a mix of oncologic (KPS (TE: 0.186), tumor location (TE: 0.0804), chemotherapy (TE: 0.082), seizure (TE: 0.059)), demographic (age (TE:0.084), sex (TE: 0.024)), and other socioeconomic (distance from hospital (TE: 0.062), household income (TE: 0.021)) variables (FIG. II).

[0355] Next, given slight differences in minority populations between institutions included in this study, additional analysis were performed on a combined cohort of minority only patients treated at UCSF, Duke, Michigan, and Dana-Farber Cancer Institute. Similar to the main minority only BNN model, the combined minority cohort BNN model prioritized socioeconomic variables (occupation classification (TE: 0.373), insurance status (TE: 0.335), and employment status (TE: 0.263) as the three variables most influential on enrollment (FIG. 4).

[0356] Mean Shapley values were calculated to assess the directionality of each predictor, with positive and negative values indicating increased and decreased probability of enrollment, respectively. For the whole cohort BNN model, mean Shapley values were significantly higher (p<0.0001) for KPS>80 (vs. <80, 0.071 vs. -0.131), treatment with chemotherapy (vs. no chemotherapy, 0.057 vs. -0.136), multifocal tumor location (vs. local, 0.109 vs. -0.001), presenting seizures (vs. no seizures, 0.035 vs. -0.058), active employment (vs. unemployment, 0.023 vs. -0.029), private insurance (vs. public insurance, 0.007 vs. -0.052), age less than 60 years (vs. >60, 0.007 vs. -0.016), White race (vs. Non-white, 0.009 vs. -0.064), WHO grade 4 (vs. grade 2-3, 0.030 vs. -0.054), and non-Hispanic ethnicity (vs. Hispanic ethnicity, 0.004 vs. -0.046) (FIG. 2). Interestingly, the mean Shapley value was significantly higher for female sex than male sex (0.001 vs. -0.001, p<0.0001), though the net difference in mean Shapley values was slight (0.002). The directionality and significance of these relationships were mainly maintained for the women and minority BNN models (FIGS. 5-7). Additionally, there were unique relationships for Shapley values for the women and minority only BNN models. For both the women and minority BNN models, mean Shapley values were significantly higher for patients with median household income by zip code greater than $75,000 ([women: vs. less than $75,000, 0.019 vs -0.031, p<0.0001], [minority: vs. less than $75,000, 0.003 vs -0.026, p<0.0001]) and patients with higher ISCO skill level occupations ([women: ISCO skill level 3-4 vs. ISCO skill level 1-2, 0.005 vs -0.032, p<0.0001], [minority: ISCO skill level 4 vs. ISCO skill level 1-2, 0.018 vs -0.057, pO.OOOl], Lastly, for the minority BNN model, the mean Shapley value was significantly higher for travel distance less than 50 miles (0.054 vs. -0.007, p<0.0001).

[0357] FIG. 1 depicts ROC curves and variables of importance for predicting therapeutic clinical trial enrollment using BNN models for the whole, women only, and minority only cohorts. In allROC curves, the red lines represent clinical trial enrollment while the blue lines represent lack of enrollment. The area under the curve (AUC) values for each ROC curve are listed. The main and total effects for each variable are listed in variables of importance output for each model.

[0358] Adjacent to the variables of importance outputs are color-coded heatmaps indicating whether the ranked factors are demographic, socioeconomic, or oncologic variables. A) The ROC curve for the whole development cohort; B) The ROC curve for the whole validation cohort; C) The variables of importance for the whole cohort model in descending order of importance; D) The ROC curve for the women development cohort; E) The ROC curve for the women validation cohort; F) The ranked variables of importance for the women cohort model in descending order of importance; G) The ROC curve for the minority development cohort; H) The ROC curve for the minority validation cohort; I) The ranked variables of importance for the minority cohort model in descending order of importance.

[0359] FIG. 2 illustrates the distribution of Shapley values for each influential predictor in the whole cohort BNN model. Each dot corresponds to the Shapley value for each predictor for a single patient. Positive and negative Shapley values indicate increased and decreased probability of enrollment, respectively. Each predictor is stratified into two levels, with the characteristic associated with higher mean Shapley values color-coded in dark blue and the characteristic with the lower mean Shapley values color-coded in dark red. T-tests were used to assess for differences in mean Shapley values for each predictor. Predictors are arranged from most influential to least influential (left to right) according to the variables of importance calculation for the predictive model. Indicators: *** = p<0.0001; ** = p<0.01.

[0360] FIG. 5 depicts the ROC curve and the variables of importance for a boosted neural network (BNN) algorithm that was trained on the combined minority patients from the development and validation cohorts. In the ROC curve, the red line represents clinical trial enrollment while the blue line represents lack of enrollment. The area under the curve (AUC) value for the ROC curve is listed. The main and total effects for each variable are listed in the variables of importance output for the BNN model. Adjacent to the variables of importance outputs are color-coded heatmaps indicating whether the ranked factors are demographic, socioeconomic, or oncologic variables. A) The ROC curve for the BNN algorithm trained on the combined minority cohort B) The variables of importance for the BNN model in descending order of importance.FIG. 6 illustrates the distribution of Shapley values for each influential predictor in the women only cohort BNN model. Each dot corresponds to the Shapley value for each predictor for a single patient. Positive and negative Shapley values indicate increased and decreased probability of enrollment, respectively. Each predictor is stratified into two levels, with the characteristic associated with higher mean Shapley values color-coded in dark blue and the characteristic with the lower mean Shapley values color-coded in dark red. T-tests were used to assess for differences in mean Shapley values for each predictor. Predictors are arranged from most influential to least influential (left to right) according to the variables of importance calculation for the predictive model. Indicators: *** = pO.OOOl.

[0361] FIG. 7 illustrates the distribution of Shapley values for each influential predictor in the minority only cohort BNN model. Each dot corresponds to the Shapley value for each predictor for a single patient. Positive and negative Shapley values indicate increased and decreased probability of enrollment, respectively. Each predictor is stratified into two levels, with the characteristic associated with higher mean Shapley values color-coded in dark blue and the characteristic with the lower mean Shapley values color-coded in dark red. T-tests were used to assess for differences in mean Shapley values for each predictor. Predictors are arranged from most influential to least influential (left to right) according to the variables of importance calculation for the predictive model. Indicators: *** = p<0.0001.

[0362] 1.3.2. Factors associated with clinical trial enrollment on multivariate logistic regression For the whole cohort, higher household income (OR [95% CI]: 1.079 per $10,000 increase [1.029-1.132]; p=0.002), higher WHO grade (4 vs 2-3, OR [95% CI]: 1.609 [1.049-2.467]; p=0.029), seizure (OR [95% CI]: 1.630 [1.181-2.249]; p=0.003), and higher KPS (>80 vs <80, OR [95% CI]: 2.802 [1.729-4.540]; p<0.0001) were associated with increased odds of enrollment. Non-English preferred language (OR [95% CI]: 0.005 [0.0001-0.206]; p=0.005), lack of interpreter use (OR [95% CI]: 0.011 [0.0003-0.426]; p=0.016), Medicare insurance (OR [95% CI]: 0.497 [0.300-0.822]; p=0.007) and lack of chemotherapy (OR [95% CI]: 0.248 [0.150-0.409]; p<0.0001) were associated with decreased odds of enrollment (FIG. 9).

[0363] Among patients screened for a trial, discussing more trials during screening (OR [95% CI]: 2 trials, (2.101 [1.336-3.302]; p=0.001); 3 trials, (OR 2.201 [1.107-4.377]; p=0.025); 5+ trials, (OR 5.073 (1.065-24.174); p=0.042) was associated with a dose-dependent increase inodds of enrollment (FIG. 12). Fewer trials were discussed with women (vs. men, mean+ / -SD: 1.61+0.96 vs. 1.89+1.16, p=0.0001), minorities (vs. non-minorities, mean+ / -SD: 1.49+0.63 vs.

[0364] 1.81+1.14, p=0.0002), and patients who preferred anon-English language (vs. English, mean+ / -SD: 1.25+0.45 vs. 1.78+1.09, p=0.002) (FIG. 13). Higher household income, seizure, higher KPS, and chemotherapy treatment were similarly associated with trial screening (FIG. 14). No unique variables were associated with enrollment only for women or minority patients (FIG. 15).

[0365] FIG. 9 shows the results of univariate and multivariate logistic regression for factors associated with therapeutic clinical trial enrollment for the whole cohort. Abbreviations: OR = odds ratio, CI = confidence interval, ISCO = International Standard Classification of Occupations, UCSF = University of California, San Francisco, WHO = World Health Organization, KPS = Karnofsky Performance Scale.

[0366] FIG. 13 demonstrates the association between sex, minority status, and preferred language and the number of clinical trials discussed with patients during trial screening among all patients with known *sex, **minority status, and ***preferred language and a specified number of clinical trials discussed during screening.

[0367] FIG. 14 shows the results of univariate and multivariate logistic regression for factors associated with screening for a clinical trial among all patients in the development cohort.

[0368] Abbreviations: OR = odds ratio, CI = confidence interval, NIH = National Institute of Health, ISCO = International Standard Classification of Occupations, UCSF = University of California, San Francisco, WHO = World Health Organization, KPS = Karnofsky Performance Scale, IDH = isocitrate dehydrogenase

[0369] FIG. 15 shows the results of univariate and multivariate logistic regression for factors associated with clinical trial enrollment for the women only and minority only subsets of the development cohort. Abbreviations: OR = odds ratio, CI = confidence interval, NIH = National Institute of Health, ISCO = International Standard Classification of Occupations, UCSF = University of California, San Francisco, WHO = World Health Organization, KPS = Karnofsky Performance Scale, IDH= isocitrate dehydrogenase

[0370] 1.4. Discussion and Conclusions

[0371] Machine learning models were designed and externally validated to accurately predict therapeutic clinical trial enrollment for patients with low- and high-grade glioma. The BNNmodels performed well across cohorts for trial enrollment and screening. For screening, oncologic variables were most influential across all cohorts. For enrollment, oncologic variables continued to be most influential for the whole cohort and women models; however, for minority patients, socioeconomic variables were most influential. The directionality of influence of enrollment predictors was maintained across cohorts.

[0372] For the whole cohort model, the oncologic variables of higher KPS, multifocal tumor, treatment with chemotherapy, and presenting seizures were most influential on enrollment and associated with a higher average probability of enrollment. The multivariate logistic regression analysis substantiated these relationships. Collectively, these findings suggest that higher symptom burden may motivate patients to enroll if their functional status meets inclusion criteria and they are willing to undergo cancer-directed treatments, such as chemotherapy. There may be a specific significance of tumor location influencing enrollment as patients with a tumor that is multifocal or difficult to resect meaningfully may derive limited potential benefits from standard treatment and instead pursue investigational therapies.

[0373] In multivariate logistic regression analysis, race and sex were not found to be independently associated with clinical trial enrollment. Further, multivariate regressions for the women and minority subgroups did not identify unique drivers of enrollment. However, the BNN models for all cohorts suggest that race, ethnicity, and sex are essential enrollment drivers. For example, non-White race and non-Hispanic ethnicity were associated with lower average probability of enrollment. Thus, in contrast to conventional statistical analysis, the BNN models can capture the interconnectedness between demographic and socioeconomic variables in influencing enrollment in a manner that parallels how such characteristics influence complex enrollment decisions for a unique patient [9], Therefore, this study can serve as a bridge to an era of precision clinical trial research in which machine learning techniques can map which factors most influence enrollment for individual patients to guide personalized recruitment interventions to increase accrual

[0011] , On the other hand, the minority cohort model underscores the importance of addressing root causes of under-accrual in trials. For example, the influence of socioeconomic variables on enrollment was more pronounced for minority patients, with higher ISCO skill level occupations, active employment, and private insurance having a higher average probability of enrollment. These findings may reflect patient-specific differences in occupation, income, health literacy, and health insurance, which produce and exacerbate gaps in trialenrollment for minority patients [26, 27], While factors influencing enrollment for minority glioma patients were assessed in aggregate in this study, the present methods of the disclosure may also be used to assess how these factors may differ (e.g., in their influence) for specific racial and ethnic groups.

[0374] In contrast to national trends, baseline enrollment rates for men and women in the present study were comparable. In the women only cohort, non-White race was associated with lower probability of enrollment, underscoring how race and sex can intersect to influence accrual

[0028] , Therefore, recruiting more women may fall short of achieving benchmark accrual of minority women. Surprisingly, in the BNN model, the mean SHAP value for female sex was slightly higher than that for male sex [3], This counterintuitive finding could relate to a smaller sample size in this multi-institution study than national cohort studies or unique institutional factors. An alternative explanation is that prior studies focused on the under-accrual of women in glioma trials have not used statistical methods that fully account for covariates, such as differences in oncologic or socio-economic factors [3], The BNN model raises the possibility that when accounting for these complex interactions, female sex may not be associated with lower probability of enrollment, but potentially even a slight increase in enrollment probability — a hypothesis that may be verified with additional exploration such as, e.g., use of the BNN model with larger patient cohorts.

[0375] It was found that oncologic variables were consistently most influential in predicting trial screening across all cohorts, suggesting that clinical status triggers clinicians to discuss trial options with patients. It was also found that discussing more trials during screening was associated with progressively higher odds of enrollment. However, fewer trials were discussed with women, minorities, and patients who preferred a non-English language. This disparity may limit shared decision-making and relate to factors including baseline patient interest or socioeconomic obstacles. Further, patients underrepresented in clinical trials may be less likely to initiate discussions with their clinicians about clinical trial options due to differences in health literacy or medical mistrust [29, 30], Therefore, encouraging clinicians to adopt a “just ask” policy for clinical trials discussion for all eligible patients, increasing provider awareness of available trials, and maximizing the number of trials discussed during screening may increase accrual. Further, among patients who preferred a non-English language, not using an interpreter was associated with an additional reduction in enrollment odds. Thus, it is imperative to not onlydemocratize access to interpreter services, but also prioritize their use during medically complex screening and enrollment discussions, even for patients with moderate English proficiency who otherwise might forego medical interpretation for other healthcare encounters [31, 32].

[0376] Notably, non-oncological attributes are influential variables on trial enrollment

[0033] , As a result, if the BNN model were used to target recruitment of patients with higher projected enrollment probability without efforts to address barriers to enrollment, this could exacerbate the under-accrual of vulnerable groups of patients [3, 4, 34], This is particularly relevant given prior studies showing an association between trial enrollment and improved survival for patients with high-grade glioma

[0035] , Ideally, future efforts will result in balanced datasets for algorithm training; however, in the path towards reaching this goal, bias-correcting techniques (such as, e.g., those of the disclosure) can be used to identify and address model bias before deployment in the clinical setting [36, 37].

[0377] Several aspects of the present study may be further developed or modified in order to address the unique issues or challenges faced during implementation. Several of such limitations have been identified (and, in some cases, addressed) herein. First, while “black box” neural networks obscure relationships between variables and outcomes [38, 39], Shapley values were used to help explain model decision-making

[0040] , However, given that a few predictors were moderately correlated, the SHAP analysis may be modified — or a different metric or method used to assess variable / outcome relationships (e.g., as discussed in the above Methods) — as the SHAP analysis of this study could potentially overestimate the individual contributions of correlated features. However, the risk of predictor misattribution is reduced due to the lack of highly correlated predictors. Second, the sample size of minority patients in the development and validation cohorts creates the risk of overfitting the minority only BNN model. Third, the racial and ethnic breakdowns of the development and validation cohorts may not perfectly reflect national demographics, which could lead to a different ranking of influential variables than found for centers in other regions of the United States. To overcome these limitations, the minority patients from the development and validation datasets were pooled to augment sample size and generate a more generalizable model, which prioritized similar model features. Fourth, since documentation in the electronic medical record was relied on to determine whether patients were screened for clinical trials, there is the possibility that some of these patient-clinician discussions occurred but were not documented. Fifth, the models of this study were not configured to capturethe variance in the number of clinical trials available to each patient throughout the study period — which may have influenced recruitment rates. Accordingly, future embodiments of the models may be modified accordingly. Sixth, enrollment outcomes may have been impacted by patients lost to follow-up — which may be accounted for in future model training datasets.

[0378] Furthermore, certain patients in the development and training cohorts may have enrolled in a therapeutic clinical trial at an outside institution that the study datasets could not capture due to non-integrated electronic health records, leading to potential misclassification of a small number of patients. Lastly, zip codes were used as a proxy for income and poverty at the population level, which may not perfectly match the socioeconomic characteristics of individual patients. Each of these issues may be addressed by future BNN model training datasets.

[0379] To facilitate the evaluation of the BNN model’s predictive performance in additional settings and cohorts, GPREDICT was created. This open-access tool estimates the probability that a given patient with a glioma will enroll in a therapeutic clinical trial based on patient, oncologic, and socioeconomic characteristics inputted by the user. GPREDICT also provides general recommendations for clinical trial screening discussions, identifies the patient factors that most influence estimated enrollment probability, and provides users with suggestions for targeted interventions to increase each patient’s accrual probability. The GPREDICT tool is available here, wherein https: / / should be inserted before the characters inside of the following quotations: “mmehari2025.github.io / glioma-trial-predictor / ”.

[0380] This study, and the methods of the present disclosure, can serve as a bridge to an era of precision clinical trial enrollment research which harnesses machine learning techniques to optimize clinical trial participation for all patients with glioma, among other health conditions. These findings can guide tailored efforts to promote equitable accrual across diverse patient populations.

[0381] FIG. 19 shows the GPREDICT tool for estimating patient enrollment probabilities for a given therapeutic clinical trial based on patient-level variables input by a user, in accordance with embodiments of the disclosure.

[0382] FIGS. 20A to 20B demonstrates the use of a database for obtaining socioeconomic variables for an individual, according to embodiments of the disclosure, as made conveniently available through the user interface of the GPREDICT tool.FTGS. 21 A to 21 B demonstrates the use of the GPREDICT tool to estimate patient enrollment probabilities for a given therapeutic clinical trial based on patient-level variables input by a user, according to embodiments of the disclosure. General recommendations for clinical trial screening discussions, identification of the patient factors that most influenced the estimated enrollment probability, and suggestions for targeted interventions to increase the patient’s accrual probability are also provided by the tool.

[0383] 1.5. Materials and Methods: Overview

[0384] 1.5.1. Study design

[0385] In this retrospective, multicenter cohort study, machine learning algorithms were trained to predict therapeutic clinical trial screening (secondary outcome) and enrollment (primary outcome). Separate models were generated for all patients, women, and minority patients in the development cohort and then independently validated in their respective external cohorts, as depicted in FIG. 3. Minority patients were defined as any patient with Black or African American, Asian, American Indian or Alaska Native, or Native Hawaiian or Pacific Islander race and / or Latino or Hispanic ethnicity according to the current United States Office of Management and Budget’s Race / Ethnicity Standards

[0041] , Additional methods are included in Example 2 of the present disclosure, provided below.

[0386] One-thousand-forty -two adult patients with low- and high-grade glioma who received care from the UCSF Brain Tumor Center between January 1, 1997, through December 31, 2017, were identified in the UCSF Cancer Registry. Twenty-four demographic, socioeconomic, oncologic, and therapeutic clinical trial screening and enrollment variables were extracted from the registry and electronic health record (FIG. 8). Therapeutic clinical trial screening was defined as a chart documented discussion about trial enrollment during consultation with neuro-oncology or neurosurgery. Clinical trial enrollment was defined as enrolling in a therapeutic phase I-IV clinical trial at UCSF. At UCSF, there were 238 therapeutic trials for glioma open at the UCSF Brain Tumor Center from January 1997 to December 2017. The total enrollment during this time period was 2661 patients, equating to an average annual enrollment of 126 patients per year.

[0387] External validation was performed on a combined cohort of adult glioma patients from three external institutions: Duke University, University of Michigan, and Dana Farber Cancer Institute (FIG. 8 and FIG. 10). Each external institution provided demographic, socioeconomic,oncologic, and therapeutic clinical trial screening and enrollment variables, with enrollment defined as enrolling in a therapeutic phase I-IV clinical trial at their respective institutions. The Duke subset consisted of 103 adult glioma patients treated between January 20, 2017, and December 4, 2023. The Michigan subset consisted of 90 adult glioma patients treated between January 1st, 2017, and December 31st, 2023. The Dana Farber Cancer Institute subset consisted of 37 patients treated between January 30, 2004, and December 20, 2017. For additional context regarding the landscape of clinical trials at each external institution, it was sought to determine the number of available clinical trials and enrollment rates during the periods specified above. At Duke, there were 57 therapeutic glioma trials with an average annual enrollment rate of 120 patients. At Michigan, there were 20 available therapeutic glioma trials with an average yearly enrollment rate of 90 patients. Lastly, at the Dana Farber Cancer Institute there were 95 available clinical trials with an average annual enrollment rate of 50 patients.

[0388] FIG. 3 provides a Prediction of Glioma Patients’ Enrollment in Therapeutic Clinical Trials (GPREDICT) schematic for boosted neural network development and validation for the whole cohort, women only cohort, and minority only cohort. The figure represents four academic medical centers, and the schematic depicts the sequence of dividing the whole cohort into the women only and minority only cohorts for model development, model screening with internal cross-validation, and external validation.

[0389] 1.5.2. Statistical analysis

[0390] Demographic, socioeconomic, and oncologic characteristics were summarized with descriptive statistics. Differences in categorical and continuous variables between cohorts were assessed using chi-square tests and two-tailed t-tests as well as Wilcoxon rank sum tests, respectively. Model screening with 10-fold internal cross validation was used to identify the supervised machine learning technique with the best predictive performance for trial enrollment among the 9 machine learning algorithms readily available in JMP Pro, finding that the boosted neural network (BNN) models performed best across cohorts (see, e.g., Example 2, below). K-fold cross-validation was elected for use, rather than fixed splitting, for internal validation to more reliably estimate model performance by averaging performance metrics across folds and avoid relying on a single test set that may not accurately represent the rest of the development cohort. Predictor screening in JMP Pro 17 was used to identify influential predictors on internalcross-validation. Feature selection with backward elimination was utilized to optimize boosted neural network model efficiency while maintaining high performance. External validation was performed using conserved model features.

[0391] For each BNN model, three hyperbolic tangent function (TanH), linear activation function (Linear), and Gaussian function (Gaussian) nodes were specified in the first layer of the hidden layer structure. To reduce the influence of outliers and heavily skewed distributions, continuous variables were transformed to near normality using the Johnson Su or Johnson Sb distribution for each BNN model. Given that it was hypothesized that most predictor variables were contributing to the predictive ability of the model, the squared method was used for the penalty function. For each model, boosting was performed by specifying 10 component models and a learning rate of 0.1. Nineteen tours were performed for each model corresponding to the number of times that the fitting process was restarted such that each iteration used different random starting points for the parameter estimates. A random number seed was used for each BNN model to create reproducible models.

[0392] Predictive performance was assessed by calculating area under the curve (AUC), accuracy, misclassification rate (MR), and Mathew correlation coefficient (MCC). Using independent resampled inputs, ranked variables of importance by mean effect (ME) and total effect (TE) were generated for each model. To help explain the directionality of predictors’ influence on enrollment, Shapley values were generated using the Permutation SHAP method for each model. Differences in mean Shapley values by predictor variable levels were assessed using two-tailed t-tests. To ensure no strict violations of the assumption of independence for SHAP analysis, correlation matrix analysis was performed for each model and highly correlated predictors were removed (r>0.7)

[0042] , Principal component analysis was further utilized to assess the number of principal components needed to explain greater than 80% of total variance in the data.

[0393] To assess the association between predictors and clinical trial enrollment, univariate followed by multivariate logistic regression was performed. Predictors with a p-value of <0.2 on univariate analysis were selected for multivariate logistic regression. To adjust for multiple comparisons and control for the false discovery rate (FDR), the Benjamini -Hochberg adjustment method was used, calculating the adjusted alpha level to be a = 0.030 based on an FDR of 0.05;therefore, p-values <0.03 were considered statistically significant in this study [43, 44], IMP Pro 17 and Python were used for all analyses.

[0394] Example 2; Supplemental Materials and Methods

[0395] 2.1. De ographic, socioeconomic, and oncologic data collection

[0396] Sex, race, employment status, insurance status, occupation, preferred language, interpreter use, in-state vs. out-of-state location, marital status, and patient zip code were extracted from the electronic health record (EHR). Patients’ occupations were categorized into major groups according to the International Standard Classification of Occupations (ISCO) after extraction from the EHR. Using the U.S. Census Bureau’s 2021 American Community Survey, patients’ zip codes were used to determine median household income and the percentage of the population living below the poverty level by zip code inputted at the population level. Distance from the hospital was determined based on the patient’s home addresses documented in the EHR. Age at diagnosis and oncologic variables, such as WHO tumor grade, tumor type, tumor hemisphere and lobe, Karnofsky Performance Scale (KPS), and receipt of chemotherapy and radiation, were extracted from the UCSF Brain Tumor Cancer Registry.

[0397] 2.2. Quantification of the extent of resection (EOR) (UCSF Cohort)

[0398] Pre- and post-operative tumor volumes were quantified using BrainLab Smartbrush software (Brainlab, Munich, Germany). Pre-operative MRI scans were obtained within 24 hours prior to resection, and post-operative scans were all obtained within 72 hours post-resection. Total contrast-enhancing (CE) and non-enhancing (NE) tumor volumes were measured at both pre-operative and post-operative time points. The total CE tumor volume was measured on Tl-weighted post-contrast images, and the NE tumor volume was measured on T2 or FLAIR sequences. Manual segmentation was performed with region-of-interest analysis painting” inclusion regions based on fluid-attenuated inversion-recovery (FLAIR) sequences from pre- and post-operative MRI scans to quantify tumor volume. Extent of Resection (EOR) was calculated as: (pre-operative tumor volume - post-operative tumor volume) / (pre-operative tumor volume) x 100%. Multifocal or multicentric disease was defined as noncontiguous areas of disease based on Tl-weighted post contrast images or FLAIR sequences. All multifocal lesions were measured separately and summed together for a single volumetric measurement. Volumetric measurementswere made blinded to clinical outcomes. To ensure that post-operative FLAIR signal was not surgically induced edema or ischemia, FLAIR pre- and post-operative MRIs were carefully compared alongside DWI sequences prior to including each region in the volume segmentation.

[0399] 2.3. Minority cohort for BNN model development

[0400] The minority patients from the development and validation cohorts were aggregated into a pooled minority-only cohort. A BNN model was developed on the entire pooled cohort, with a holdback proportion of 0.01. As described above, the model parameters and variables of importance calculations for the BNN model were determined.

[0401] 2.4. Correlation matrix analysis and principal component analysis

[0402] Continuous variables were scaled, and categorical variables were one-hot and encoded to binary predictor levels. The correlation coefficients for each predictor pair were generated. Correlation coefficients between predictor pairs greater than 0.7 were considered highly correlated and coefficients greater than 0.4 and less than 0.7 were considered moderately correlated. Highly correlated predictors were removed from the models to reduce redundancy and generate new models. For each PCA analysis, the percentage of variance explained by each principal component was determined to assess for any dominant principal components and the number of principal components needed to surpass the common threshold of 80% of total variance to evaluate model redundancy. Principal component loadings were generated to assess how individual predictors contributed to each principal component and to explore which features were clustered.

[0403] For the whole cohort, on correlation matrix analysis, no variables were highly correlated (r > 0.7)

[0042] , A few variables were moderately correlated (0.4 < r < 0.7), such as age at diagnosis and WHO grade 4 tumor (r=0.586), age at diagnosis and Medicare insurance (r=0.545), age at diagnosis and retired employment status (r=0.565), and Medicare insurance and retired employment status (r = 0.545). While these values demonstrate a modest correlation between some variables, the correlations do not violate the assumption of independence for SHAP analysis, as each variable still contributes uniquely to the model and is not interchangeable. Principal component analysis (with one-hot encoding to transform categorical predictors into numerical representations) was performed, finding that 13 principal componentswere required to explain more than 80% of the variance. These 13 principal components drew from all 11 included variables with no dominant principal component, suggesting that a substantial number of independent dimensions remain.

[0404] Similar to the whole cohort, for the women cohort, there were no highly correlated predictors (r > 0.7). Similar variables were moderately correlated, such as age and Medicare insurance (r=0.590), age and retirement (r=0.560), age and WHO grade 4 tumor (r=0.580), Hispanic or Latino ethnicity and other race (r=0.450), retirement and Medicare insurance (r=0.523), active employment and professional occupation ISCO (r=0.420), and WHO grade 4 tumor and no chemotherapy treatment (r=-0.420). PC A analysis showed 13 principal components required to explain greater than 80% of the variance. These 13 principal components drew from all 12 included predictor variables with no dominant principal component, suggesting that a substantial number of independent dimensions remain.

[0405] In contrast, for the minority cohort model, the median household income by zip code and the percentage of households living below the poverty line by zip code were highly correlated (r=-0.711). Therefore, to reduce redundancy, neighborhood poverty levels were removed as a predictor for the minority BNN model. No additional predictors were highly correlated after removing poverty as a predictor (r>0.7). Age at diagnosis and Medicare insurance (r=0.549), age at diagnosis and retirement (r=0.612), and Medicare insurance and retirement (r=0.663) were moderately correlated. PC A analysis was performed, finding that 13 principal components were again required to explain more than 80% of the variance. These 13 principal components drew from all 13 predictor variables with no dominant principal component.

[0406] References

[0407] The numbering related to the following references apply with respect to the experimental results presented in Example 1 and Example 2:

[0408] [1] S. M. Chang et al., Clinical trial participation among patients enrolled in the Glioma Outcomes Project. Cancer 94, 2681-2687 (2002).

[0409] [2] K. Escritt, M. Mann, A. Nelson, E. Harrop, Hope and meaning-making in phase 1 oncology trials: a systematic review and thematic synthesis of qualitative evidence on patientparticipant experiences. Trials 23, 409 (2022).[3] S. J. Reihl et al., A population study of clinical trial accrual for women and minorities in neuro-oncology following the NIH Revitalization Act. Neuro Oncol 24, 1341-1349 (2022).

[0410] [4] V. H. Murthy, H. M. Krumholz, C. P. Gross, Participation in cancer clinical trials: race-, sex-, and age-based disparities. Jama 291, 2720-2726 (2004).

[0411] [5] U. C. P. Law, 103-43. National Institutes of Health Revitalization Act of 1993.

[0412] Washington, DC, June 10, (1993).

[0413] [6] R. A. Morshed et al., The influence of race and socioeconomic status on therapeutic clinical trial screening and enrollment. J Neurooncol 148, 131-139 (2020).

[0414] [7] D. Presti et al., Factors associated with enrolment in clinical trials among women with early-stage breast cancer. ESMO Open 7, 100513 (2022).

[0415] [8] C. P. Williams, N. Senft Everson, N. Shelburne, W. E. Norton, Demographic and Health Behavior Factors Associated With Clinical Trial Invitation and Participation in the United States. JAMA Netw Open 4, e2127792 (2021).

[0416] [9] K. H. Yu, A. L. Beam, I. S. Kohane, Artificial intelligence in healthcare. Nat Biomed Eng 2, 719-731 (2018).

[0417]

[0010] C. J. Haug, J. M. Drazen, Artificial Intelligence and Machine Learning in Clinical Medicine, 2023. N Engl J Med 388, 1201-1208 (2023).

[0418]

[0011] M. Mehari et al., Enhancing neuro-oncology care through equity-driven applications of artificial intelligence. Neuro Oncol 26, 1951-1963 (2024).

[0419]

[0012] I. E. Eckel-Passow etal., Glioma Groups Based on lp / 19q, IDH, and TERT Promoter Mutations in Tumors. N Engl J Med 372, 2499-2508 (2015).

[0420]

[0013] J. K. Wiencke et al., Molecular features of adult glioma associated with patient race / ethnicity, age, and a polymorphism in O6-methylguanine-DNA-methyltransferase. Cancer Epidemiol Biomarkers Prev 14, 1774-1783 (2005).

[0421]

[0014] C. Bieganek, C. Aliferis, S. Ma, Prediction of clinical trial enrollment rates. PLoS One 17, e0263193 (2022).

[0422]

[0015] R. Chow et al., Use of Artificial Intelligence for Cancer Clinical Trial Enrolment: A Systematic Review and Meta-Analysis. J Natl Cancer Inst, (2023).

[0423]

[0016] K. Wu et al., Machine Learning Prediction of Clinical Trial Operational Efficiency. Aaps 24, 57 (2022).

[0017] R. Liu et l., Evaluating eligibility criteria of oncology trials using real-world data and Al. Nature 592, 629-633 (2021).

[0424]

[0018] J. T. Beck et al., Artificial Intelligence Tool for Optimizing Eligibility Screening for Clinical Trials in a Large Community Cancer Center. JCO Clin Cancer Inform 4, 50-59 (2020).

[0425]

[0019] T. Haddad et al., Accuracy of an Artificial Intelligence System for Cancer Clinical Trial Eligibility Screening: Retrospective Pilot Study. JMIR Med Inform 9, e27767 (2021).

[0426]

[0020] Y. Ni et al., Increasing the efficiency of trial -patient matching: automated clinical trial eligibility pre-screening for pediatric oncology patients. BMC Med Inform Decis Mak 15, 28 (2015).

[0427]

[0021] S. M. Meystre, P. M. Heider, Y. Kim, D. B. Aruch, C. D. Britten, Automatic trial eligibility surveillance based on unstructured clinical data. Int J Med Inform 129, 13-19 (2019).

[0428]

[0022] C. Shivade et al., Textual inference for eligibility criteria resolution in clinical trials. J Biomed Inform 58 Suppl, S211-s218 (2015).

[0429]

[0023] Y. Lan, G. Tang, D. F. Heitjan, Statistical modeling and prediction of clinical trial recruitment. StatMed , 945-955 (2019).

[0430]

[0024] V. V. Anisimov, V. V. Fedorov, Modelling, prediction and adaptive adjustment of recruitment in multi centre trials. Stat Med 26, 4958-4975 (2007).

[0431]

[0025] U. S. C. Bureau, "2020 Census Demographic Profile," (U.S. Department of Commerce, https: / / www-.census.gov / data / tables / 2023 / dec / 2020-census-demographic-profile.html, 2021).

[0432]

[0026] Z. D. Bailey et al., Structural racism and health inequities in the USA: evidence and interventions. Lancet 389, 1453-1463 (2017).

[0433]

[0027] J. Feagin, Z. Bennefield, Systemic racism and U.S. health care. Soc SciMed 103, 7-14 (2014).

[0434]

[0028] K. Crenshaw, Mapping the Margins: Intersectionality, Identity Politics, and Violence against Women of Color. Stanford Law Review 43, 1241 (1991).

[0435]

[0029] O. O. Shiyanbola, M. J. Wen, M. A. Maurer, Incorporating health literacy principles into the adaptation of a methods motivational interviewing approach for enrolling black adults in a pilot randomized trial. Sci Rep 15, 4833 (2025).

[0436]

[0030] P. Natale et al., Transparency, trust and minimizing burden to increase recruitment and retention in trials: a systematic review. J Clin Epidemiol 134, 35-51 (2021).

[0031] E. M. Cheng, A. Chen, W. Cunningham, Primary language and receipt of recommended health care among Hispanics in the United States. J Gen Intern Med 22 Suppl 2, 283-288 (2007).

[0437]

[0032] J. Gregg, S. Saha, Communicative competence: a framework for understanding language barriers in health care. J Gen Intern Med 22 Suppl 2, 368-370 (2007).

[0438]

[0033] D. Leslie, A. Mazumder, A. Peppin, M. K. Wolters, A. Hagerty, Does "Al" stand for augmenting inequality in the era of covid- 19 healthcare? Bmj 312, n304 (2021).

[0439]

[0034] A. Brown et al., Detecting shortcut learning for fair medical Al using shortcut testing. Nat Commun 14, 4314 (2023).

[0440]

[0035] R. Iskander, H. Moyer, K. Vigneault, S. M. Mahmud, J. Kimmelman, Survival Benefit Associated With Participation in Clinical Trials of Anticancer Drugs: A Systematic Review and Meta-analysis. Jama, (2024).

[0441]

[0036] S. Afrose, W. Song, C. B. Nemeroff, C. Lu, D. D. Yao, Subpopulation-specific machine learning prognosis for underrepresented patients with double prioritized bias correction.

[0442] Commun Med (Land) 2, 111 (2022).

[0443]

[0037] Y. Gao, Y. Cui, Deep transfer learning for reducing health care disparities arising from biomedical data inequality. Nat Commun 11, 5131 (2020).

[0444]

[0038] V. Hassija et al., Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence. Cogn Comput 16, 45-74 (2024).

[0445]

[0039] C. Molnar, Interpretable Machine Learning.

[0446]

[0040] S. M. Lundberg, S.-I. Lee. (Curran Associates Inc., 2017), pp. 4768-4777.

[0447]

[0041] Revisions to OMB's Statistical Policy Directive No. 15: Standards for Maintaining, Collecting, and Presenting Federal Data on Race and Ethnicity. Federal Register, (2024).

[0448]

[0042] M. M. Mukaka, A guide to appropriate use of Correlation coefficient in medical research. Malawi Med J 24, 69-71 (2012).

[0449]

[0043] S. Y. Chen, Z. Feng, X. Yi, A general introduction to adjustment for multiple comparisons. J Thorac Dis 9, 1725-1729 (2017).

[0450]

[0044] S. Lee, D. K. Lee, What is the proper way to apply the multiple comparison test? Korean J Anesthesio! 1\, 353-360 (2018).In at least some of the previously described embodiments, one or more elements used in an embodiment can interchangeably be used in another embodiment unless such a replacement is not technically feasible. It will be appreciated by those skilled in the art that various other omissions, additions and modifications may be made to the methods and structures described above without departing from the scope of the claimed subject matter. All such modifications and changes are intended to fall within the scope of the subject matter, as defined by the appended claims.

[0451] It will be understood by those within the art that, in general, terms used herein, and especially in the appended claims (e.g, bodies of the appended claims) are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes but is not limited to,” etc.). It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases “at least one” and “one or more” to introduce claim recitations.

[0452] However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to embodiments containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a” and / or “an” should be interpreted to mean “at least one” or “one or more”); the same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the bare recitation of “two recitations,” without other modifiers, means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to “at least one of A, B, and C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “ a system having at least one of A, B, and C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). In those instances where a convention analogous to “at leastone of A, B, or C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “ a system having at least one of A, B, or C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and / or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase “A or B” will be understood to include the possibilities of “A” or “B” or “A and B.”

[0453] In addition, where features or aspects of the disclosure are described in terms of Markush groups, those skilled in the art will recognize that the disclosure is also thereby described in terms of any individual member or subgroup of members of the Markush group.

[0454] As will be understood by one skilled in the art, for any and all purposes, such as in terms of providing a written description, all ranges disclosed herein also encompass any and all possible sub-ranges and combinations of sub-ranges thereof. Any listed range can be easily recognized as sufficiently describing and enabling the same range being broken down into at least equal halves, thirds, quarters, fifths, tenths, etc. As a non-limiting example, each range discussed herein can be readily broken down into a lower third, middle third and upper third, etc. As will also be understood by one skilled in the art all language such as “up to,” “at least,” “greater than,” “less than,” and the like include the number recited and refer to ranges which can be subsequently broken down into sub-ranges as discussed above. Finally, as will be understood by one skilled in the art, a range includes each individual member. Thus, for example, a group having 1-3 articles refers to groups having 1, 2, or 3 articles. Similarly, a group having 1-5 articles refers to groups having 1, 2, 3, 4, or 5 articles, and so forth.

[0455] Although the foregoing invention has been described in some detail by way of illustration and example for purposes of clarity of understanding, it is readily apparent to those of ordinary skill in the art in light of the teachings of this invention that certain changes and modifications may be made thereto without departing from the spirit or scope of the appended claims.

[0456] Accordingly, the preceding merely illustrates the principles of the invention. It will be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the invention and areincluded within its spirit and scope. Furthermore, all examples and conditional language recited herein are principally intended to aid the reader in understanding the principles of the invention and the concepts contributed by the inventors to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the invention as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents and equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims.

[0457] The scope of the present invention, therefore, is not intended to be limited to the exemplary embodiments shown and described herein. Rather, the scope and spirit of present invention is embodied by the appended claims. In the claims, 35 U.S.C. §112(f) or 35 U.S.C. §112(6) is expressly defined as being invoked for a limitation in the claim only when the exact phrase "means for" or the exact phrase "step for" is recited at the beginning of such limitation in the claim; if such exact phrase is not used in a limitation in the claim, then 35 U.S.C. § 112 (f) or 35 U.S.C. §112(6) is not invoked.

Claims

WHAT IS CI, AIMED IS:

1. A method for optimizing participant demographics of a clinical trial, the method comprising:obtaining values for multiple patient-level variables for each of a plurality of individuals associated with the clinical trial, the patient-level variables comprising social identifier variables and prognostic variables, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a primary social identifier variable;determining the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals using a first ensemble machine learning (ML) model, wherein the first ensemble ML model is trained using a dataset comprising individuals from each subpopulation of the plurality of individuals;determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of the second subpopulation defined by the primary social identifier variable using a second ensemble ML model, wherein the second ensemble ML model is trained using a dataset comprising individuals of the second subpopulation; andoptimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first and second ensemble ML models.

2. The method according to Claim 1, wherein the multiple social identifier variables comprise demographic variables and socioeconomic variables.

3. The method according to Claim 1 or 2, wherein the second ensemble ML model is trained using a dataset consisting of individuals of the second subpopulation.

4. The method according to any one of the preceding claims, wherein the ensemble ML models are boosted neural networks (BNNs).

5. The method according to any one of the preceding claims, wherein the primary social identifier variable comprises a biologically based social identifier variable.

6. The method according to Claim 5, wherein the primary social identifier variable comprises age, race, or biological sex.

7. The method according to any one of the preceding claims, wherein the plurality of clinical trial associated individuals have been diagnosed with a health condition addressed by the clinical trial.

8. The method according to Claim 8, wherein the clinical trial is a therapeutic clinical trial.

9. The method according to Claim 7 or 8, wherein the plurality of clinical trial associated individuals have a known status of clinical trial participation, and wherein at least a portion of the individuals have participated in a clinical trial addressing the health condition.

10. The method according to any one of Claims 7-9, wherein the health condition comprises cancer, and wherein the prognostic variables comprise oncologic variables.

11. The method according to Claim 10, wherein the cancer is a glioma.

12. The method according to Claim 11, wherein the oncologic variables comprise one or more of Karnofsky Performance Status (KPS), tumor grade, tumor type, tumor hemisphere and / or lobe, receipt of chemotherapy and / or radiation, extent of resection (EOR), and seizure presentation.

13. The method according to any one of the preceding claims, wherein the multiple social identifier variables comprise a plurality of socioeconomic variables, the socioeconomic variables including one or more of insurance status, employment status, occupation classification, household income, distance from clinical trial facility, and interpreter use.

14. The method according to any one of the preceding claims, wherein the multiple social identifier variables comprise a plurality of demographic variables, the demographic variables including one or more of biological sex, race, ethnicity, age, and preferred language.

15. The method according to any one of the preceding claims, wherein one or more of the patient-level variables are derived from electronic health record (EHR) data obtained for each of the plurality of individuals.

16. The method according to Claim 15, wherein the method further comprises determining one or more patient-level variable values based on proxy indicators of the variables found in the EHR data.

17. The method according to any one of the preceding claims, wherein the primary social identifier variable comprises race, the first subpopulation of individuals defined by the primary social identifier variable comprises individuals of a majority racial group, and the second subpopulation of individuals defined by the primary social identifier variable comprises individuals of one or more racial groups other than the majority racial group.

18. The method according to Claim 17, wherein the second subpopulation of individuals defined by the primary social identifier comprises one or more racial groups underrepresented in clinical trial enrollment.

19. The method according to any one of the preceding claims, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a secondary social identifier variable.

20. The method according to Claim 19, wherein the method further comprises:determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of the first subpopulation defined by the secondary social identifier variable using a third ensemble ML model, wherein the third ensemble ML model is trainedusing a dataset comprising individuals of the first subpopulation defined by the secondary social identifier variable; andoptimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first, second, and third ensemble ML models.

21. The method according to Claim 20, wherein the secondary social identifier variable comprises age, race, or biological sex.

22. The method according to Claim 21, wherein the secondary social identifier variable comprises biological sex and the first subpopulation of individuals defined by the secondary social identifier variable comprises biological females.

23. The method according to any one of the preceding claims, wherein the method further comprises:identifying a plurality of target subpopulations of interest, each defined by a distinct set of social identifier variable values;determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of each target subpopulation using a corresponding ensemble ML model, wherein each ensemble ML model is trained using a dataset comprising individuals of the respective target subpopulation; andoptimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by the ensemble ML models for each target subpopulation.

24. The method according to Claim 23, wherein the plurality of target subpopulations are identified based on disparities between the incidence of the health condition among subpopulations defined by social identifier variable values and the corresponding enrollment patterns observed forthose subpopulations in clinical trials addressing the health condition.

25. The method according to Claim 24, wherein the method further comprises identifying the plurality of target subpopulations from the multiple patient-level variables obtained for each of the plurality of clinical trial associated individuals.

26. The method according to any one of the preceding claims, wherein the patient-level variables comprise a screening status indicating whether an individual has been screened for a clinical trial addressing the health condition.

27. The method according to Claim 26, wherein the patient-level variables comprise the number of clinical trials discussed with an individual.

28. The method according to Claim 26 or 27, wherein, for each ensemble ML model used to determine the relative influence of patient-level variables on clinical trial enrollment, the method further comprises training a corresponding ensemble ML model to determine the relative influence of the patient-level variables on clinical trial screening for the same subpopulation.

29. The method according to Claim 26, wherein the method further comprises:determining the relative influence of the patient-level variables on clinical trial screening for individuals of each subpopulation using each screening ensemble ML model; and optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by each screening and enrollment ensemble ML model.

30. The method according to any one of the preceding claims, wherein the method further comprises preprocessing the patient-level variables.

31. The method according to Claim 30, the patient-level variables are preprocessed by: one-hot encoding each categorical variable into a set of binary variables; and transforming each continuous variable to reduce the influence of outliers or skewed distributions.

32. The method according to Claim 31 , wherein each continuous variable is transformed to approximate a normal distribution using a Johnson Su or a Johnson Sb distribution.

33. The method according to any one of Claims 30 to 32, wherein the preprocessing comprises, for each ensemble ML model, filtering the patient-level variables to remove variables that are highly correlated with other variables in predicting enrollment outcomes.

34. The method according to Claim 33, wherein the highly correlated variables are determined using correlation matrix analysis.

35. The method according to any one of the preceding claims, wherein each ensemble ML model comprises a BNN, the BNN comprising five or more component models trained using boosting techniques.

36. The method according to Claim 35, wherein each BNN comprises ten or more component models.

37. The method according to Claim 35 or 36, wherein the first layer of the hidden layer structure of each BNN comprises a plurality of different types of nodes, wherein the plurality of different types of nodes are configured to capture a plurality of different types of effects of patient-level variables.

38. The method according to Claim 37, wherein the first layer of each component model of each BNN comprises a plurality of different types of nodes.

39. The method according to any one of Claims 35 to 38, wherein the method further comprises training each ensemble ML model.

40. The method according to Claim 39, wherein for each enrollment ensemble ML model the training comprises:sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial enrollment based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; andminimizing a loss function comprising a squared penalty function during training of each component model, the squared penalty function comprising a squared error between predicted and actual enrollment outcomes.

41. The method according to Claim 39, wherein for each screening ensemble ML model the training comprises:sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial screening based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; andminimizing a loss function comprising a squared penalty function during training of each component model, the squared penalty function comprising a squared error between predicted and actual screening outcomes.

42. The method according to Claim 40 or 41, wherein the training is configured to preserve the predictive influence of multiple patient-level variables during training.

43. The method according to any one of Claims 39 to 42, wherein the plurality of individuals associated with the clinical trial comprises individuals whose distribution of social identifier variable values approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial.

44. The method according to any one of Claims 39 to 43, wherein the training comprises performing k-fold cross-validation.

45. The method according to any one of Claims 39 to 44, wherein the training comprises removing patient-level variables having an estimated influence below a predetermined threshold from consideration in the final model.

46. The method according to Claim 45, wherein the removing comprises performing backward elimination.

47. The method according to any one of Claims 39 to 46, wherein the dataset used to train each ensemble ML model consists only of individuals from the subpopulation for which the model is configured to determine the relative influence of patient-level variables on clinical trial screening or enrollment.

48. The method according to any one of the preceding claims, wherein determining the relative influence of each patient-level variable for an ensemble ML model comprises calculating a quantitative influence metric for each patient-level variable.

49. The method according to Claim 45, wherein the quantitative influence metric comprises a Shapley-based value representing the average contribution of each patient-level variable to model predictions.

50. The method according to any one of the preceding claims, wherein the method further comprises using the trained ensemble ML models to determine an individual patient’s probability of clinical trial enrollment based on the patient-level variables of the patient.

51. The method according to any one of the preceding claims, wherein optimizing recruitment of individuals for the clinical trial comprises allocating recruitment resources based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

52. The method according to any one of the preceding claims, wherein optimizing recruitment of individuals for the clinical trial comprises generating a recruitment plan based ona comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

53. The method according to any one of the preceding claims, wherein the method further comprises generating a recruitment report based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

54. The method according to Claim 53, wherein the recruitment report comprises one or more of a summary of differences in the relative influence of patient-level variables between the overall population cohort determined by the first ensemble ML model and each of the one or more subpopulations, a recommended allocation of recruitment resources, and a proposed recruitment plan.

55. The method according to Claim 54, wherein the recruitment report is generated using a natural language processing (NLP) model.

56. The method according to Claim 55, wherein the NLP model is a large language model (LLM).

57. The method according to any one of the preceding claims, wherein the participant demographics of the clinical trial are optimized such that the distribution of social identifier variable values among recruited individuals better approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial.

58. The method according to any one of the preceding claims, wherein the participant demographics of the clinical trial are optimized in order to better represent one or more subpopulations underrepresented in prior clinical trials addressing the health condition.

59. The method according to any one of the preceding claims, wherein the participant demographics of the clinical trial are optimized to comply with one or more regulatory or institutional demographic guidelines for clinical trial enrollment.

60. The method according to any one of the preceding claims, wherein the participant demographics of the clinical trial are optimized to increase biological heterogeneity among trial participants.

61. The method according to any one of the preceding claims, wherein the method is implemented via a computer.

62. The method according to Claim 54, wherein the method is implemented automatically using lines of computer code.

63. A computer system configured to perform the method according to any one of Claims 1 to 56.

64. A system for optimizing participant demographics of a clinical trial, the system comprising:a processor comprising memory operably coupled to the processor, wherein the memory comprises instructions stored thereon, which, when executed by the processor, cause the processor to:obtain values for multiple patient-level variables for each of a plurality of individuals associated with the clinical trial, the patient-level variables comprising social identifier variables and prognostic variables, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a primary social identifier variable;determine the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals using a first ensemble machine learning (ML) model, wherein the first ensemble ML model is trained using a dataset comprising individuals from each subpopulation of the plurality of individuals;determine the relative influence of the patient-level variables on clinical trial enrollment for individuals of the second subpopulation defined by the primary social identifier variable using a second ensemble ML model, wherein the second ensemble ML model is trained using a dataset comprising individuals of the second subpopulation; and optimize recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first and second ensemble ML models.

65. The system according to Claim 64, wherein the multiple social identifier variables comprise demographic variables and socioeconomic variables.

66. The system according to Claim 64 or 65, wherein the second ensemble ML model is trained using a dataset consisting of individuals of the second subpopulation.

67. The system according to any one of Claims 64 to 66, wherein the ensemble ML models are boosted neural networks (BNNs).

68. The system according to any one of Claims 64 to 67, wherein the primary social identifier variable comprises a biologically based social identifier variable.

69. The system according to Claim 68, wherein the primary social identifier variable comprises age, race, or biological sex.

70. The system according to any one of Claims 64 to 69, wherein the plurality of clinical trial associated individuals have been diagnosed with a health condition addressed by the clinical trial.

71. The system according to Claim 70, wherein the clinical trial is a therapeutic clinical trial.

72. The system according to Claim 70 or 71 , wherein the plurality of clinical trial associated individuals have a known status of clinical trial participation, and wherein at least a portion of the individuals have participated in a clinical trial addressing the health condition.

73. The system according to any one of Claims 70-72, wherein the health condition comprises cancer, and wherein the prognostic variables comprise oncologic variables.

74. The system according to Claim 73, wherein the cancer is a glioma.

75. The system according to Claim 74, wherein the oncologic variables comprise one or more of Kamofsky Performance Status (KPS), tumor grade, tumor type, tumor hemisphere and / or lobe, receipt of chemotherapy and / or radiation, extent of resection (EOR), and seizure presentation.

76. The system according to any one of Claims 64 to 75, wherein the multiple social identifier variables comprise a plurality of socioeconomic variables, the socioeconomic variables including one or more of insurance status, employment status, occupation classification, household income, distance from clinical trial facility, and interpreter use.

77. The system according to any one of Claims 64 to 76, wherein the multiple social identifier variables comprise a plurality of demographic variables, the demographic variables including one or more of biological sex, race, ethnicity, age, and preferred language.

78. The system according to any one of Claims 64 to 77, wherein one or more of the patientlevel variables are derived from electronic health record (EHR) data obtained for each of the plurality of individuals.

79. The system according to Claim 78, wherein the instructions, when executed by the processor, further cause the processor to determine one or more patient-level variable values based on proxy indicators of the variables found in the EHR data.

80. The system according to any one of Claims 64 to 79, wherein the primary social identifier variable comprises race, the first subpopulation of individuals defined by the primary social identifier variable comprises individuals of a majority racial group, and the second subpopulation of individuals defined by the primary social identifier variable comprises individuals of one or more racial groups other than the majority racial group.

81. The system according to Claim 80, wherein the second subpopulation of individuals defined by the primary social identifier comprises one or more racial groups underrepresented in clinical trial enrollment.

82. The system according to any one of Claims 64 to 81, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a secondary social identifier variable.

83. The system according to Claim 82, wherein the instructions, when executed by the processor, further cause the processor to:determine the relative influence of the patient-level variables on clinical trial enrollment for individuals of the first subpopulation defined by the secondary social identifier variable using a third ensemble ML model, wherein the third ensemble ML model is trained using a dataset comprising individuals of the first subpopulation defined by the secondary social identifier variable; andoptimize recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first, second, and third ensemble ML models.

84. The system according to Claim 83, wherein the secondary social identifier variable comprises age, race, or biological sex.

85. The system according to Claim 84, wherein the secondary social identifier variable comprises biological sex and the first subpopulation of individuals defined by the secondary social identifier variable comprises biological females.

86. The system according to any one of Claims 64 to 85, wherein the instructions, when executed by the processor, further cause the processor to:identify a plurality of target subpopulations of interest, each defined by a distinct set of social identifier variable values;determine the relative influence of the patient-level variables on clinical trial enrollment for individuals of each target subpopulation using a corresponding ensemble ML model, wherein each ensemble ML model is trained using a dataset comprising individuals of the respective target subpopulation; andoptimize recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by the ensemble ML models for each target subpopulation.

87. The system according to Claim 86, wherein the plurality of target subpopulations are identified based on disparities between the incidence of the health condition among subpopulations defined by social identifier variable values and the corresponding enrollment patterns observed forthose subpopulations in clinical trials addressing the health condition.

88. The system according to Claim 87, wherein the instructions, when executed by the processor, further cause the processor to identify the plurality of target subpopulations from the multiple patient-level variables obtained for each of the plurality of clinical trial associated individuals.

89. The system according to any one of Claims 64 to 88, wherein the patient-level variables comprise a screening status indicating whether an individual has been screened for a clinical trial addressing the health condition.

90. The system according to Claim 89, wherein the patient-level variables comprise the number of clinical trials discussed with an individual.

91. The system according to Claim 89 or 90, wherein, for each ensemble ML model used to determine the relative influence of patient-level variables on clinical trial enrollment, the methodfurther comprises training a corresponding ensemble ML model to determine the relative influence of the patient-level variables on clinical trial screening for the same subpopulation.

92. The system according to Claim 91, wherein the instructions, when executed by the processor, further cause the processor to:determine the relative influence of the patient-level variables on clinical trial screening for individuals of each subpopulation using each screening ensemble ML model; and optimize recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by each screening and enrollment ensemble ML model.

93. The system according to any one of Claims 64 to 92, wherein the instructions, when executed by the processor, further cause the processor to preprocess the patient-level variables.

94. The system according to Claim 93, the patient-level variables are preprocessed by:one-hot encoding each categorical variable into a set of binary variables; and transforming each continuous variable to reduce the influence of outliers or skewed distributions.

95. The system according to Claim 94, wherein each continuous variable is transformed to approximate a normal distribution using a Johnson Su or a Johnson Sb distribution.

96. The system according to any one of Claims 93 to 95, wherein the preprocessing comprises, for each ensemble ML model, filtering the patient-level variables to remove variables that are highly correlated with other variables in predicting enrollment outcomes.

97. The system according to Claim 96, wherein the highly correlated variables are determined using correlation matrix analysis.

98. The system according to any one of Claims 64 to 97, wherein each ensemble ML model comprises a BNN, the BNN comprising five or more component models trained using boosting techniques.

99. The system according to Claim 98, wherein each BNN comprises ten or more component models.

100. The system according to Claim 98 or 99, wherein the first layer of the hidden layer structure of each BNN comprises a plurality of different types of nodes, wherein the plurality of different types of nodes are configured to capture a plurality of different types of effects of patient-level variables.

101. The system according to Claim 100, wherein the first layer of each component model of each BNN comprises a plurality of different types of nodes.

102. The system according to any one of Claims 98 to 100, wherein the instructions, when executed by the processor, further cause the processor to train each ensemble ML model using an initiation component.

103. The system according to Claim 102, wherein for each enrollment ensemble ML model the training comprises:sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial enrollment based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; andminimizing a loss function comprising a squared penalty function during training of each component model, the squared penalty function comprising a squared error between predicted and actual enrollment outcomes.

104. The system according to Claim 102, wherein for each screening ensemble ML model the training comprises:sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial screening based on the patient-level variables,wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; andminimizing a loss function comprising a squared penalty function during training of each component model, the squared penalty function comprising a squared error between predicted and actual screening outcomes.

105. The system according to Claim 103 or 104, wherein the training is configured to preserve the predictive influence of multiple patient-level variables during training.

106. The system according to any one of Claims 102 to 105, wherein the plurality of individuals associated with the clinical trial comprises individuals whose distribution of social identifier variable values approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial.

107. The system according to any one of Claims 102 to 106, wherein the training comprises performing k-fold cross-validation.

108. The system according to any one of Claims 102 to 107, wherein the training comprises removing patient-level variables having an estimated influence below a predetermined threshold from consideration in the final model.

109. The system according to Claim 108, wherein the removing comprises performing backward elimination.

110. The system according to any one of Claims 102 to 109, wherein the dataset used to train each ensemble ML model consists only of individuals from the subpopulation for which the model is configured to determine the relative influence of patient-level variables on clinical trial screening or enrollment.

111. The system according to any one of Claims 64 to 110, wherein determining the relative influence of each patient-level variable for an ensemble ML model comprises calculating a quantitative influence metric for each patient-level variable.

112. The system according to Claim 111, wherein the quantitative influence metric comprises a Shapley-based value representing the average contribution of each patient-level variable to model predictions.

113. The system according to any one of Claims 64 to 112, wherein the instructions, when executed by the processor, further cause the processor to use the trained ensemble ML models to determine an individual patient’s probability of clinical trial enrollment based on the patientlevel variables of the patient.

114. The system according to any one of Claims 64 to 113, wherein optimizing recruitment of individuals for the clinical trial comprises allocating recruitment resources based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

115. The system according to any one of Claims 64 to 114, wherein optimizing recruitment of individuals for the clinical trial comprises generating a recruitment plan based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

116. The system according to any one of Claims 64 to 115, wherein the instructions, when executed by the processor, further cause the processor to generate a recruitment report based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

117. The system according to Claim 116, wherein the recruitment report comprises one or more of a summary of differences in the relative influence of patient-level variables between the overall population cohort determined by the first ensemble ML model and each of the one ormore subpopulations, a recommended allocation of recruitment resources, and a proposed recruitment plan.

118. The system according to Claim 117, wherein the recruitment report is generated using an integration component comprising a natural language processing (NLP) model.

119. The system according to Claim 118, wherein the NLP model is a large language model (LLM).

120. The system according to any one of Claims 64 to 119, wherein the participant demographics of the clinical trial are optimized such that the distribution of social identifier variable values among recruited individuals better approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial.

121. The system according to any one of Claims 64 to 120, wherein the participant demographics of the clinical trial are optimized in order to better represent one or more subpopulations underrepresented in prior clinical trials addressing the health condition.

122. The system according to any one of Claims 64 to 121, wherein the participant demographics of the clinical trial are optimized to comply with one or more regulatory or institutional demographic guidelines for clinical trial enrollment.

123. The system according to any one of Claims 64 to 122, wherein the participant demographics of the clinical trial are optimized to increase biological heterogeneity among trial participants.

124. The system according to any one of Claims 64 to 123, wherein the system is a computer system.

125. A computer program product comprising instructions which, when executed by a computer, cause the computer to perform operations comprising:obtaining values for multiple patient-level variables for each of a plurality of individuals associated with the clinical trial, the patient-level variables comprising social identifier variables and prognostic variables, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a primary social identifier variable;determining the relative influence of the patient-level variables on clinical trial enrollment across the plurality of individuals using a first ensemble machine learning (ML) model, wherein the first ensemble ML model is trained using a dataset comprising individuals from each subpopulation of the plurality of individuals;determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of the second subpopulation defined by the primary social identifier variable using a second ensemble ML model, wherein the second ensemble ML model is trained using a dataset comprising individuals of the second subpopulation; andoptimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first and second ensemble ML models.

126. The computer program product according to Claim 125, wherein the multiple social identifier variables comprise demographic variables and socioeconomic variables.

127. The computer program product according to Claim 125 or 126, wherein the second ensemble ML model is trained using a dataset consisting of individuals of the second subpopulation.

128. The computer program product according to any one of Claims 125 to 127, wherein the ensemble ML models are boosted neural networks (BNNs).

129. The computer program product according to any one of Claims 125 to 128, wherein the primary social identifier variable comprises a biologically based social identifier variable.

130. The computer program product according to Claim 129, wherein the primary social identifier variable comprises age, race, or biological sex.

131. The computer program product according to any one of Claims 125 to 130, wherein the plurality of clinical trial associated individuals have been diagnosed with a health condition addressed by the clinical trial.

132. The computer program product according to Claim 131, wherein the clinical trial is a therapeutic clinical trial.

133. The computer program product according to Claim 131 or 132, wherein the plurality of clinical trial associated individuals have a known status of clinical trial participation, and wherein at least a portion of the individuals have participated in a clinical trial addressing the health condition.

134. The computer program product according to any one of Claims 131-133, wherein the health condition comprises cancer, and wherein the prognostic variables comprise oncologic variables.

135. The computer program product according to Claim 134, wherein the cancer is a glioma.

136. The computer program product according to Claim 135, wherein the oncologic variables comprise one or more of Karnofsky Performance Status (KPS), tumor grade, tumor type, tumor hemisphere and / or lobe, receipt of chemotherapy and / or radiation, extent of resection (EOR), and seizure presentation.

137. The computer program product according to any one of Claims 125 to 136, wherein the multiple social identifier variables comprise a plurality of socioeconomic variables, the socioeconomic variables including one or more of insurance status, employment status, occupation classification, household income, distance from clinical trial facility, and interpreter use.

138. The computer program product according to any one of Claims 125 to 137, wherein the multiple social identifier variables comprise a plurality of demographic variables, the demographic variables including one or more of biological sex, race, ethnicity, age, and preferred language.

139. The computer program product according to any one of Claims 125 to 138, wherein one or more of the patient-level variables are derived from electronic health record (EHR) data obtained for each of the plurality of individuals.

140. The computer program product according to Claim 139, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of determining one or more patient-level variable values based on proxy indicators of the variables found in the EHR data.

141. The computer program product according to any one of Claims 125 to 140, wherein the primary social identifier variable comprises race, the first subpopulation of individuals defined by the primary social identifier variable comprises individuals of a majority racial group, and the second subpopulation of individuals defined by the primary social identifier variable comprises individuals of one or more racial groups other than the majority racial group.

142. The computer program product according to Claim 141, wherein the second subpopulation of individuals defined by the primary social identifier comprises one or more racial groups underrepresented in clinical trial enrollment.

143. The computer program product according to any one of Claims 125 to 142, wherein the plurality of individuals comprises at least a first subpopulation and a second subpopulation of individuals having different values of a secondary social identifier variable.

144. The computer program product according to Claim 143, wherein the instructions, when executed by the processor, further cause the processor to perform the operations ofdetermining the relative influence of the patient-level variables on clinical trial enrollment for individuals of the first subpopulation defined by the secondary social identifier variable using a third ensemble ML model, wherein the third ensemble ML model is trained using a dataset comprising individuals of the first subpopulation defined by the secondary social identifier variable; andoptimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables on clinical trial enrollment determined by the first, second, and third ensemble ML models.

145. The computer program product according to Claim 144, wherein the secondary social identifier variable comprises age, race, or biological sex.

146. The computer program product according to Claim 145, wherein the secondary social identifier variable comprises biological sex and the first subpopulation of individuals defined by the secondary social identifier variable comprises biological females.

147. The computer program product according to any one of Claims 125 to 146, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of:identifying a plurality of target subpopulations of interest, each defined by a distinct set of social identifier variable values;determining the relative influence of the patient-level variables on clinical trial enrollment for individuals of each target subpopulation using a corresponding ensemble ML model, wherein each ensemble ML model is trained using a dataset comprising individuals of the respective target subpopulation; andoptimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by the ensemble ML models for each target subpopulation.

148. The computer program product according to Claim 147, wherein the plurality of target subpopulations are identified based on disparities between the incidence of the health conditionamong subpopulations defined by social identifier variable values and the corresponding enrollment patterns observed for those subpopulations in clinical trials addressing the health condition.

149. The computer program product according to Claim 148, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of identifying the plurality of target subpopulations from the multiple patient-level variables obtained for each of the plurality of clinical trial associated individuals.

150. The computer program product according to any one of Claims 125 to 149, wherein the patient-level variables comprise a screening status indicating whether an individual has been screened for a clinical trial addressing the health condition.

151. The computer program product according to Claim 150, wherein the patient-level variables comprise the number of clinical trials discussed with an individual.

152. The computer program product according to Claim 150 or 151, wherein, for each ensemble ML model used to determine the relative influence of patient-level variables on clinical trial enrollment, the method further comprises training a corresponding ensemble ML model to determine the relative influence of the patient-level variables on clinical trial screening for the same subpopulation.

153. The computer program product according to Claim 152, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of:determining the relative influence of the patient-level variables on clinical trial screening for individuals of each subpopulation using each screening ensemble ML model; and optimizing recruitment of individuals for the clinical trial based on the relative influence of the patient-level variables determined by each screening and enrollment ensemble ML model.

154. The computer program product according to any one of Claims 125 to 153, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of preprocessing the patient-level variables.

155. The computer program product according to Claim 154, the patient-level variables are preprocessed by:one-hot encoding each categorical variable into a set of binary variables; and transforming each continuous variable to reduce the influence of outliers or skewed distributions.

156. The computer program product according to Claim 155, wherein each continuous variable is transformed to approximate a normal distribution using a Johnson Su or a Johnson Sb distribution.

157. The computer program product according to any one of Claims 154 to 156, wherein the preprocessing comprises, for each ensemble ML model, filtering the patient-level variables to remove variables that are highly correlated with other variables in predicting enrollment outcomes.

158. The computer program product according to Claim 157, wherein the highly correlated variables are determined using correlation matrix analysis.

159. The computer program product according to any one of Claims 125 to 158, wherein each ensemble ML model comprises a BNN, the BNN comprising five or more component models trained using boosting techniques.

160. The computer program product according to Claim 159, wherein each BNN comprises ten or more component models.

161. The computer program product according to Claim 159 or 160, wherein the first layer of the hidden layer structure of each BNN comprises a plurality of different types of nodes, whereinthe plurality of different types of nodes are configured to capture a plurality of different types of effects of patient-level variables.

162. The computer program product according to Claim 161, wherein the first layer of each component model of each BNN comprises a plurality of different types of nodes.

163. The computer program product according to any one of Claims 159 to 162, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of training each ensemble ML model.

164. The computer program product according to Claim 163, wherein for each enrollment ensemble ML model the training comprises:sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial enrollment based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; andminimizing a loss function comprising a squared penalty function during training of each component model, the squared penalty function comprising a squared error between predicted and actual enrollment outcomes.

165. The computer program product according to Claim 163, wherein for each screening ensemble ML model the training comprises:sequentially training a plurality of component models, each component model configured as a neural network, to predict clinical trial screening based on the patient-level variables, wherein each successive component model is trained to reduce residual prediction error from the aggregation of one or more preceding component models; andminimizing a loss function comprising a squared penalty function during training of each component model, the squared penalty function comprising a squared error between predicted and actual screening outcomes.

166. The computer program product according to Claim 164 or 165, wherein the training is configured to preserve the predictive influence of multiple patient-level variables during training.

167. The computer program product according to any one of Claims 163 to 166, wherein the plurality of individuals associated with the clinical trial comprises individuals whose distribution of social identifier variable values approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial.

168. The computer program product according to any one of Claims 163 to 167, wherein the training comprises performing k-fold cross-validation.

169. The computer program product according to any one of Claims 163 to 168, wherein the training comprises removing patient-level variables having an estimated influence below a predetermined threshold from consideration in the final model.

170. The computer program product according to Claim 169, wherein the removing comprises performing backward elimination.

171. The computer program product according to any one of Claims 163 to 170, wherein the dataset used to train each ensemble ML model consists only of individuals from the subpopulation for which the model is configured to determine the relative influence of patientlevel variables on clinical trial screening or enrollment.

172. The computer program product according to any one of Claims 125 to 171, wherein determining the relative influence of each patient-level variable for an ensemble ML model comprises calculating a quantitative influence metric for each patient-level variable.

173. The computer program product according to Claim 172, wherein the quantitative influence metric comprises a Shapley -based value representing the average contribution of each patient-level variable to model predictions.

174. The computer program product according to any one of Claims 125 to 173, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of using the trained ensemble ML models to determine an individual patient’s probability of clinical trial enrollment based on the patient-level variables of the patient.

175. The computer program product according to any one of Claims 125 to 174, wherein optimizing recruitment of individuals for the clinical trial comprises allocating recruitment resources based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

176. The computer program product according to any one of Claims 125 to 175, wherein optimizing recruitment of individuals for the clinical trial comprises generating a recruitment plan based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

177. The computer program product according to any one of Claims 125 to 176, wherein the instructions, when executed by the processor, further cause the processor to perform the operations of generating a recruitment report based on a comparison of the relative influence of the patient-level variables on clinical trial enrollment determined by each of the ensemble ML models.

178. The computer program product according to Claim 177, wherein the recruitment report comprises one or more of a summary of differences in the relative influence of patient-level variables between the overall population cohort determined by the first ensemble ML model and each of the one or more subpopulations, a recommended allocation of recruitment resources, and a proposed recruitment plan.

179. The computer program product according to Claim 178, wherein the recruitment report is generated using a natural language processing (NLP) model.Ill180. The computer program product according to Claim 179, wherein the NLP model is a large language model (LLM).

181. The computer program product according to any one of Claims 125 to 180, wherein the participant demographics of the clinical trial are optimized such that the distribution of social identifier variable values among recruited individuals better approximates that of the general patient population diagnosed with the health condition addressed by the clinical trial.

182. The computer program product according to any one of Claims 125 to 181, wherein the participant demographics of the clinical trial are optimized in order to better represent one or more subpopulations underrepresented in prior clinical trials addressing the health condition.

183. The computer program product according to any one of Claims 125 to 182, wherein the participant demographics of the clinical trial are optimized to comply with one or more regulatory or institutional demographic guidelines for clinical trial enrollment.

184. The computer program product according to any one of Claims 125 to 183, wherein the participant demographics of the clinical trial are optimized to increase biological heterogeneity among trial participants.

185. The computer program product according to any one of Claims 125 to 184, wherein the operations are implemented automatically using lines of computer code.

186. The computer program product according to Claim 185, wherein the instructions are embodied on a non-transitory computer-readable medium.