Systems and methods for designing random control studies

Abstract

Methods for designing studies, such as random control tests (RCTs), as well as methods for estimating sample sizes for unstarted or ongoing studies are provided. Also provided are methods of covariable adjustments in study using covariables (e.g., prognostic covariables) obtained by a deep learning model.

Description

Technical Field

[0001] The present invention generally relates to adjustment for covariates and study design procedures. Background Art

[0002] In a study aimed at testing the existence of a treatment effect (b1), the relationship between the outcome (Y), treatment assignment (T), and the prognostic covariates (X) associated with Y can be provided, for example, as:

[0003] Y=b0+b1T+b2X+eps

[0004] where b0, b1, and b2 represent coefficients, and eps (epsilon) represents the error. In this formula, covariate adjustment (i.e., providing b2X) allows for a more precise estimate of the treatment effect (b1).

[0005] Adjusting the prognostic covariate allows to improve the precision and efficiency of analysis, and improves the statistical power of treatment effect evaluation in studies such as randomized clinical trials. Covariate adjustment enables the same statistical power to be achieved with a smaller sample size. The adjustment covariate should be specified in advance and should be selected based on its prognostic value rather than any imbalance criteria. Recognizing the importance of covariate adjustment in research, regulatory agencies including the European Medical Association (EMA) and the U.S. Food and Drug Administration (FDA) have implemented this methodological consensus as a regulatory guide. Using a smaller sample size and / or less stringent eligibility criteria to achieve the same statistical power will enable the reduction of the population size that needs to be screened to participate in the study. Therefore, systems and methods are needed to efficiently adjust covariates, reduce sample size and / or relax eligibility criteria in research and improve the efficiency of hypothesis testing. Summary of the invention

[0006] The present disclosure provides methods for designing (e.g., reducing) sample sizes in trials based on known correlations of adjustment covariates with outcomes, and methods for reestimating and re-adjusting sample sizes while maintaining trial blindness in both continuous outcome settings and time-to-event outcome settings based on data that become available during the trial, e.g., methods for performing blind sample size reestimation. The methods disclosed herein can integrate new prognostic signal sources, e.g., prognostic scores obtained by deep learning, as adjustment covariates for calculating adjusted (e.g., reduced) sample sizes. The present disclosure also provides methods for achieving the statistical power of a target by using covariate adjustments to use less stringent eligibility criteria in trials, which covariate adjustments can compensate for increased heterogeneity with less restrictive inclusion criteria.

[0007] In one aspect, the present disclosure provides a method of designing a randomized controlled trial (RCT) with a time-to-event outcome, the method comprising:

[0008] Selecting covariates for adjustment; and

[0009] The number of events required to achieve statistical power was calculated based on the following formula:

[0010] in

[0011] RCTs are conducted using a calculated number of events.

[0012] Where N original is the raw number of events required to achieve statistical power without covariate adjustment,

[0013] Where N adjusted is the adjusted number of events required to achieve statistical power with covariate adjustment, and

[0014] in It is calculated based on the following formula for RCT external data:

[0015] in

[0016] Cox-Snell R 2 ,

[0017] n is the number of participants,

[0018] l0 is the log-likelihood of the Cox model explaining the time-to-event outcome using only the intercept, and

[0019] l1 is the log-likelihood of the Cox model explaining the time-to-event outcome with the intercept and adjustment for covariates.

[0020] In another aspect, the present disclosure provides a method for estimating sample size at an interim stage of an ongoing randomized controlled trial (RCT), the method comprising:

[0021] Selection of covariates for adjustment;

[0022] Access to blinded RCT data; and

[0023] In the mid-term, use The blinded sample size re-estimation was performed using the following formula:

[0024] in

[0025] The RCT was further conducted using blinded sample size re-estimation,

[0026] Where N adjusted is the re-estimated number of events required to obtain statistical energy efficiency;

[0027] in It is calculated for blinded RCT data at the interim stage based on the following formula: in

[0028] Cox-Snell R 2 ,

[0029] n is the number of participants,

[0030] l0 is the log-likelihood of the Cox model explaining the time-to-event outcome using only the intercept, and

[0031] l1 is the log-likelihood of the Cox model explaining the time-to-event outcome with the intercept and adjustment for covariates.

[0032] In some embodiments, the raw number of events (N) required to achieve statistical power without covariate adjustment is estimated based on the following formula: original ):

[0033] N original =(z β +z 1-α )1 / (P1P2log 2 (hr)), where

[0034] N original is the estimated number of events required to achieve statistical power based on Schoenfeld's formula,

[0035] α is the Type I error level,

[0036] β is the Type II error level,

[0037] P1 and P2 are the proportions of the experimental sample included in the treatment and control groups, respectively (e.g., if the treatment assignments are balanced, both are equal to 1 / 2),

[0038] hr is the stated hazard ratio, and

[0039] z p is the p-quantile of the standard normal distribution.

[0040] In some embodiments, the time-to-event outcome is overall survival, disease-free survival, or time to disease recurrence. In some embodiments, an RCT is performed to estimate the efficacy of treatment in cancer patients. In some embodiments, the cancer is hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.

[0041] In some embodiments, the clinical risk score is adjusted for covariates, the method comprising:

[0042] Obtain clinical attributes derived from subjects;

[0043] Calculate a clinical risk score using a clinical model trained using one or more subject attributes,

[0044] The clinical risk score quantifies the prognosis of the subject.

[0045] In some embodiments, covariates obtained by the deep learning model are covariate adjusted.In some embodiments, the deep learning model is based on a histopathology slide obtained from a cancer subject, and the covariate is a prognostic covariate.

[0046] In some embodiments, the covariates are obtained by computer-generated methods for determining the likelihood of prognosis of a subject with a disease, comprising:

[0047] accessing digital histology images of histology sections obtained from the subject;

[0048] extracting a plurality of feature vectors of the histological image by applying a first convolutional neural network, wherein each feature of the plurality of feature vectors represents a local descriptor of the histological image;

[0049] classifying the histological image using at least the plurality of feature vectors and the classification model, wherein the classification model is trained using a training set of known histological images and known prognostic information; and

[0050] The likelihood of a prognosis for the subject is determined based on the classification of at least the histological image.

[0051] In some embodiments, the covariate is obtained by a computer-implemented method for determining a prognosis of a subject suffering from a disease, the method comprising:

[0052] obtaining digital histology images of histology sections from the subject;

[0053] Dividing a digital image into a collection of tiles;

[0054] Extracting a plurality of feature vectors from the set of tiles or a subset thereof; and

[0055] An artificial intelligence (AI) risk score is calculated based on the histology images using a machine learning model that has been trained to predict prognosis by processing multiple training images.

[0056] The AI ​​risk score quantifies the subject's prognosis.

[0057] In some embodiments, the method further comprises:

[0058] Obtain clinical attributes derived from subjects;

[0059] calculating a clinical risk score using a clinical model trained using one or more subject attributes; and

[0060] The final risk score of the subject is calculated based on the AI ​​risk score and the clinical risk score.

[0061] The final risk score quantifies the subject's prognosis.

[0062] In some embodiments, the digital histology image is a whole slide image (WSI). In some embodiments, the histology section has been stained with a dye. In some embodiments, the dye is hematoxylin and eosin (H&E). In some embodiments, the disease is cancer, e.g., hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.

[0063] In some embodiments, enrollment of subjects based on restrictive eligibility criteria does not improve statistical power relative to enrollment of subjects based on less restrictive eligibility criteria. In some embodiments, less stringent eligibility criteria are used in the trial to achieve the target statistical power. In some embodiments, the method is implemented by a computer.

[0064] In some aspects, the present disclosure provides a machine-readable medium having executable instructions that cause one or more processing units to perform a method for designing a randomized controlled trial (RCT) as provided herein, or a method for estimating the sample size required to achieve statistical power in an interim phase of an ongoing randomized controlled trial (RCT). BRIEF DESCRIPTION OF THE DRAWINGS

[0065] This patent or application file contains at least one drawing printed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0066] The present invention is illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements.

[0067] Figure 1 An exemplary workflow for designing a randomized trial and reestimating and rescaling sample size to optimize trial efficiency and statistical power is illustrated.

[0068] Figure 2 An example workflow for designing sample size based on adjusting for known correlations between covariates and outcomes is illustrated.

[0069] Figure 3 An exemplary workflow for blinded sample size re-estimation in an ongoing trial is illustrated.

[0070] Figure 4 An exemplary workflow for parameter simulation is illustrated.For a set of parameters corresponding to a clinical trial scenario, an instance of the clinical trial is simulated to allow comparison of adjusted and unadjusted analyses.

[0071] Figure 5A-Figure 5C Shows Behavior over the C-index range, outcome incidence (λ), treatment effect (hr), Weibull shape (w), and dropout rate (d). Figure 5A :hr=0.7. Figure 5B :hr=0.4. Figure 5C : hr=0.7, d=0.01, w=0.5.

[0072] Figure 6A-6C shows the proposed R2 measurement with The relationship between. Fig. 6A :w=0.5. Figure 6B :w=1. Figure 6C :w=1.5.

[0073] Figure 7 Efficacy curves resulting from adjustment for clinical variables only (tumor stage and ECOG score) and adjustment for clinical variables and additional deep learning covariates HCCnet are shown. Covariates were sampled from TCGA-HCC.

[0074] Figure 8A-8C Shown are the efficacy curves resulting from adjustment for histological subtype only (current trial) and adjustment for histological subtype and the additional deep learning covariate MesoNet (trial using MesoNet), for the PROMISE-meso trial ( Fig. 8A ), BEAT-meso test ( Figure 8B )、CheckMate743 test( Figure 8C ). All of these simulations were run using the MesoNet training dataset from Mesobank.

[0075] Fig. 9 A and Fig. 9 B shows the interaction between eligibility criteria and adjusted choices. Fig. 9 A shows the results of a parametric simulation where patients with the lowest risk were selected at 50% inclusion. Fig. 9 B shows the results of a semisynthetic simulation based on the HCC-TCGA dataset. The three inclusion levels were based on the eligibility criteria of past and ongoing trials.

[0076] Fig.10 An example of a computer system that may be used in conjunction with the embodiments described herein is illustrated. DETAILED DESCRIPTION

[0077] Systems and methods for covariate analysis are described. In some aspects, the present invention provides a method for using Cox-SnellR 2 In some embodiments, the prognostic score for a disease (e.g., cancer) obtained using deep learning on histology slides can be used as a predictor to be used using Cox-Snell R 2 Covariates for adjustment.

[0078] In the following description, many specific details are set forth to provide a thorough explanation of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention can be practiced without these specific details. In other instances, well-known components, structures, and techniques are not shown in detail to avoid obscuring an understanding of this description.

[0079] References in the specification to "one embodiment" or "some embodiments" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The phrase "in one embodiment" that appears in multiple places in this specification does not necessarily refer to the same embodiment. The term "exemplary" is used herein in the sense of "example" rather than "ideal". In light of this disclosure, it should be understood that the present invention is not limited to the examples described herein.

[0080] For any method described herein, the order of the steps presented, whether in the text or in the attached flow chart, should not be considered to mean that those steps must be performed in the order presented, unless the context otherwise specifies or requires. Instead, the order of the steps presents an embodiment of the method provided, and generally such steps can be performed in a different order or simultaneously instead. The process shown in each of the following figures can be performed by processing logic, which includes hardware (e.g., circuit systems, special logic, etc.), software (such as, running on a general-purpose computer system or a special-purpose machine), or a combination of the two. Although these processes are described below with respect to some sequential operations, it should be appreciated that some of the operations described can be performed in different orders. In addition, some operations can be performed in parallel rather than sequentially.

[0081] Computational methods for implementing the methods provided herein can include, for example, machine learning, artificial intelligence (AI), deep learning (DL), neural networks, classification and / or clustering algorithms, and regression algorithms.

[0082] The terms "server," "client," and "device" are intended to refer generally to a data processing system and not specifically to a particular form factor of a server, client, and / or device.

[0083] The articles "a" and "an" are used herein to refer to one or to more than one (ie, to at least one) of the grammatical object of the article. As an example, "an element" refers to one element or more than one element, for example, a plurality of elements.

[0084] As used herein, the term “comprising” means, and is used interchangeably with, the phrase “including, but not limited to.” The term “comprising” does not necessarily imply that additional elements other than the listed elements must be present.

[0085] When referring to a number or a numerical range, the term "about" or "approximately" means that the number or numerical range mentioned is an approximation within the experimental variability range (or within the statistical experimental error range), and therefore, the number or numerical range can vary from, for example, between 1% and 20% of the stated number or numerical range. In some aspects, "about" means a value within 20% of the stated value. In a more preferred aspect, "about" means a value within 10% of the stated value. In an even more preferred aspect, "about" means a value within 1% of the stated value.

[0086] Unless otherwise stated, all numerals expressing the quantity, properties (such as molecular weight, reaction conditions, etc.) of the components used in this specification and claims should be understood to be modified by the term "about" in all instances. Therefore, unless otherwise stated, the numerical properties set forth in the following specification and claims are approximate, which may be different depending on the desired properties sought to be obtained in various aspects of the present invention. Although the numerical ranges and parameters setting forth the wide range of the present invention are approximate, the numerical values ​​set forth in the specific examples are reported as accurately as possible. However, any numerical value inherently contains certain errors caused by the errors that must occur in its corresponding measurement.

[0087] The term "at least" preceding a number or a range of numbers should be understood to include the number adjacent to the term "at least", as well as all logically included subsequent numbers or integers, as the context makes clear. When "at least" precedes a series of numbers or a range, it should be understood that "at least" can modify each number in the series or range.

[0088] As used herein, "not more than" or "less than" should be understood to mean the value adjacent to the phrase and the logical lower value or integer, according to the logic of the context, to zero (if negative values ​​are not possible). When "not more than" appears before a series of numbers or a range, it should be understood that "not more than" can modify each number in the series or range.

[0089] As used herein, the "up to" in "up to 10" should be understood as up to and including 10, i.e., in the context of non-negative integers, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 or 10.

[0090] When providing a range of values, it should be understood that each intermediate value between the upper and lower limits of the range (e.g., to one-tenth of the unit of the lower limit unless the context clearly dictates otherwise) and any other declared values ​​or intermediate values ​​within the stated range are encompassed within the present invention. The upper and lower limits of these smaller ranges may be independently included in the smaller ranges and are also encompassed within the present invention, but are subject to any specific exclusions within the stated range. When the stated range includes one or both of the limits, the range excluding either or both of the included limits is also encompassed within the present invention.

[0091] "Patient" refers to a subject who exhibits symptoms and / or complications of a disease or condition (e.g., cancer), is being treated by a clinician (e.g., an oncologist), has been diagnosed with a disease or condition, and / or is at risk of developing a disease or condition. The term "patient" includes human and veterinary subjects. Any reference to a subject in this disclosure should be understood to include the assumption that the subject is a "patient," unless the context clearly dictates otherwise.

[0092] As used herein, "predict" or "predict" refers to determining the likelihood of a disease, symptom, or event (e.g., death of a cancer subject) occurring in the future. In some embodiments, a model (e.g., a DL model) can predict the likelihood of survival by one or more of the following measures of test accuracy:

[0093] An odds ratio greater than 1, preferably about 2 or more or about 0.5 or less, about 3 or more or about 0.33 or less, about 4 or more or about 0.25 or less, about 5 or more or about 0.2 or less, or about 10 or more or about 0.1 or less;

[0094] A specificity greater than 0.5, preferably at least about 0.6, at least about 0.7, at least about 0.8, at least about 0.9 or at least about 0.95, wherein the corresponding sensitivity is greater than 0.2, preferably at least about 0.3, at least about 0.4, at least about 0.5, at least about 0.6, at least about 0.7, at least about 0.8, at least about 0.9 or at least about 0.95;

[0095] A sensitivity of at least 0.5, preferably at least about 0.6, at least about 0.7, at least about 0.8, at least about 0.9 or at least about 0.95, wherein the corresponding sensitivity is at least 0.2, preferably at least about 0.3, at least about 0.4, at least about 0.5, at least about 0.6, at least about 0.7, at least about 0.8, at least about 0.9 or at least about 0.95;

[0096] A sensitivity of at least about 75%, combined with a specificity of at least about 75%;

[0097] The positive likelihood ratio [calculated as sensitivity / (1-specificity)] is greater than 1, preferably at least about 2, at least about 3, at least about 4, at least about 5, at least about 10; or

[0098] The negative likelihood ratio [calculated as (1-sensitivity) / specificity] is less than 1, preferably about 0.5 or less, about 0.33 or less, about 0.25 or less or about 0.1 or less.

[0099] As used herein, a "risk score" refers to the likelihood that a certain event (e.g., disease, recurrence) will occur in the future. In some embodiments, the risk score represents the likelihood that a patient will experience a recurrence after treatment. In some embodiments, the risk score is expressed as a classification. In other embodiments, the risk score is expressed as a continuous range. In one embodiment, the risk score represents overall survival, i.e., the duration of the period from cancer diagnosis to death of the subject.

[0100] As used herein, a "subject" is an animal that benefits from methods according to the present disclosure, such as a mammal, including primates (such as humans, monkeys, and chimpanzees) or non-primates (such as cattle, pigs, and horses). In some aspects of the invention, the subject is a human, such as a human diagnosed with cancer. The subject can be a female human. The subject can be a male human. In some aspects, the subject is an adult subject.

[0101] I. Summary of the Invention

[0102] Adjusting covariates (e.g., prognostic covariates) allows improving the precision of treatment effect estimation in studies (e.g., randomized controlled trials (RCTs)) and improving their statistical power. However, calculating sample size under covariate adjustment usually requires knowledge of the magnitude of the adjusted covariate, which is usually not available at the beginning of the trial, especially in the case of new endpoints and / or indications. Therefore, it is usually started with a preliminary calculation of the sample size based on the sparse information available in the planning stage, and re-estimating the value of the adjusted covariate (and the sample size accompanying it) when a portion of the planned patient population has completed the study (Friede & Kieser 2011 Pharmaceut. Statist. 10: 8-13). In this regard, it is important that the sample size re-estimation is performed so that all personnel involved in the study remain blind to the treatment group allocation, and the procedure does not increase the type I error rate.

[0103] The reduction in sample size that results from reducing the number of events to achieve a target statistical power can be estimated based on adjusting for known associations between covariates and the outcome (when the outcome is continuous). In contrast, for time-to-event outcomes (e.g., survival data), there is no unique definition for the proportion of variation explained by covariates. There is no clear way to take into account the association between covariates and the outcome when calculating sample size before starting a trial (e.g., pretrial sample size design) or during a trial when a portion of the data becomes available (e.g., blinded sample size reestimation).

[0104] Thus, the present disclosure provides methods for designing (e.g., reducing) sample sizes in trials based on known correlations of adjustment covariates and outcomes in both continuous and time-to-event settings. The present disclosure also provides methods for re-estimating and re-adjusting sample sizes while maintaining trial blindness based on data that becomes available during the trial (including adjustment covariates and outcome data) in both continuous and time-to-event settings, e.g., methods for performing blinded sample size re-estimation. These methods are, for example, Figure 1-Figure 3 An overview is given in .

[0105] Specifically, parametric simulations in this disclosure indicate that the predictive performance (C-index) and outcome incidence in a trial are the two main determinants of sample size reduction. Based on an analysis of several proposed generalizations of the proportion of variability explained, this disclosure identifies the Cox-Snell best fits the sample size reduction observed in the covariate-adjusted setting with time-to-event outcomes. Methods for designing studies by calculating sample size in covariate-adjusted analyses ( Figure 2 The present disclosure also provides methods for using Cox-Snell in covariate-adjusted analysis of time to event outcomes. Methods for reestimating sample size in a blinded manner ( Figure 3 ).

[0106] The methods disclosed herein can integrate new sources of prognostic signals, such as prognostic scores obtained through deep learning, as adjusted covariates for calculating adjusted (e.g., reduced) sample sizes. Using publicly available clinical trial information on hepatocellular carcinoma and mesothelioma, the present disclosure provides validation that this covariate adjustment method using deep learning prognostic covariates can improve the statistical power achieved based on the same sample size, and can reduce the sample size required to achieve the same statistical power.

[0107] Also provided herein are methods for achieving a target statistical power by using covariate adjustments to use less stringent eligibility criteria in a trial, which can compensate for the increased heterogeneity that occurs with less restrictive inclusion criteria. Therefore, systematic adjustments to prognostic covariates according to the present disclosure can reduce the sample size required to achieve the target power, as well as enable efficient and inclusive clinical trials (e.g., relaxing eligibility criteria), particularly in disease settings with a high incidence of the outcome, e.g., in metastatic cancer.

[0108] In one embodiment, reducing the sample size for randomized clinical trials (or re-estimating the sample size in a blinded manner) improves the functionality of the device used to compile the results of randomized clinical trials. In this embodiment, reducing the randomized clinical trial sample size means that a smaller number of participants are needed to achieve the same statistical power in the randomized clinical trial. In addition, due to the smaller number of participants, the randomized clinical trial compiles a smaller amount of data. This improves the efficiency of the device that compiles, analyzes, stores and / or otherwise processes these data, because the randomized clinical trial with a smaller sample size produces less data. The efficiency of the device is improved by reducing the computational and storage load on the device.

[0109] II. Parametric Simulations with Covariate Adjustment and Sample Size Reduction

[0110] The Fleiss formula below shows the number of events (denoted as N) required for a given statistical power in an unadjusted analysis relative to the correlation between the outcome and the adjusted covariates (denoted as r) in a continuous outcome setting. original ) and the number of events required for a given statistical power of the adjusted analysis (denoted as N adjusted ):

[0111] N adjusted =N original (1-R 2 ).

[0112] Perform parametric simulations to calculate the observed sample size reduction The sample size reduction is a function of the C-index and the outcome incidence rate for a single adjusted covariate in both the continuous outcome and time-to-event outcome settings. As used herein, the "C-index" is a measure of the predictive power of a model. The C-index is controlled via the covariate coefficients. As used herein, the "outcome incidence rate" refers to In the case of survival data, the outcome rate can be estimated using Kaplan-Meier curves. Other parameters of interest are the size of the treatment effect, the Weibull shape of the baseline hazard function, and the withdrawal rate. Simulations are performed under the proportional hazards assumption, as in the Cox model.

[0113] Survival times are generated from a Weibull distribution with shape w and scale depending on the treatment hazard ratio hr and on a standard Gaussian covariate x. Censored times are drawn from an exponential distribution with a specified dropout rate d. Denoting z as the treatment assignment variable, for patient i, the generative model can be theoretically summarized as follows:

[0114]

[0115] All patients who remained at risk at 5 years were censored at that time. Treatment assignment was independent of covariates, and the groups of the trial were balanced. For each set of input parameters, auxiliary parameters and numerically optimized κ and θ were numerically optimized to achieve the target outcome incidence λ in the placebo group at 5 years, and the C-index C was estimated for the entire trial population.

[0116] Figure 4 Pictured When survival times are simulated, the statistical power of the unadjusted model and the model adjusted for the X covariate is estimated on the sample size grid as the percentage of treatment effects detected on 10e+3 resamples (Heo et al., 1998 Mech Ageing Dev 102:45-53). The resulting power curve gives the power curve for the model derived from it. The number of events N required for both models to achieve 80% power adjusted and N original These simulations explore a wide range of parameter values ​​as shown in Table 1, allowing for an extensive study of the effect of λ as a function of C in the setting of different proportional hazards. Behavior.

[0117] Table 1. Description of simulation parameters

[0118]

[0119]

[0120] Parametric simulations showed that the sample size reduction obtained through covariate adjustment increased with both the model's C-index and the outcome incidence rate. Figure 5A-Figure 5C The results are shown as a function of the c-index, outcome incidence, treatment effect, Weibull shape and dropout rate (A, C: hr = 0.7, B: hr = 0.4). The impact of the outcome rate increases with the C-index. Figure 5A Same graph as in the top left panel (hr=0.7, w=0.5, d=0.01) Figure 5C When comparing the outcome rates for λ=10% and λ=90%, there is a 5 percentage point difference in sample size reduction for a C-index of 0.55, and a 60 percentage point difference in sample size reduction for a C-index of 0.85. Figure 5A-Figure 5B The relationship between sample size reduction and C-index and outcome incidence was shown to be independent of the size of the treatment effect, the Weibull shape parameter, or the dropout rate. The values ​​chosen for the dropout rate, d = 0.01 or d = 0.1, resulted in a median of 3.7% (range 0.7%-4.8%) and 28.1% (range 5.4%-38.4%) of patients being censored before the end of follow-up, respectively.

[0121] The parametric simulations provided herein show that the main determinants of the sample size reduction required for a given efficacy are the C-index for adjusting covariates and the outcome incidence in the trial. Other parameters such as Weibull shape, withdrawal rate, or effect size do not affect the precision gain obtained by covariate adjustment, which is consistent with previous reports (Hernández et al., 2006 Ann Epidemiol 16: 41-8). The present disclosure uniquely identifies the strong dependence of sample size reduction on outcome incidence in a simulation setting with a limited time frame (i.e., follow-up stopped at 5 years).

[0122] Identifying key parameters (e.g., outcome incidence, C-index) that determine sample size reduction through covariate adjustment helps prioritize indications that should be given more attention to covariate adjustment. For example, in diseases with high incidence of prognostic outcomes, e.g., metastatic cancer, invasive cancer, e.g., mesothelioma (Kerr et al., 2017 Clin Trials 14:629-38), covariate adjustment will have an impact, with each additional point of the C-index turning into a significant gain in precision. In diseases with low incidence of prognostic outcomes, e.g., secondary cardiovascular prevention, prognostic signals can be used to perform prognostic enrichment (a supplement or alternative to covariate adjustment).

[0123] III. Designing Sample Size Based on Time-to-Event Data Adjusting for Covariates

[0124] Covariate adjustment improves statistical power and leads to more precise estimates of treatment effects. Figure 1 An exemplary flow chart for designing a sample size for a trial (e.g., a randomized controlled trial (RCT)) according to the methods of the present disclosure is shown. Process 100 begins by selecting an adjustment covariate at box 101. As used herein, an "adjustment covariate" can be any variable other than the treatment assignment tested in the trial that is associated with an outcome. An adjustment covariate can be associated with a continuous outcome. Alternatively, an adjustment covariate can be associated with a non-continuous (e.g., time-to-event) outcome. Sample size calculations under covariate adjustment can be performed based on the magnitude of the adjustment covariate relative to the outcome. Therefore, at box 103, process 100 investigates whether the magnitude of the correlation between the adjustment covariate and the outcome is known. If so, process 100 proceeds to box 105, where it designs the sample size based on the known correlation. Figure 2 Block 105 is further elaborated. Based on the sample size designed at block 105, process 100 proceeds to start experimentation at block 109.

[0125] Figure 2 An exemplary flow chart for designing a sample size for a trial based on a known correlation between an adjustment covariate and an outcome is shown. Block 105 is shown as process 200. At 201, process 200 begins by investigating whether the outcome associated with the adjustment covariate is continuous or time-to-event. For continuous outcomes, there is a formula that converts the number of events (denoted as N) required for a given statistical power of the unadjusted analysis from the correlation between the outcome and the adjustment covariate (denoted as R) to original ) is related to the number of events required for a given statistical power of the adjusted analysis (denoted as N adjusted ) to contact:

[0126] N adjusted =N original (1-R 2).

[0127] For example, a correlation of 0.5 between a baseline covariate and an outcome means that the sample size required for an adjusted analysis is reduced by 25% compared to an unadjusted analysis. This formula may be referred to as the Fleiss formula. Process 200 continues at 203 to obtain R 2 , and at 205 the number of events N is designed based on the above Fleiss formula adjusted .

[0128] In contrast, if the outcome associated with adjusting for the covariate at 201 is a time-to-event outcome, such as survival data, then the proportion of the variation explained by the covariate is not uniquely defined. As provided herein in the parametric simulation, the observed reduction in sample size The effect of λ increases with the increase of the predictive power of the prognostic variable (C-index) and / or the increase of the outcome incidence (λ), where for medium C-index values ​​(e.g., between 0.7 and 0.8), the effect of λ is greater than that of C-index values ​​in other ranges. However, there is no clear way to account for the association between covariates and outcomes when calculating sample size in a time-to-event outcome setting. The present disclosure provides a method for using the Cox-Snell A new method for designing a study by calculating sample size in a covariate-adjusted analysis. That is, if the outcome associated with adjusting for covariates at 201 is time to event, then process 200 continues to calculate at 207 and to the use of the Fleiss formula at 209 Instead of R 2 To design the number of events N adjusted .

[0129] To obtain the method described herein, for example, Figure 2 The method illustrated in the example achieves the generalization of R 2 , and investigated whether there is any means to generalize Fleiss's formula for the time-to-event setting using parametric simulations with a finite time horizon.

[0130] In the classic R 2 Among several classes of proposed measures generalized to time-to-event data, explained variation (EV) and explained randomness (ER) measures are tested. The explained variation (EV) measure is an extension of the classical measure of the proportion of variance explained by a set of covariates used in linear regression. On the other hand, the explained randomness (ER) measure builds on the concept of entropy and compares the amount of information contained in the model with and without the covariate of interest. In this disclosure, three EV measures [ (Royston&Sauerbrei 2004Stat Med 23:723-48), (Kent & O'Quigley 1988 Biometrika 75:525-534) and (Royston2006Stata J Promot Commun Stat Stata 6:83-96)] and five ER measures [ (Royston&Sauerbrei 2004Stat Med 23:723-48), (O'Quigley et al. 2005 Stat Med 24:479-89), (Royston&Sauerbrei 2004Stat Med 23:723-48), (Ronghui&O'Quigley 1999JNonparametric Stat 12:83-107) and (Cox et al., 1989, Analysis of binary data, 2nd ed., 1. CRC Press reprint, Boca Raton, Fla., Chapman & Hall)]. As used herein, the term "proposed R 2 The potential R 2 Measurements, including and and For example, the log-likelihood of the model with l0 and l1 representing the base model and the model adjusting for additional covariates is provided as:

[0131]

[0132] Figure 6A-6C Collected all R 2 Measure the values ​​and compare them to the Surprisingly, only two of the eight measurements ( and ) actually show values ​​increasing with outcome incidence. The others only increase with C-index value and remain generally constant with respect to outcome incidence, forming a staircase. This is because most measures were developed with censoring robustness in mind, which is directly related to outcome incidence in a finite time horizon setting. Cox & Snell Better than other proposed R 2measured, and captures the observed sample size reduction best across all our simulations: The median absolute error is 2.1% (0.8% and 3.9% for the first and third quartiles, respectively).

[0133] Therefore, we provide a measure that generalizes Fleiss's formula to non-continuous data (e.g., time-to-event) settings: Cox-Snell Having such a metric is of practical importance because it can help design clinical trials without the aid of simulations. This paper provides a method for designing a randomized controlled trial (RCT) that includes calculating the number of events required to obtain statistical power based on the following formula:

[0134]

[0135] Where N original is the number of events required to achieve statistical power without covariate adjustment, and N adjusted is the number of events required to achieve statistical power with covariate adjustment. In some embodiments, the methods of the present disclosure enable the design of clinical trials without the use of simulations. Figure 2 , shown at 201, 207, and 209. In some embodiments, RCT is performed based on the calculated number of events.

[0136] IV. Blinded Sample Size Re-estimation with Covariate Adjustment Based on Time-to-Event Data

[0137] The metrics and methods disclosed herein can also be used in situations where blinded sample size re-estimation is performed during a trial when there is uncertainty about the predictive performance of adjusting for covariates at the start of the trial. Calculating sample size under covariate adjustment generally requires knowledge of the magnitude of the correlation between the adjusted covariate and the clinical outcome. However, when planning the sample size for a trial, it may be difficult to obtain a good estimate of such a correlation from prior studies, especially in the case of new endpoints and / or indications. Figure 1 An exemplary workflow for blinded sample size re-estimation in a trial (e.g., an RCT) is shown in FIG. 100 . At box 103, process 100 investigates whether a measure of the correlation between the adjusted covariate and the outcome is known. If not, process 100 proceeds to box 107 where it sets a preliminary sample size based on known information without correlation information. In some embodiments, a provisional sample size is calculated based on an initial guess at the covariate value. Based on the sample size designed based on the preliminary sample size setting without correlation information at box 107, process 100 continues to box 111 and starts the trial.

[0138] As the trial progresses, data is obtained from the subjects in a blinded fashion, ie, the investigators do not know whether the subjects belong to the treatment group or the control group.Process 100 continues to block 113 where it obtains observations from the trial in a blinded fashion.

[0139] When observations are available for a predetermined portion of the sample size, the correlation between the adjusted covariates and the outcome is re-estimated, and if necessary, the sample size is adjusted accordingly. From a regulatory perspective, it is important that the sample size recalculation is performed so that all personnel involved in the study remain blinded to the treatment group assignment and so that the procedure does not increase the type I error rate (Friede & Kieser 2011 Pharmaceut. Statist. 10: 8-13). At 115, process 100 performs a blinded sample size re-estimation based on the observations obtained in the ongoing trial (including data on the adjusted covariates and the outcome). Figure 3 Block 115 is described in further detail in FIG.

[0140] Figure 3 An exemplary flow chart for blinded sample size re-estimation, such as during a trial, is shown. Block 113 is shown as process 300. At 301, process 300 begins by investigating whether the outcome associated with the adjusted covariate is continuous or time-to-event. In a continuous outcome setting, process 300 continues to block 303, where the correlation (denoted as R) between the adjusted covariate and the outcome is obtained based on data obtained during the trial. 2 ). Process 300 continues to block 305 where it uses R 2 Re-estimate sample size. The sample size can be re-estimated according to certain procedures reported, for example, by Friede & Kieser 2011 Pharmaceut. Statist. 10:8-13 and Zimmermann et al. 2020 Univ. Kentucky, Statistics Faculty Publications 28 for blinded sample size re-estimation, each of which is hereby incorporated by reference in its entirety.

[0141] For example, the re-estimated number of events can be obtained by the following formula:

[0142] N adjusted =N original (1-R 2 );

[0143] Where N original is the raw number of events required to achieve statistical power without covariate adjustment, N adjustedN is the re-estimated number of events needed to achieve statistical power, and R is the correlation between the adjusted covariates and the outcome based on the blinded RCT data. Further RCTs can be performed using blinded sample size re-estimation. In some embodiments, N original It is estimated based on the following formula:

[0144] N original =(z β +z 1-α ) 2 / (P1P2log 2 (hr)), where

[0145] N original The number of events required to achieve statistical power was estimated based on the Schoenfeld formula (Schoenfeld, 1983 Biometrics, 39:2:499-503),

[0146] α is the Type I error level,

[0147] β is the Type II error level,

[0148] P1 and P2 are the proportions of the trial sample included in the treatment and control groups, respectively (e.g., if the treatment assignments are balanced, both are equal to 1 / 2),

[0149] hr is the stated hazard ratio, and

[0150] z p is the p-quantile of the standard normal distribution.

[0151] Additionally or alternatively, the adjusted number of events (denoted as N A ) can be obtained by the basic approximation formula:

[0152]

[0153] in

[0154] N A and N GS Each is the re-estimated number of events required to achieve statistical power,

[0155] α is the Type I error level,

[0156] β is the Type II error level,

[0157] is the allocation ratio, where n1 and n2 are the sample sizes of group 1 and group 2, respectively,

[0158] is the variance of the result,

[0159] Δ is the specified deviation from the adjusted mean, and

[0160] z p is the p-quantile of the standard normal distribution.

[0161] Alternatively or additionally, the adjusted number of events (denoted as N GS 、N DF or N GS,DF ) can be obtained based on the basic approximation formula with Guenther-Schouten type adjustment (GS):

[0162] or

[0163] Based on the basic approximation formula with adjustment for degrees of freedom (DF) we obtain:

[0164] or

[0165] Based on the basic approximation formula with combined Guenther-Schouten and degrees of freedom adjustment (GS+DF) we obtain:

[0166]

[0167] In contrast, if the outcome associated with adjusting for covariates is a time-to-event outcome, e.g., survival data, then there is no unique definition for the proportion of variance explained by the covariates. When the magnitude of the association between the prognostic covariate and the outcome is uncertain, there is no clear way to perform blinded sample size re-estimation in the time-to-event outcome setting. A new method for performing blinded sample size re-estimation. That is, if the outcome associated with the adjusted covariate at block 301 is non-continuous, then process 300 continues at block 307 to calculate It can be calculated in interim data based on the following formula:

[0168] in

[0169] Cox-Snell R 2 ,

[0170] n is the number of participants,

[0171] l0 is the log-likelihood of the model without covariate adjustment, and

[0172] l1 is the log-likelihood of the model with covariate adjustment.

[0173] Process 300 continues to block 309 where it uses A blinded sample size re-estimation was performed. The number of events can be estimated based on the following formula:

[0174]

[0175] Where N adjusted is the re-estimated number of events needed to achieve statistical power. Further RCTs can be conducted using blinded sample size re-estimation.

[0176] In some embodiments, N is estimated based on the following formula original :

[0177] N original =(z β +z 1-α ) 2 / (P1P2log 2 (hr)), where

[0178] N original is the number of events required to achieve statistical power based on the Schoenfeld formula,

[0179] α is the Type I error level,

[0180] β is the Type II error level,

[0181] P1 and P2 are the proportions of the trial sample included in the treatment and control groups, respectively (e.g., if the treatment assignments are balanced, both are equal to 1 / 2),

[0182] hr is the stated hazard ratio, and

[0183] z p is the p-quantile of the standard normal distribution.

[0184] At box 117, process 100 adjusts the sample size based on the results of the blinded sample size re-estimation at box 115. Process 100 then continues the experiment at box 119 based on the adjusted sample size at box 117.

[0185] V. Covariate Adjustment Based on Deep Learning Covariates

[0186] Provided herein is a method for increasing the statistical power of a study (e.g., a clinical trial) and / or reducing the number of events required for a target efficacy by integrating new sources of prognostic signals obtained using a deep learning prognostic model as an adjustment covariate. The deep learning prognostic model can be based on histopathology slides of cancer patients. In some embodiments, covariate adjustment is performed based on deep learning covariates. Histological slides have been used to obtain covariates for adjustment, but the covariates are obtained by anatomical pathologists for determining histological subtypes (e.g., MAPS tests). With deep learning technology, in addition to more traditional clinical estimates of subtypes, digitized slides are automatically processed by a deep learning model.

[0187] Prognosis prediction based on deep learning model

[0188] This article provides a method for predicting patient outcomes based on deep learning of histological slides in the context of covariate adjustment. Predicting survival based on deep learning of histological slides can be performed using, for example, the deep learning models described in Saillard et al. 2020 Hepatology 72:6; 2000-2013 (HCCnet) and Courtiol et al. 2019 Nat Medicine 25:10; 1519-1525 (MesoNet), each of which is incorporated herein by reference in its entirety.

[0189] Histology is a field of study related to the microscopic features of biological specimens. Histopathology refers to the microscopic examination of specimens (e.g., tissues) obtained from subjects (e.g., patients) or otherwise derived in order to assess disease states. Histopathological specimens are generally produced by processing specimens (e.g., tissues) in a manner that fixes the specimen or a portion thereof to a microscope slide. For example, thin sections of tissue specimens can be obtained using a microtome or other suitable equipment, and the thin sections can be fixed to a slide. To aid in the visualization of the specimen, the specimen can be optionally further processed, for example, by applying a stain. Many stains for visualizing cells and tissues have been developed. These include, but are not limited to, hematoxylin and eosin (H&E), methylene blue, Masson trichomes, Congo red, Oil Red O, and safranin. Pathologists typically use H&E to help visualize cells within tissue specimens. Hematoxylin stains the nucleus blue, and eosin stains the cytoplasm and extracellular matrix pink. Pathologists who visually inspect H&E stained slides can use this information to assess the morphological features of the tissue. However, H&E stained slides typically contain insufficient information to assess the presence or absence of a specific biomarker by visual inspection. Visualization of specific biomarkers (e.g., protein or RNA biomarkers) can be achieved with additional staining techniques that depend on the use of labeled detection reagents that specifically bind to the marker of interest, such as immunofluorescence, immunohistochemistry, in situ hybridization, etc. Such techniques are useful for determining the expression of a single gene or protein, but are not practical for assessing complex expression patterns involving a large number of biomarkers. Global expression profiles can be achieved in a genomic and proteomic manner using separate samples derived from the same tissue source as the specimen for histopathological analysis. However, such methods are expensive and time consuming, require the use of specialized equipment and reagents, and do not provide any information that correlates biomarker expression with specific regions within a tissue specimen (e.g., specific regions within an H&E stained image).

[0190] As used herein, the term "digital image" refers to an electronic image represented by a collection of pixels, which can be viewed, processed and / or analyzed by a computer. In some aspects of the present disclosure, a digital image of a histological slide (e.g., an H&E stained slide) allows a computational evaluation of a tissue specimen as a supplement or alternative to a visual inspection performed by a pathologist. In some embodiments, a digital image can be acquired by means of a digital camera or other optical device capable of capturing a digital image from a slide or a portion thereof. In other embodiments, a digital image can be acquired by means of a non-electronic image of a scanned slide or a portion thereof. In some embodiments, the digital image used in the application provided herein is a whole slide image. As used herein, the term "whole slide image (WSI)" refers to an image including all or nearly all parts of a tissue section (e.g., a tissue section present on a histological slide). In some embodiments, a WSI includes an image of an entire slide. In other embodiments, the digital image used in the application provided herein is a selected portion of a tissue section (e.g., a tissue section present on a histological slide). In some embodiments, a digital image is acquired after the tissue section has been treated with a stain (e.g., H&E).

[0191] In some aspects, a computer-implemented method of predicting the likelihood of a certain prognosis (e.g., recurrence, overall survival, or disease-free survival) comprises the following:

[0192] access to digital histology images of histology sections obtained from subjects;

[0193] extracting a plurality of feature vectors of the histological image by applying a first convolutional neural network, wherein each feature of the plurality of feature vectors represents a local descriptor of the histological image;

[0194] classifying the histological image using at least a plurality of feature vectors and a classification model, wherein the classification model is trained using a training set of known histological images and known prognostic information; and

[0195] The likelihood of a prognosis for the subject is determined based on the classification of at least the histological image.

[0196] In some aspects, a computer-implemented method of predicting the likelihood of a certain prognosis (e.g., recurrence, overall survival, or disease-free survival) comprises the following:

[0197] obtaining digital images of histological sections from the subject;

[0198] Dividing a digital image into a collection of tiles;

[0199] Extracting a plurality of feature vectors from the set of tiles or a subset thereof;

[0200] Calculate an artificial intelligence (AI) risk score based on histology images using a machine learning model that has been trained by processing multiple training images to predict certain outcomes, such as the likelihood of recurrence, overall survival, or disease-free survival,

[0201] The AI ​​risk score represents the likelihood of the subject's prognosis.

[0202] In some embodiments, the method may further include:

[0203] Obtain clinical attributes derived from subjects;

[0204] calculating a clinical risk score using a clinical model trained using one or more subject attributes; and

[0205] A final risk score for the subject is calculated based on the machine learning risk score and the clinical risk score, where the final risk score represents the likelihood of the subject's prognosis.

[0206] In some embodiments, the digital image of the method is a whole slide image (WSI). In some embodiments, a histological section of a disease (e.g., cancer) sample has been stained with a dye (e.g., with hematoxylin and eosin (H&E)) to visualize underlying tissue structures. Other common stains that can be used to visualize tissue structures in the input image include, for example, Masson trichome stain, Periodic Acid Schiff stain, Prussian Blue stain, Gomori trichome stain, Alcian Blue stain, or Ziehl Neelsen stain.

[0207] The machine learning model can be a self-supervised machine learning model using histological data (such as whole slide images) by extracting features from tiles of whole slide images. In one embodiment, the feature extractor is trained on histological tiles in the domain without annotation. In one embodiment, in order to apply the self-supervised framework to histological data, tiles from all WSIs extracted in the tile step are cascaded to form a training data set. The feature extractor is then trained on this unlabeled tile image set using MoCo v2. Initially, the tile set can be divided into two batches of tiles. The first batch of tiles and the second batch of tiles can be modified by, for example, adding 90° rotation and vertical flipping and also performing color enhancement. Since histological tiles contain the same information regardless of their orientation, rotation is a good enhancement for execution. Since histological tiles are images containing cells or tissues and are not dependent on orientation, such tiles can be viewed correctly without considering rotation, horizontal flipping, etc. Therefore, rotating images provides valuable enhancements without losing any important features of the image. By applying tile batches to their corresponding feature extractors, tile embeddings can be generated.

[0208] In one embodiment, the tile embedding is the output of the feature extractor and serves as a signature for each tile, which contains the semantic information of each tile. In other words, the tile embedding is a representation of a tile that contains semantic information about the tile.

[0209] In some embodiments, the self-supervised learning algorithm uses a contrastive loss to shape the patch embeddings so that different enhanced views of the same image are close to each other, or have similar patch embeddings. In other words, the contrastive loss can compare the two patch embeddings, and based on the comparison, the first feature extractor can be adjusted so that its patch embedding is similar to the patch embedding of the second feature extractor. The gradient is back-propagated through the first feature extractor. In some embodiments, the weights of the second feature extractor are updated with the exponential moving average (EMA) of the weights of the first extractor. In some embodiments, the use of EMA can avoid overfitting. Therefore, the output of the system is a trained feature extractor that has been trained using histology patches in the domain so that multiple enhanced patch embeddings of the same image are similar. This type of specific trained feature extractor can significantly improve downstream performance, as discussed below. In some embodiments, after a certain number of epochs of training, a trained feature extractor can be achieved. In some embodiments, training is performed until the accuracy is at or close to 1 (or 100%), until the AUC is at or close to 1 (or 100%), or until the loss is close to zero. In some embodiments, during the training of a feature extractor with contrastive loss, a number of helpful metrics may not be accessible. Therefore, one of the available metrics for the downstream task (such as AUC) can be monitored to see how the feature extractor performs. In one example, a downstream weakly supervised task can be trained using a feature extractor trained at some stage to evaluate performance. Additional training may be justified if it improves downstream performance.

[0210] In some embodiments, the second feature extractor may be optional, and a single feature extractor may be used to generate tile embeddings from both batches of tiles. In such embodiments, one feature extractor is used to generate tile embeddings from both batches of tiles, and the two tile embeddings are compared using a contrastive loss as described above, and the first feature extractor is adjusted so that the first tile embedding is similar to the second tile embedding.

[0211] In some embodiments, the machine learning algorithm extracts a plurality of feature vectors from the digital image and uses a first convolutional neural network (e.g., a ResNet50 neural network) to extract the plurality of feature vectors.

[0212] In some embodiments, the computer-implemented method further comprises removing background segments from the image. In some embodiments, removing background segments from the image is performed using a second convolutional neural network. In some embodiments, the second convolutional neural network is a semantic segmentation deep learning network.

[0213] In some embodiments, the computer-implemented method further comprises selecting a subset of tiles to apply to the machine learning model. In some embodiments, the subset of tiles is selected by random sampling.

[0214] In some embodiments, a machine learning model is trained using a plurality of training images, and the plurality of training images include digital images of histological sections of a disease (e.g., cancer samples derived from a plurality of control subjects suffering from the disease (e.g., cancer)). In some embodiments, the plurality of training images include images without local annotations. In some further embodiments, the plurality of training images include images associated with one or more global labels indicating one or more disease features of the control patients from which the samples were derived.

[0215] In one embodiment of self-supervised learning of histology images to train a feature extractor, the process begins by receiving a training set of histology images. In some embodiments, each image in the training set of images is an unlabeled whole slide image. The process then continues by tiling and enhancing the training set of images into a set of tiles. In one embodiment, a digital image can be divided into a set of tiles. Tiling the images can include dividing the original image into more manageable smaller images, which are called tiles. In one embodiment, the tiling operation is performed by applying a fixed grid to the whole slide image, using a segmentation mask generated by a segmentation method, and selecting tiles that contain tissue or any other area of ​​interest. In order to further reduce the number of tiles to be processed, in one embodiment, additional or alternative selection methods can be used, such as random subsampling to retain only a given number of slides.

[0216] In one embodiment, enhancement may be applied to each set in a set of tiles. The process continues by performing the following operations on each batch of tiles selected from the set of tiles to generate a processed set of tiles. A first set of features is extracted from the first batch of enhanced tiles, and a second set of features is extracted from the second batch of enhanced tiles. In some embodiments, the enhanced tiles include magnified or rotated views, or views with color enhancement. For example, since orientation is not important in a histology slide, the slide may be rotated at a variety of angles. The slide may also be enlarged or magnified. The process then uses a contrast loss between pairs of the first extracted feature set and the second extracted feature set to bring matching pairs of tiles closer together and different pairs of tiles more apart. The contrast loss is applied in order to focus on positive pairs extracted from the first feature set and the second feature set, rather than negative pairs.

[0217] The process continues by training a feature extractor using the processed set of tiles generated via operation of the embodiment. In some embodiments, the trained feature extractor disclosed herein may be used to improve the classification of histology images. The process continues by outputting a trained feature extractor that has been trained using a self-supervised ML algorithm. In some embodiments, the feature extractor may be trained for a specific number of epochs such that each training image is seen a specific number of times.

[0218] Using the trained machine learning model, a risk score can be calculated. The process begins by receiving an input histology image. In some embodiments, the input histology image is a WSI, and it can be derived from a patient tissue sample. In some embodiments, the patient tissue sample is known or suspected to contain a tumor.

[0219] In some embodiments, the process includes removing background fragments from the input image. In some embodiments, material detection can be used to obtain only patches of tissue regions from the input image. In some embodiments, after transforming the input image into a hue, saturation, value (HSV) color space, the background can be removed using Otsu's method applied to the hue and saturation channels.

[0220] The process continues by tiling the histological image into a collection of tiles. In one embodiment, the process uses tiling to increase the ability to pre-process the image. For example, in one embodiment, due to the large size of the whole slide image, the use of a tiling method is helpful for histopathological analysis. More generally, when processing specialized images (such as histopathological slides, or satellite images, or other types of large images), the resolution of the image sensors used in these fields can grow as fast as the capacity of the random access memory associated with the sensor. As the image size increases, it is difficult to store batches of images (sometimes even single images) inside the random access memory of a computer. If an attempt is made to store these large images in the dedicated memory of a graphics processing unit (GPU), this difficulty will be exacerbated. This situation makes it computationally difficult to process image slides or any other images of similar size as a whole.

[0221] In one embodiment, tiled images (or background-subtracted images) address this challenge by dividing the original image (or background-subtracted image) into more manageable smaller images (called tiles). In one embodiment, the tiled operation is performed by applying a fixed grid to the whole slide image, using a segmentation mask generated by a segmentation method, and selecting tiles containing tissue or any other type of region of interest for a subsequent classification process. As used herein, a "region of interest" of an image can be any area that is semantically relevant to the task to be performed, particularly in the context of histopathology corresponding to tissue, organs, bones, cells, body fluids, etc. To further reduce the number of tiles to be processed, additional or alternative selection methods may be used, such as random subsampling to retain only a given number of slides.

[0222] For example, and in one embodiment, the process divides the image (or background-subtracted image) into fixed-size tiles (e.g., each tile is 774x774 pixels in size). Alternatively, the tile size can be smaller or larger. In this example, the number of tiles generated depends on the size of the material detected and can vary from a few hundred tiles to 50,000 or more tiles. In one embodiment, the number of tiles is limited to a fixed number that can be set based at least on computational time and memory requirements (e.g., 10,000 tiles).

[0223] For each tile, the process continues by extracting one or more features of the tile. In one embodiment, each of the features is extracted by applying a trained feature extractor, which is trained using an image training set by a contrast loss ML algorithm. In one embodiment, the image training set is a collection of unlabeled images. In one embodiment, the input image and the image training set are from the same domain, which means that they are images of the same category or type. For example, both the input image and the image training set can be histology images. This contrasts with an embodiment in which the image training set includes out-of-domain images or images that are not non-histology images or images that are not of the same category or type as the analyzed image. In one embodiment, the contrast loss ML algorithm is momentum contrast (Momentum Contrast) or momentum contrast v7 (MoCo v7). In some embodiments, the trained feature extractor is an ImageNet type feature extractor. In one embodiment, the trained machine learning model is a machine learning model as described herein.

[0224] In some embodiments, the machine learning model is a deep multi-instance learning model. In some embodiments, the machine learning model is a Weldon model. In some embodiments, the machine learning model is applied to the entire tile group. In some embodiments, the machine learning model is applied to a subset of tiles. The training image may include a digital image of a histological section of a disease (e.g., cancer) sample derived from a plurality of control subjects. In some cases, the training image is not locally annotated. The training image may include an image associated with one or more global tags, which indicates one or more disease characteristics of the control patient from which the sample is derived. In some embodiments, the disease characteristics may include the duration of the survival period, the duration of the disease-free survival period, or the duration of the period to the recurrence of the disease (e.g., cancer). In some embodiments where the disease is cancer, one or more disease characteristics may include one or more of histological subtype, tumor stage, tumor size, number of positive lymph nodes, biomarker status, pathological stage, clinical stage, patient age and / or treatment history, or a combination thereof. In some embodiments, one or more disease characteristics of the patient may be obtained, and the machine learning model may be applied to the extracted features and the patient's disease characteristics. The patient's example disease characteristics may be the same as the disease characteristics represented in the global label associated with the training image.

[0225] The process then calculates the AI ​​risk score for the subject using a machine learning model. The AI ​​risk score can represent the likelihood of a certain prognosis, such as overall survival, disease-free survival, early recurrence, or recurrence after treatment, and can be expressed as a categorical or continuous range. In one embodiment, the AI ​​risk score represents the predicted time from diagnosis to death (overall survival) of the subject.

[0226] In some embodiments, the final risk score can be calculated as a weighted average of the AI ​​risk score and the clinical risk score, where the weights can be the same or different. Using the trained machine learning and clinical models, the process can receive the WSI and clinical attributes to determine the risk score of the subject. In one embodiment, the WSI is a digital image associated with the subject, and the one or more clinical attributes are attributes used as input to the clinical model. The process can then calculate the AI ​​risk score using the trained machine learning model and calculate the clinical risk score using the trained clinical model.

[0227] In one embodiment, the final risk score is the average of the machine learning and clinical risk scores. For example, in one embodiment,

[0228]

[0229] Where R f is the final risk score, R m is the AI ​​risk score, and Rc is the machine clinical risk score. Alternatively, the final risk score may be a weighted average of the machine learning and clinical risk scores. For example, in one embodiment,

[0230] R f =a m R m +a m R m

[0231] where a m and a m They are R m and R c In further embodiments, the process may calculate the final risk score in a different manner (eg, the square root of the sum of squares or another function of the two inputs).

[0232] The final risk score can represent the likelihood of a certain prognosis, such as overall survival, disease-free survival, early relapse, or relapse after treatment, and can be expressed as a categorical or continuous range. In one embodiment, the final risk score represents the predicted time from diagnosis to death (overall survival) of the subject.

[0233] Deep learning models can go beyond conventional biomarkers or histological classifications routinely used by pathologists in prognosis prediction, and can provide improvements in C-indexes when predicting clinical outcomes such as overall survival (OS). For example, a deep learning model (HCCnet) was trained on 390 whole slide images (WSIs) obtained from 194 patients. HCCnet was then tested on 342 WSIs obtained from 328 patients in the Cancer Genome Atlas (TCGA). In this validation set, the performance of the deep learning model was better than the composite score. When the composite score had only a C-index of 0.63, HCCnet obtained a C-index of 0.70. The composite score is based on disease stage according to the American Joint Committee on Cancer (AJCC), age at diagnosis, sex, serum AFP, alcohol consumption, HBV (hepatitis B virus) or HCV infection, other etiologies, undetermined etiology, tumor differentiation according to World Health Organization (WHO) criteria, macrovascular and microvascular invasion, positive surgical margins, and non-neoplastic liver fibrosis (cirrhosis).

[0234] VI. Semi-synthetic simulation of covariate adjustment for deep learning covariates based on clinical trial data

[0235] A. Simulation of hepatocellular carcinoma (HCC) data using HCCnet

[0236] Patients with early-stage hepatocellular carcinoma (HCC) are candidates for local therapy (resection or local ablation). In HCC, adjuvant therapy after resection or local ablation does not exist, although survival outcomes are poor. In the STORM trial, sorafenib, the standard of care for advanced HCC, did not demonstrate benefit over placebo in the adjuvant setting (Bruix et al., 2015 Lancet Oncol 2015;16:1344-54). Trials sponsored by large pharmaceutical companies are currently investigating whether immunotherapy can improve outcomes for patients with HCC after local therapy (clinicaltrials.gov NCT03383458, NCT04102098, NCT03867084, NCT03847428). A deep learning model (HCCnet) for histology slides captures important prognostic signals for overall survival in HCC after curative therapy (Saillard et al., 2020 Hepatology 72:6;2000-2013). The sample size reduction in adjuvant trials that could be obtained by adapting HCCnet was tested in a semisynthetic simulation based on the original external validation set from TCGA-HCC (Cancer Genome Atlas Research Network 2017 Cell 169:1327-1341).

[0237] Eligibility criteria for clinical trials are too restrictive, which leads to limited generalizability and difficulty in enrollment (Kim et al., 2017 J Clin Oncol Off J Am Soc Clin Oncol 35:3737-44; FDA 2020 https: / / www.fda.gov / regulatory-information / search-fda-guidance-documents / enhancing-diversity-clinical-trial-populations-eligibility-criteria-enrollment-practices-and-trial). In addition to ensuring patient safety, restricting eligibility may also be used to ensure the homogeneity of the trial population (FDA 2018 Workshop Rep 12). In non-small cell lung cancer, the use of observational cohorts showed that many inclusion criteria are redundant because they limit potential enrollment in the trial, even if the treatment is equally effective for excluded patients and included patients (Liu et al., 2021 Nature 592:629-633). Since covariate adjustment allows analyses to compensate for heterogeneity in the patient population, we investigated whether adequate covariate adjustment would allow broadening of eligibility criteria while maintaining statistical power. To this end, we used both parametric and semisynthetic (adjuvant HCC) simulation settings.

[0238] HCCnet was applied to 328 early HCC patients from the TCGA HCC dataset (Saillard et al. 2020 Hepatology 72:6; 2000-2013; Cancer Genome Atlas Research Network 2017 Cell 169:1327-1341). Based on this dataset with HCCnet predictions, we performed semi-synthetic simulations. In order to have a dataset with no missing data, we imputed all missing values ​​among 73 clinical variables with less than 50% missing variables and more than one mode. We used a method that relies on factor analysis of mixed data (FAMD), a principal component method for data involving continuous and categorical variables (Josse & Husson 2016 J Stat Softw 70). The imputed variables used for adjustment were tumor stage (1% missing values) and ECOG score (missing values ​​20%). The imputed variables used as eligibility criteria were ECOG score, Child Pugh classification (33% missing), macrovascular invasion (15% missing), and hepatitis B or C infection status (15% and 5% missing values, respectively). Due to the large proportion of early censoring in the HCC-TCGA dataset, we assigned new event times while retaining the observed survival curves and dependence on covariates. To this end, a Cox model for overall survival was fitted on the available prognostic variables (tumor stage, ECOG score, and HCCnet variables). For each simulated patient, we sampled her / his clinical covariates from TCGA. The hazard rate was defined the same as before, except that θ was replaced by the vector of coefficients from the Cox regression The Weibull distribution is replaced by an empirical survival function that depends on the hazard rate from Cox regression and on the baseline survival function. replace:

[0239]

[0240] As before, κ was numerically optimized to set the outcome rate in the placebo group to the observed outcome rate. Finally, all patients with events after 5 years were censored at that time. The sample size for the simulation experiment was selected to be 760 because this is the average sample size among 4 ongoing trials (clinicaltrials.gov NCT03383458, NCT04102098, NCT03867084, NCT03847428) for adjuvant therapy in early HCC. The treatment effect size hr was set to make the estimated statistical power achieved by adjusting clinical variables be 80% for the sample size. The randomization of treatment assignment was stratified according to tumor stage. The efficacy curves of two selections of adjustment variables were compared: adjustment of clinical variables (tumor stage and ECOG score), and adjustment of HCCnet and clinical variables. Statistical power was estimated based on 5e+3 resamplings.

[0241] Figure 7 Figure 2 shows the efficacy curve when HCCnet is added to tumor stage and ECOG. HCCnet allows sample size to be reduced And it allows an increase in efficacy of 4.4%, with an efficacy of 80%.

[0242] Compatibility of this result with the parametric simulation results. The mortality incidence in the HCC-TCGA population was 32.3% at 5 years. Tumor stage and ECOG score in the source population had a C-index of 0.65, while adding HCCnet resulted in a C-index of 0.72.

[0243] As used herein, quantities associated with adjustment for clinical variables (tumor stage and ECOG) are labeled 1, and quantities associated with adjustment for the HCCnet model (tumor stage, ECOG, and HCCnet) are labeled 2. Figure 7 A's Applying the Fleiss equation for both adjustments, the following results were obtained, indicating that adjusting for both HCCnet and clinical variables allowed for a 10.9% reduction in sample size compared to adjusting for clinical variables alone:

[0244]

[0245] Therefore, the semisynthetic simulation results are consistent with the parametric simulation results. In addition, the TCGA population was calculated between HCCnet and the clinical model. The measurement gave a value of 9.0%.

[0246] B. Simulating mesothelioma data using MesoNet

[0247] MesoNet, a deep learning model that uses whole slide (histology) images of tumor tissue to predict overall survival in patients with malignant mesothelioma, has been developed (Courtiol et al., 2019 Nat Medicine 25:10;1519-1525). Deep learning survival predictions can outperform existing subtype classifications used by pathologists. Figure 8A-8C As shown in , using the MesoNet risk score as a covariate adjustment in addition to histologic subtype in the primary analysis of clinical trials would allow researchers to reduce sample size requirements by 6-13% for three large phase 3 clinical trials in mesothelioma (PROMISE-meso trial, BEAT-meso trial, CheckMate 743 trial). In these simulations, current clinical trials were assumed to stratify patients and adjust analyses according to histologic subtype (epithelioid vs. sarcomatoid vs. mixed), and were assumed to have sample sizes corresponding to 80% statistical power. All of these simulations were run using the MesoNet training dataset from Mesobank.

[0248] Furthermore, assuming that patient enrollment occurs on a rolling basis, the reduction in sample size obtained using MesoNet would allow trial duration to drop by 2–8 months compared with current practice. Thus, utilizing MesoNet could reduce sample size, cost, and trial length in mesothelioma trials.

[0249] VII. Achieving target statistical power using less stringent eligibility criteria

[0250] The present disclosure provides methods for achieving target statistical power using less stringent eligibility criteria in a trial by using covariate adjustment, which can compensate for the increased heterogeneity associated with less restrictive inclusion criteria. Using parametric simulation and semi-synthetic simulation, the interaction between more restrictive inclusion criteria or less restrictive inclusion criteria and adjustment for covariates was considered. In the case of parametric simulation, the restrictive inclusion criteria are based on the value of X, and the covariate summarizes the prognostic information: only patients with X below 0, that is, patients at lower risk (for this experiment, X is associated with a hazard ratio of 2) are included with the most restrictive eligibility criteria. Then, when combining two possible eligibility criteria (all patients or restrictive inclusion) and two options for adjustment (no adjustment or adjustment for X), there are 4 scenarios. Other parameters are Weibull shape and treatment risk ratio, which are set to 1 and 0.7, respectively. When all patients are included, the observed incidence of the outcome is 96.5%, and for restrictive inclusion, the observed incidence of the outcome is 93.5%.

[0251] In the case of HCC semisynthetic mimics, we defined two additional levels of restrictive eligibility criteria, as shown in Table 2. Moderately restrictive eligibility level Two inclusion criteria exist in all 4 ongoing large trials for adjuvant therapy in early HCC (clinicaltrials.gov NCT03383458, NCT04102098, NCT03867084, NCT03847428): only patients with a Child-Pugh score of A and an ECOG status of 0 or 1 were included. The most restrictive eligibility criteria further restricted ECOG status to 0, as in the STORM trial (Bruix et al. 2015 Lancet Oncol 16:1344-54), excluded patients with dual infection of hepatitis B and C, as in the KEYNOTE-937 trial (clinicaltrials.gov NCT03867084), and excluded patients with macrovascular invasion, as in the IMBRAVE050 trial (clinicaltrials.gov NCT04102098). We considered only the eligibility criteria available in the TCGA HCC dataset. Therefore, when combining the two options for adjustment considered (clinical adjustment: tumor stage and ECOG or HCCnet adjustment) and the three levels of eligibility, there were 6 different scenarios. In the scenario with the most restrictive eligibility level, every patient had an ECOG of 0, so the analysis was not adjusted according to ECOG.

[0252] In both cases, changing the inclusion criteria changes the number of events, which directly affects the statistical power. Therefore, the statistical power of the different scenarios is presented as a function of the number of events. In both cases, no dropouts were added and 5000 replicates were performed.

[0253] Table 2. Definition of eligibility levels for the HCC-TCGA dataset

[0254]

[0255] Fig. 9 A shows the interaction between eligibility criteria and adjusted selection in a parametric simulation, where the least at-risk patients were selected at 50% inclusion. Fig. 9 B shows the interaction between eligibility criteria and adjustment options in a semisynthetic simulation based on the HCC-TCGA dataset. Fig. 9 A and Fig. 9B shows that in both parametric simulations and semi-synthetic simulations, restricting the population can allow for higher statistical power for the same number of events in the case of unadjusted analysis. This is due to the increased homogeneity of the population. However, when all prognostic information is used to adjust the analysis, the same power curve can be obtained regardless of the inclusion (eligibility) criteria. When considering more restrictive criteria, patients at a lower risk of events (e.g., death) are selected for enrollment. Therefore, considering sample size (i.e., including subjects with events and subjects without events) rather than the number of events will favor more inclusive criteria.

[0256] Although the adjusted analyses with different eligibility criteria had the same statistical power, they implied very different sizes of the screening populations. For example, in the HCC example, the required size of the screening population was 604 for the less restrictive inclusion and 1729 for the most restrictive population. Thus, the screening population was reduced by 65% ​​while achieving the same statistical power. This difference in the screening population can be explained by a smaller proportion of patients included and a smaller proportion of events with restrictive eligibility criteria (20.5% at 5 years vs. 32.3% in the entire population).

[0257] Restrictive eligibility criteria ensure a homogeneous population, but lead to difficulty in enrollment and questionable generalization of trial results. This leads to calls for less restrictive eligibility criteria (FDA 2020https: / / www.fda.gov / regulatory-information / search-fda-guidance-documents / enhancing-diversity-clinical-trial-populations-eligibility-criteria-enrollment-practices-and-trial;FDA2018Workshop Rep 12). The present disclosure provides that adequate covariate adjustment eliminates any motivation to homogenize the population using restrictive eligibility criteria. In fact, in both parameter simulations and semi-synthetic simulations based on actual clinical trial data sets, the adjusted analysis is equally effective regardless of the strictness of the eligibility criteria. Therefore, by using less restrictive eligibility criteria, the size of the population that needs to be screened for inclusion can be greatly reduced while maintaining the same statistical power by performing covariate adjustment.

[0258] As described in the FDA draft guidance, covariate adjustment changes the estimated target, a phenomenon known as non-collapsibility (FDA 2021 https: / / www.fda.gov / regulatory-information / search-fda-guidance-documents / adjusting-covariates-randomized-clinical-trials-drugs-and-biological-products). If marginal estimators are preferred, then adjusted marginal estimators can be considered, which target the estimator of the unadjusted analysis while taking advantage of the precision gain provided by the covariates (Daniel et al. 2020 Biom J 63:528-557; Permutt 2020 Stat Biopharm Res 12:45-53).

[0259] The methods described herein encompass adjusting covariates associated with relative measures of treatment effect (e.g., hazard ratios) as well as absolute measures of treatment efficacy (such as restricted mean survival time or absolute risk reduction). According to the present disclosure, the use of prognostic signals of covariates can make both relative and absolute measures of treatment efficacy more precise.

[0260] VIII. Computer Systems and Machine-Readable Media

[0261] like Fig.10 As shown in FIG. 1 , a computer system 1000 in the form of a data processing system includes a bus 1003 coupled to (one or more) microprocessors 1005 and ROM (read only memory) 1007 and volatile RAM 1009 and non-volatile memory 1013. The microprocessor 1005 may include one or more CPUs, (one or more) GPUs, special processors, and / or combinations thereof. The microprocessor 1005 may communicate with a cache 1004 and may retrieve instructions from memory 1007, 1009, 1013 and execute the instructions to perform the above operations. The bus 1003 interconnects these various components together and also interconnects these components 1005, 1007, 1009, and 1013 to a display controller and display device 1015 and to peripheral devices such as input / output (I / O) devices 1011 which may be a mouse, keyboard, modem, network interface, printer, and other devices well known in the art. Typically, the input / output device 1011 is coupled to the system through an input / output controller 1017. Volatile RAM (Random Access Memory) 1009 is typically implemented as dynamic RAM (DRAM), which requires continuous power to refresh or maintain data in the memory.

[0262] The non-volatile memory 1013 may be, for example, a magnetic hard drive or a magneto-optical drive or an optical drive or a DVD RAM or a flash memory or other type of memory system that maintains data (e.g., large amounts of data) even after power is removed from the system. Typically, the non-volatile memory 1013 will also be a random access memory, although this is not required. Although Fig.10 Non-volatile memory 1013 is shown as a local device directly coupled to the rest of the components in the data processing system, but it should be appreciated that the present invention may utilize non-volatile memory remote from the system, such as a network storage device coupled to the data processing system through a network interface such as a modem, an Ethernet interface, or a wireless network. Bus 1003 may include one or more buses connected to each other through a variety of bridges, controllers, and / or adapters as are well known in the art.

[0263] The parts described above can be implemented with logic circuit systems such as special logic circuits or with processing cores of microcontrollers or other forms of execution program code instructions. Therefore, the process taught by the above discussion can be performed with program codes such as machine executable instructions, which make the machine executing these instructions perform certain functions. In this context, "machine" can be a machine (e.g., an abstract execution environment, such as a "virtual machine" (e.g., a Java virtual machine), an interpreter, a common language runtime, a high-level language virtual machine, etc.) that converts intermediate form (or "abstract") instructions into processor-specific instructions, and / or an electronic circuit system deployed on a semiconductor chip (e.g., a "logic circuit system" implemented with transistors) designed to execute instructions, such as a general-purpose processor and / or a special-purpose processor. The processing taught by the above discussion can also be performed by an electronic circuit system (as a substitute for a machine or in combination with a machine) designed to perform processing (or a part thereof) without executing program code.

[0264] The present invention also relates to an apparatus for performing the operations described herein. The apparatus may be specially constructed for the desired purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs and magneto-optical disks, read-only memories (ROMs), RAMs, EPROMs, EEPROMs, magnetic or optical cards, or any type of medium suitable for storing electronic instructions, and each coupled to a computer system bus.

[0265] A machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer). For example, machine-readable media include read-only memory ("ROM"); random access memory ("RAM"); magnetic disk storage media; optical storage media; flash memory devices; and the like.

[0266] Articles of manufacture may be used to store program code. Articles of manufacture storing program code may be implemented as, but not limited to, one or more memories (e.g., one or more flash memories, random access memories (static, dynamic or other)), optical disks, CD-ROMs, DVD ROMs, EPROMs, EEPROMs, magnetic or optical cards, or other machine-readable media suitable for storing electronic instructions. Program code may also be downloaded from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of a data signal implemented in a propagation medium (e.g., via a communication link (e.g., a network connection)).

[0267] The foregoing detailed description has been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are tools used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here and generally considered to be a self-consistent sequence of operations leading to a desired result. The operations are those requiring physical manipulation of physical quantities. Typically, although not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. At times, it has proven convenient, primarily for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0268] It should be remembered, however, that all of these and similar terms will be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise apparent from the above discussion, it should be recognized that discussions throughout the specification using terms such as "segmentation," "tiling," "receiving," "computing," "extracting," "processing," "applying," "enhancing," "normalizing," "pre-training," "sorting," "selecting," "aggregating," "sorting," and the like refer to the behavior and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage, transmission, or display devices.

[0269] The processes and displays presented herein are not inherently related to any particular computer or other device. A variety of general purpose systems may be used together with the programs according to the teachings herein, or the teachings may prove convenient to construct more specialized devices to perform the described operations. The structures required for a variety of these systems will be apparent from the description herein. In addition, the present invention is not described with reference to any specific programming language. It should be appreciated that a variety of programming languages ​​may be used to implement the teachings of the present invention as described herein.

[0270] The above discussion describes only some exemplary embodiments of the present invention. Those skilled in the art will readily recognize from these discussions, the accompanying drawings, and the claims that various modifications may be made without departing from the spirit and scope of the present invention. In addition, where feasible, any of the aspects disclosed herein may be combined with each other (e.g., features according to one aspect may be added to features of another aspect or replaced with equivalent features of another aspect) or with features well known in the art, unless the context indicates otherwise.

[0271] All references cited, including, for example, citations to patents, published patent applications, and articles, are incorporated herein by reference in their entirety.

[0272] The section headings used herein are for organizational purposes only and are not to be construed as limiting the subject matter described in any way.

Claims

1. A method for designing a randomized controlled trial (RCT) with time-to-event outcomes, the method comprising: Selection of covariates for adjustment; as well as The number of events required to achieve statistical power was calculated based on the following formula: in RCTs are conducted using a calculated number of events. Where N original is the raw number of events required to achieve statistical power without covariate adjustment, Where N adjusted is the adjusted number of events required to achieve statistical power with covariate adjustment, and in It is calculated based on the following formula for RCT external data: in Cox-Snell 2 , n is the number of participants, l0 is the log-likelihood of the Cox model explaining the time-to-event outcome using only the intercept, and l1 is the log-likelihood of the Cox model explaining the time-to-event outcome with the intercept and adjusting for covariates.

2. A method for estimating sample size at an interim stage of an ongoing randomized controlled trial (RCT), the method comprising: Selection of covariates for adjustment; Obtain blinded RCT data; as well as In the mid-term, use The blinded sample size re-estimation was performed using the following formula: in The RCT was further conducted using blinded sample size re-estimation, Where N original is the raw number of events required to achieve statistical power without covariate adjustment, Where N adjusted is the re-estimated number of events required to achieve statistical power; in It is calculated for blinded RCT data at the interim stage based on the following formula: in Cox-Snell 2 , n is the number of participants, l0 is the log-likelihood of the Cox model explaining the time-to-event outcome using only the intercept, and l1 is the log-likelihood of the Cox model explaining the time-to-event outcome with the intercept and adjusting for covariates.

3. The method of claim 1 or 2, further comprising estimating the original number of events (N) required to obtain statistical power without covariate adjustment based on the following formula: original ): N ariginal =(z β +z1-α) 2 / (P1P2log 2 ((hr)), where N original is the estimated number of events required to achieve statistical power based on Schoenfeld's formula, α is the Type I error level, β is the Type II error level, P1 and P2 are the proportions of the experimental sample included in the treatment and control groups, respectively (e.g., if the treatment assignments are balanced, both are equal to 1 / 2), hr is the stated hazard ratio, and z p is the p-quantile of the standard normal distribution.

4. The method of any one of claims 1-3, wherein the time-to-event outcome is overall survival, disease-free survival, or time to disease recurrence.

5. The method of any one of claims 1-4, wherein an RCT is performed to assess the efficacy of treatment in cancer patients.

6. The method of claim 5, wherein the cancer is hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.

7. The method of any one of claims 1-6, wherein the clinical risk score is adjusted for covariates, the method comprising: Obtain clinical attributes derived from subjects; calculating a clinical risk score using a clinical model trained using one or more subject attributes, The clinical risk score quantifies the prognosis of the subject.

8. The method of any one of claims 1-6, wherein covariate adjustment is performed on covariates obtained by the deep learning model.

9. The method of claim 8, wherein the deep learning model is based on histopathology slides obtained from cancer subjects and the covariates are prognostic covariates.

10. The method of claim 8 or 9, wherein the covariates are obtained by a computer-implemented method for determining the likelihood of a prognosis of a subject suffering from a disease, comprising: access to digital histology images of histology sections obtained from subjects; extracting a plurality of feature vectors of the histology image by applying a first convolutional neural network, wherein each of the features of the plurality of feature vectors represents a local descriptor of the histology image; classifying the histological image using at least the plurality of feature vectors and the classification model, wherein the classification model is trained using a training set of known histological images and known prognostic information; and The likelihood of a prognosis for the subject is determined based on the classification of at least the histological image.

11. The method of claim 8 or 9, wherein the covariate is obtained by a computer-implemented method for determining the prognosis of a subject suffering from a disease, the method comprising: obtaining digital histology images of histology sections from the subject; Dividing a digital image into a collection of tiles; Extracting a plurality of feature vectors from the set of tiles or a subset thereof; as well as Calculate an artificial intelligence (AI) risk score based on histology images using a machine learning model that has been trained to predict prognosis by processing multiple training images, The AI ​​risk score quantifies the subject's prognosis.

12. The method of claim 11, further comprising: Obtain clinical attributes derived from subjects; calculating a clinical risk score using a clinical model trained using one or more subject attributes; as well as The final risk score of the subject is calculated based on the AI ​​risk score and the clinical risk score. The final risk score quantifies the subject's prognosis.

13. The method of any one of claims 10 to 12, wherein the digital histology image is a whole slide image (WSI).

14. The method of any one of claims 10-13, wherein the histological section has been stained with a dye.

15. The method of claim 14, wherein the dye is hematoxylin and eosin (H&E).

16. The method of any one of 10-15, wherein the disease is cancer.

17. The method of claim 16, wherein the cancer is hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.

18. The method of any one of claims 1-17, wherein enrollment of subjects based on restrictive eligibility criteria does not improve statistical power relative to enrollment of subjects based on less restrictive eligibility criteria.

19. The method of any one of claims 1-18, wherein less stringent eligibility criteria are used in the trial to achieve a target statistical power.

20. The method of any one of claims 1 to 19, wherein the method is computer-implemented.

21. A machine-readable medium having executable instructions, the instructions causing one or more processing units to perform a method of designing a randomized controlled trial (RCT), the method comprising: Selection of covariates for adjustment; as well as The sample size required to achieve statistical power was calculated based on the following formula: in RCTs are conducted using a calculated sample size. Where N original is the raw number of events required to achieve statistical power without covariate adjustment, Where N adjusted is the adjusted number of events required to achieve statistical power with covariate adjustment, and in It is calculated based on the following formula for RCT external data: in Cox-Snell 2 , n is the number of participants, l0 is the log-likelihood of the Cox model explaining the time-to-event outcome using only the intercept, and l1 is the log-likelihood of the Cox model explaining the time-to-event outcome with the intercept and adjusting for covariates.

22. A machine-readable medium having executable instructions, the instructions causing one or more processing units to perform a method for estimating a sample size required to achieve statistical power at an interim stage of an ongoing randomized controlled trial (RCT), the method comprising: Selection of covariates for adjustment; Obtain blinded RCT data; as well as In the mid-term, use The blinded sample size re-estimation was performed using the following formula: in The RCT was further conducted using blinded sample size re-estimation. Where N original is the raw number of events required to achieve statistical power without covariate adjustment, Where N adjusted is the re-estimated number of events required to achieve statistical power; in It is calculated for blinded RCT data at the interim stage based on the following formula: in Cox-Snell 2 , n is the number of participants, l0 is the log-likelihood of the Cox model explaining the time-to-event outcome using only the intercept, and l1 is the log-likelihood of the Cox model explaining the time-to-event outcome with the intercept and adjusting for covariates.

23. The machine-readable medium of claim 21 or 22, wherein the method further comprises estimating the original number of events (N) required to achieve statistical power without covariate adjustment based on the following formula: original ): N original =(z β +z 1-α ) 2 / (P1P2log 2 (hr)), where N original is the estimated number of events required to achieve statistical power based on Schoenfeld's formula, α is the Type I error level, β is the Type II error level, P1 and P2 are the proportions of the experimental sample included in the treatment and control groups, respectively (e.g., both equal 1 / 2 if the treatment assignments are balanced), and hr is the stated hazard ratio, and z p is the p-quantile of the standard normal distribution.

24. The machine-readable medium of any one of claims 21-23, wherein the time-to-event outcome is overall survival, disease-free survival, or time to disease recurrence.

25. The machine-readable medium of any one of claims 21-24, wherein the RCT is performed to assess the efficacy of treatment in cancer patients.

26. The machine-readable medium of claim 25, wherein the cancer is hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.

27. The machine-readable medium of any one of claims 21-26, wherein the clinical risk score is adjusted for covariates, the machine-readable medium comprising: Obtain clinical attributes derived from subjects; calculating a clinical risk score using a clinical model trained using one or more subject attributes, The clinical risk score quantifies the prognosis of the subject.

28. The machine-readable medium of any one of claims 21-26, wherein covariates obtained by the deep learning model are covariate adjusted.

29. The machine-readable medium of claim 28, wherein the deep learning model is based on histopathology slides obtained from cancer patients and the covariates are prognostic covariates.

30. The machine-readable medium of claim 28 or 29, wherein the covariates are obtained by the computer-implemented machine-readable medium for determining the likelihood of a prognosis of a subject having a disease, comprising: access to digital histology images of histology sections obtained from subjects; extracting a plurality of feature vectors of the histology image by applying a first convolutional neural network, wherein each of the features of the plurality of feature vectors represents a local descriptor of the histology image; classifying the histological image using at least the plurality of feature vectors and the classification model, wherein the classification model is trained using a training set of known histological images and known prognostic information; and The likelihood of a prognosis for the subject is determined based on the classification of at least the histological image.

31. The machine-readable medium of claim 28 or 29, wherein the covariate is obtained by a computer-implemented machine-readable medium for determining a prognosis of a subject suffering from a disease, the machine-readable medium comprising: obtaining digital histology images of histology sections from the subject; Dividing a digital image into a collection of tiles; Extracting a plurality of feature vectors from the set of tiles or a subset thereof; as well as Calculate an artificial intelligence (AI) risk score based on histology images using a machine learning model that has been trained to predict prognosis by processing multiple training images, The AI ​​risk score quantifies the subject's prognosis.

32. The machine-readable medium of claim 31 , further comprising: Obtain clinical attributes derived from subjects; calculating a clinical risk score using a clinical model trained using one or more subject attributes; as well as The final risk score of the subject is calculated based on the AI ​​risk score and the clinical risk score. The final risk score quantifies the subject's prognosis.

33. The machine-readable medium of any one of claims 30-32, wherein the digital histology image is a whole slide image (WSI).

34. The machine-readable medium of any one of claims 30-33, wherein the histological section has been stained with a dye.

35. The machine-readable medium of claim 34, wherein the dye is hematoxylin and eosin (H&E).

36. The machine-readable medium of any one of claims 30-35, wherein the disease is cancer.

37. The machine-readable medium of claim 36, wherein the cancer is hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.

38. The machine-readable medium of any one of claims 21-37, wherein enrollment of subjects based on restrictive eligibility criteria does not improve statistical power relative to enrollment of subjects based on less restrictive eligibility criteria.

39. The machine-readable medium of any one of claims 21-38, wherein less stringent eligibility criteria are used in the trial to achieve a target statistical power.

Images