Systems and methods for designing randomized controlled studies

Abstract

Methods are provided for designing studies, e.g., randomized controlled trials (RCTs), and for assessing sample sizes in pre-start or ongoing studies. Also provided are methods for performing covariate adjustments using covariates (e.g., prognostic covariates) captured by deep learning models in studies.

Description

[Technical field]

[0001] The present invention relates generally to adjusting for covariates and the survey design process. [Background technology]

[0002] In a study whose goal is to test for the presence of a treatment effect (b1), the relationship between the outcome (Y), the treatment assignment (T), and a prognostic covariate (X) associated with Y can be expressed as, for example, Y=b0+b1T+b2X+eps where b0, b1, and b2 represent coefficients and eps (epsilon) represents error. In this formula, covariate adjustment (i.e., giving b2X) allows for a more precise estimation of the treatment effect (b1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Friede and Kieser, 2011, Pharmaceut. Statist. 10:8-13 [Non-Patent Document 2] Heo et al., 1998, Mech Aging Dev 102:45–53 [Non-Patent Document 3] Hernandez et al., 2006, Ann Epidemiol 16:41–48 [Non-Patent Document 4] Kerr et al., 2017, Clin Trials 14:629–638 [Non-Patent Document 5] Royston and Sauerbrei, 2004, Stat Med 23:723-748 [Non-Patent Document 6] Kent and O'Quigley, 1988, Biometrika 75:525-534 [Non-Patent Document 7] Royston, 2006, Stata J Promot Commun Stat Stata 6:83~96

Non-licensed Document 8

Non-licensed literature 9

Non-licensed literature 10

Non-licensed Document 11

Non-licensed Document 12

Non-licensed Document 13

Non-licensed Document 14

Non-licensed Document 15

Non-licensed Document 16

Non-licensed Document 17

Non-licensed Document 18

Non-licensed Document 19

Non-licensed Document 20

Non-licensed Document 21

Non-licensed Document 22

Non-licensed Document 23

Non-licensed Document 24

Non-licensed Document 25

Non-licensed Document 26

Non-licensed Document 27

[0004] Adjustment for prognostic covariates allows for improved precision and efficiency in analysis and increased statistical power for estimating treatment effects in studies, for example, randomized clinical trials. Covariate adjustment allows for achieving the same statistical power with smaller sample sizes. Adjustment covariates must be pre-specified and selected based on their prognostic value and not based on any imbalance criteria. Recognizing the significance of covariate adjustment in studies, this methodological consensus has been implemented by regulatory authorities, including the European Medical Association (EMA) and the Food and Drug Administration (FDA) as regulatory guidance. Achieving the same statistical power with smaller sample sizes and / or less stringent eligibility criteria would allow for a reduction in the size of the population that needs to be screened for enrollment in the study. Thus, there is a need for systems and methods to efficiently adjust covariates in studies, reduce sample sizes, and / or relax eligibility criteria, and improve the efficiency of hypothesis testing. [Means for solving the problem]

[0005] The present disclosure provides methods for designing (e.g., shrinking) sample sizes in clinical trials based on known correlations of adjustment covariates and outcomes, as well as methods for, e.g., blinded sample size reestimation, to re-estimate and re-adjust sample sizes while maintaining the blindness of the trial based on data that becomes available during the trial in both continuous and time-to-event outcome settings. The methods disclosed herein can incorporate new prognostic sources, e.g., prognostic scores obtained by deep learning, as adjustment covariates to calculate adjusted (e.g., shrink) sample sizes. The present disclosure also provides methods to achieve target statistical power with less stringent eligibility criteria in clinical trials by using covariate adjustments that can compensate for the increased heterogeneity brought about by less restrictive inclusion criteria.

[0006] In one aspect, the present disclosure provides a method of designing a randomized controlled trial (RCT) with a time-to-event outcome, the method comprising: selecting covariates for adjustment; N adjusted =N original (1-R 2 CS ) Calculating the number of events required to obtain statistical power based on the formula: where RCTs were conducted using event rate estimates, N original is the original number of events required to obtain statistical power without covariate adjustment, N adjusted is the adjusted number of events required to obtain statistical power using covariate adjustment, R 2 CS is the expression: Calculated for data outside the RCT based on JPEG2025512702000002.jpg1259, where: R 2 CS Cox·Snell R 2 and n is the number of participants, l0 is the log-likelihood of a Cox model explaining the time-to-event outcome by the intercept alone, l1 is the log likelihood of the Cox model explaining the time-to-event outcome by intercept and covariate adjustment.

[0007] In another aspect, the present disclosure provides a method for assessing sample size at an intermediate stage of an ongoing randomized controlled clinical trial (RCT), the method comprising: selecting covariates for adjustment; The stage of obtaining blinded RCT data and In the intermediate stage, 2 CS and the formula:N adjusted =N original (1-R 2 CS ) performing a blind sample size re-estimation using where The RCT was further conducted using blinded sample size re-estimation. N adjusted is the number of re-estimated events required to obtain statistical power, R 2 CS At the intermediate stage, the expression: Calculated based on blinded RCT data based on JPEG2025512702000003.jpg1259, where: R 2 CS Cox·Snell R 2 and n is the number of participants, l0 is the log-likelihood of a Cox model explaining the time-to-event outcome by the intercept alone, l1 is the log likelihood of the Cox model explaining the time-to-event outcome by intercept and covariate adjustment.

[0008] In some embodiments, the original number of events required to obtain statistical power without covariate adjustment (N original ) is the formula: It is evaluated based on JPEG2025512702000004.jpg10110, where: N original is the estimated number of events required to obtain statistical power based on the Schoenfeld formula, α is the Type I error level, β is the Type II error level, P1 and P2 are the proportions of the trial sample included in the treatment and control groups, respectively (e.g., both are equal to 1 / 2 in the case of balanced treatment allocation); hr is the specified hazard ratio, z p is the pth quantile of the standard normal distribution.

[0009] In some embodiments, the time-to-event outcome is overall survival, disease-free survival, or time to disease recurrence. In some embodiments, the RCT is performed to evaluate the efficacy of treatment in cancer patients. In some embodiments, the cancer is hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.

[0010] In some embodiments, the covariate adjustment is made to a clinical risk score, and the method comprises: obtaining subject-derived clinical attributes; calculating a clinical risk score using a clinical model trained using one or more subject attributes; wherein the clinical risk score quantifies the prognosis of the subject.

[0011] In some embodiments, the covariate adjustment is performed on a covariate obtained by a deep learning model, hi some embodiments, the deep learning model is based on histopathological slides obtained from cancer subjects, and the covariate is a prognostic covariate.

[0012] In some embodiments, the covariate is: accessing a digital histology image of a histology section obtained from the subject; extracting a plurality of feature vectors of the histology image by applying a first convolutional neural network, each feature of the plurality of feature vectors representing a local descriptor of the histology image; classifying the histology image using at least a plurality of feature vectors and a classification model, the classification model being trained using a training set of known histology images and known prognostic information; determining a prognostic likelihood for the subject based on at least the classification of the histology image; The method is obtained by a computer-implemented method for determining a prognostic likelihood for a subject having a disease comprising:

[0013] In some embodiments, the covariates are obtained by a computer-implemented method for determining a prognosis of a subject with a disease, the method comprising: obtaining a digital histology image of a histology section from a subject; Segmenting the digital image into a set of tiles; extracting a plurality of feature vectors from the tile set or a subset thereof; calculating an artificial intelligence (AI) risk score based on the histological image using a machine learning model trained by processing a plurality of training images to predict prognosis; and the AI ​​risk score quantifies the subject's prognosis.

[0014] In some embodiments, the method further comprises: obtaining subject-derived clinical attributes; calculating a clinical risk score using a clinical model trained using one or more subject attributes; calculating a final risk score for the subject from the AI ​​risk score and the clinical risk score; wherein the final risk score quantifies the subject's prognosis.

[0015] In some embodiments, the digital histology image is a whole slide image (WSI). In some embodiments, the histology section has been stained with a dye. In some embodiments, the dye is hematoxylin and eosin (H&E). In some embodiments, the disease is cancer, for example, hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.

[0016] In some embodiments, subject enrollment based on restrictive eligibility criteria does not improve statistical power relative to subject enrollment based on less restrictive eligibility criteria. In some embodiments, the target statistical power is achieved using less stringent eligibility criteria in the clinical trial. In some embodiments, the method is computer-implemented.

[0017] In some aspects, the present disclosure provides a machine-readable medium having executable instructions to cause one or more processing units to execute a method of designing a randomized controlled clinical trial (RCT) or a method of assessing the sample size required to obtain statistical power at an intermediate stage of an ongoing randomized controlled clinical trial (RCT), as provided herein.

[0018] The patent or application file contains at least one color drawing. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0019] The present invention is illustrated by way of example, and not limitation, in the figures of the accompanying drawings in which like references indicate similar elements. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 shows an illustrative workflow for designing a randomized clinical trial and re-estimating and readjusting sample size to optimize the efficiency and statistical power of the trial. [Diagram 2] FIG. 1 shows an illustrative workflow for designing sample size based on known correlations between adjustment covariates and outcomes. [Diagram 3] FIG. 1 shows an illustrative workflow for blinded sample size re-estimation in an ongoing clinical trial. [Figure 4] FIG. 1 shows an exemplary workflow of a parametric simulation in which clinical trial cases are simulated to enable comparison of adjusted and unadjusted analyses for parameter sets corresponding to clinical trial scenarios. [Figure 5A] FIG. 1 depicts the behavior of R2 obs at hr=0.7 over the range of C-index, outcome incidence (λ), treatment effect (hr), Weibull shape (w) and dropout rate (d). [Figure 5B] FIG. 1 depicts the behavior of R2 obs at hr=0.4 over the range of C-index, outcome incidence (λ), treatment effect (hr), Weibull shape (w) and dropout rate (d). [Figure 5C] FIG. 1 depicts the behavior of R2 obs over the range of C-index, outcome incidence (λ), treatment effect (hr), Weibull shape (w) and dropout rate (d) at hr=0.7, d=0.01, w=0.5. [Figure 6A] FIG. 13 depicts the relationship between the proposed R2 measure and R2 obs when w=0.5. [Figure 6B] This figure depicts the relationship between the proposed R2 measure and R2 obs when w=1. [Figure 6C] FIG. 13 depicts the relationship between the proposed R2 measure and R2 obs when w=1.5. [Figure 7] FIG. 1 depicts power curves resulting from adjustment for clinical variables only (tumor stage and ECOG score) and adjustment for clinical variables and additional deep learning covariates HCCnet sampled from TCGA-HCC. [Figure 8A] FIG. 1 depicts power curves resulting from adjustment for histologic subtype for the PROMISE-meso trial (current trial) and from adjustment for histologic subtype and the additional deep learning covariate MesoNet when running simulations with the MesoNet training dataset from Mesobank (trial with MesoNet). [Figure 8B] FIG. 1 depicts the power curves resulting from adjusting for histological subtype for the BEAT-meso trial (current trial) and adjusting for histological subtype and the additional deep learning covariate MesoNet when running simulations with the MesoNet training dataset from Mesobank (trial with MesoNet). [Figure 8C] FIG. 1 depicts power curves resulting from adjustment for histological subtype for the CheckMate743 trial (current trial) and from adjustment for histological subtype and the additional deep learning covariate MesoNet when simulations were run with the MesoNet training dataset from Mesobank (trial with MesoNet). [Figure 9](A) depicts the results of the interaction between eligibility criteria and adjustment options in a parametric simulation in which the lowest-risk patients were selected with an inclusion rate of 50%, with three inclusion levels based on eligibility criteria from past and ongoing clinical trials; (B) depicts the results of the interaction between eligibility criteria and adjustment options in a semi-synthetic simulation based on the HCC-TCGA dataset, with three inclusion levels based on eligibility criteria from past and ongoing clinical trials. [Figure 10] FIG. 1 illustrates an example of a computer system that can be used in conjunction with the embodiments described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Systems and methods for covariate analysis are described. In some embodiments, the Cox·Snell R 2 In some embodiments, the prognostic score for a disease, e.g., cancer, obtained using deep learning on a histological slide is based on the Cox·Snell R 2 can be used as a covariate to be adjusted for using

[0022] In the following description, numerous specific details are set forth in order to provide a thorough description of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention can be implemented without using these specific details. In other respects, well-known components, structures, and techniques are not shown in detail in order to avoid obscuring the understanding of the present description.

[0023] References herein to "one embodiment" or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present invention. The appearances of the phrase "in one embodiment" in various places in the present specification do not necessarily all refer to the same embodiment. The term "exemplary" is used herein to mean "example" rather than "ideal." It should be understood from the present disclosure that the present invention is not limited to the examples described herein.

[0024] With respect to any method described herein, the ordering of steps presented, whether described in the text or in the accompanying flow charts, should not be understood to imply that these steps must be performed in the order presented, unless otherwise expressly stated or required by context. In other words, the order of steps represents one embodiment of the presented method, and generally such steps can alternatively be performed in different orders or simultaneously. The processes depicted in the figures that follow can be performed by hardware (e.g., circuits, dedicated logic, etc.), software (such as those executed on a general-purpose computer system or a dedicated machine), or a combination of both. Although the processes are described below with respect to some sequential operations, it should be appreciated that some of the operations described can be performed in different orders. Additionally, some operations can be performed in parallel rather than sequentially.

[0025] Computational methods used to implement the methods provided herein can include, for example, machine learning, artificial intelligence (AI), deep learning (DL), neural networks, classification and / or clustering algorithms, and regression algorithms.

[0026] The terms "server," "client," and "device" are intended to refer generally to data processing systems, rather than to specifically refer to particular form factors for servers, clients, and / or devices.

[0027] The articles "a" and "an" are used to refer to one or to more than one (i.e., to at least one) of the grammatical object of the article. By way of example, "an element" means one element or more than one element, e.g., a plurality of elements.

[0028] The term "comprising" is used herein to mean, and is used interchangeably with, the phrase "including but not limited to." The term "comprising" does not necessarily imply that there must be additional elements other than those listed.

[0029] The term "about" or "approximately" when referring to a number or range of numbers means that the referenced number or range of numbers is approximate within experimental variability (or within statistical experimental error), and thus the number or range of numbers may vary, for example, between 1% and 20% of the stated number or range of numbers. In some embodiments, "about" refers to a value that is within 20% of the stated value. In more preferred embodiments, "about" refers to a value that is within 10% of the stated value. In even more preferred embodiments, "about" refers to a value that is within 1% of the stated value.

[0030] Unless otherwise indicated, all numbers used in the specification and claims expressing properties such as amounts of ingredients, molecular weights, reaction conditions, and the like, are to be understood as being modified in all instances by the term "about." Thus, unless otherwise indicated, the numerical properties set forth in the following specification and claims are approximations that may vary depending on the desired properties sought to be obtained in the various embodiments of the present invention. Notwithstanding that the numerical ranges and parameters setting forth the broad scope of the invention are approximations, the numerical values ​​set forth in the specific examples are numerical values ​​that are reported as precisely as possible. However, any numerical value inherently contains certain errors necessarily resulting from errors found in its respective measurements.

[0031] The term "at least" preceding a number or series shall be understood to include the number adjacent to the term "at least" and all subsequent numbers or integers that may be included as is clear and logical given the context. When "at least" is present before a series or range, it shall be understood that "at least" can modify each of the numbers in the series or range.

[0032] As used herein, "not more than" or "less than" shall be understood to mean the value immediately adjacent to these phrases and any logically lower value or integer up to zero (where negative values ​​are not possible) as is logical under the circumstances. When "not more than" is present after a series or range, it shall be understood that "not more than" can modify each of the numbers in the series or range.

[0033] As used herein, "up to," as in "up to 10," shall be understood to mean values ​​up to and including 10 as a maximum value, i.e., 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 in the context of non-negative integers. When a range of values ​​is provided, it is understood that each intervening value between the upper and lower limit of that range (e.g., to one-tenth of the unit of the lower limit, unless the context clearly dictates otherwise), and any other stated or intervening value within that stated range, is encompassed within the scope of the invention. The upper and lower limits of these smaller ranges may be independently included in the smaller ranges, subject to any specific exclusion of any limit within that range, and are likewise encompassed within the scope of the invention. Where a stated range includes one or both of the limits, ranges excluding either or both of those included limits are also included within the scope of the invention.

[0034] "Patient" refers to a subject exhibiting symptoms and / or complications of a disease or condition (e.g., cancer), a subject under the care of a clinician (e.g., an oncologist), a subject diagnosed with a disease or condition, and / or a subject at risk for developing a disease or condition. The term "patient" includes human and veterinary subjects. Any reference to a subject in this disclosure shall be understood to include the possibility that the subject is a "patient" unless the context clearly dictates otherwise.

[0035] As used herein, "predict" or "predicting" means determining the likelihood of a disease, condition, or event (e.g., death in a cancer subject) occurring in the future. In some embodiments, the model (e.g., DL model) can predict the likelihood of survival by one or more of the following measures of test accuracy: an odds ratio of greater than 1, preferably about 2 or more or about 0.5 or less, about 3 or more or about 0.33 or less, about 4 or more or about 0.25 or less, about 5 or more or about 0.2 or less, or about 10 or more or about 0.1 or less; a specificity of greater than 0.2, preferably at least about 0.3, at least about 0.4, at least about 0.5, at least about 0.6, at least about 0.7, at least about 0.8, at least about 0.9, or at least about 0.95, with a corresponding sensitivity of greater than 0.2, preferably at least about 0.3, at least about 0.4, at least about 0.5, at least about 0.6, at least about 0.7, at least about 0.8, at least about 0.9, or at least about 0.95; a sensitivity of at least 0.5, preferably at least about 0.6, at least about 0.7, at least about 0.8, at least about 0.9, or at least about 0.95, with a corresponding sensitivity of at least 0.2, preferably at least about 0.3, at least about 0.4, at least about 0.5, at least about 0.6, at least about 0.7, at least about 0.8, at least about 0.9, or at least about 0.95; a combination of a sensitivity of at least about 75% and a specificity of at least about 75%; a positive likelihood ratio (calculated as sensitivity / (1-specificity)) of greater than 1, preferably at least about 2, at least about 3, at least about 4, at least about 5, at least about 10; or A negative likelihood ratio [calculated as (1-sensitivity) / specificity] that is less than 1, preferably about 0.5 or less, about 0.33 or less, about 0.25 or less, or about 0.1 or less.

[0036] As used herein, "risk score" refers to the likelihood that a certain future event, e.g., disease, recurrence, will occur. In some embodiments, the risk score represents the likelihood that a patient will experience a recurrence following treatment. In some embodiments, the risk score is expressed as a classification. In other embodiments, the risk score is expressed as a continuum. In one embodiment, the risk score represents overall survival, i.e., the time elapsed from a subject's cancer diagnosis to death.

[0037] As used herein, a "subject" is an animal, such as a mammal, including primates (such as humans, monkeys, and chimpanzees), that will benefit from the methods of the present disclosure. In some aspects of the present invention, the subject is a human, such as a human diagnosed with cancer. The subject may be a female human. The subject may be a male human. In some aspects, the subject is an adult subject.

[0038] I. Overview of the Invention Adjustment for covariates (e.g., prognostic covariates) allows for improved precision and increased statistical power for treatment effect estimates in studies, e.g., randomized controlled trials (RCTs). However, sample size calculation under covariate adjustment generally requires knowledge of the magnitude of the adjusted covariate, which is usually not available at the beginning of a trial, especially with new endpoints and indications. It is therefore common to start with a preliminary sample size calculated based on sparse information available during the planning phase, and to reestimate the value of the adjusted covariate (and thus the sample size) once a portion of the planned number of patients has been studied (Friede and Kieser, 2011, Pharmaceut. Statist. 10:8-13). In this regard, it is important that sample size reestimation is performed such that all study participants remain blinded to treatment group assignment and that processing procedures do not inflate the Type I error rate.

[0039] Induced sample size reduction, driven by a reduction in the number of events to obtain a target statistical power, can be estimated based on known correlations between the adjusting covariates and the outcome when the outcome is continuous. In contrast, for time-to-event outcomes, e.g., survival data, there is no unique definition for the proportion of variation explained by the covariate. There is no clear way to account for the degree of association between the covariate and the outcome when calculating the sample size before the start of the trial (e.g., pretrial sample size design) or when a portion of the data becomes available during the trial (e.g., blinded sample size re-estimation).

[0040] Thus, the present disclosure provides methods for designing (e.g., reducing) sample sizes in clinical trials based on known correlations between adjustment covariates and outcomes in both continuous and time-to-event settings. Additionally, the present disclosure provides methods for, e.g., performing blinded sample size re-estimation, to re-estimate and re-adjust sample sizes while maintaining the blinding of the clinical trial based on data, including adjustment covariates and outcome data, that become available during the trial in both continuous and time-to-event settings. These methods are outlined, e.g., in Figures 1-3.

[0041] Specifically, the parametric simulations of the present disclosure stipulate that predictive performance in clinical trials (C-index) and outcome incidence are the two main determinants of sample size reduction. Based on the analysis of several generalized proposals of the percentage of variance explained value, the present disclosure also stipulates that the Cox·Snell R 2 CS reveals that the Cox Snell R 2 CS The present disclosure also provides a method for designing clinical trials by calculating sample size using the Cox·Snell R function for time-to-event outcomes in adjusted covariate analyses. 2 CS We provide a method to blindly re-estimate the sample size using (Figure 3).

[0042] The methods disclosed herein can incorporate new prognostic sources, such as prognostic scores obtained by deep learning, as adjustment covariates to calculate adjusted (e.g., reduced) sample sizes. The present disclosure provides evidence using publicly available clinical trial information for hepatocellular carcinoma and mesothelioma that such adjustment covariate methods using deep learning prognostic covariates can improve the statistical power obtained based on the same sample size, and can reduce the sample size required to obtain the same statistical power.

[0043] Further provided herein is a method to obtain target statistical power with less stringent eligibility criteria in clinical trials by using adjustment covariates that can compensate for the high heterogeneity associated with less restrictive inclusion criteria. Thus, systematic adjustment for prognostic covariates according to the present disclosure can lead to reduced sample sizes required for target statistical power, as well as efficient and more inclusive clinical trials (e.g., relaxed eligibility criteria), particularly in disease settings with high outcome incidence, such as metastatic cancer.

[0044] In one embodiment, reducing the sample size used in a randomized trial (or re-estimating the sample size in a blinded manner) improves the functionality of the device used to accumulate the results of the randomized trial. In this embodiment, reducing the sample size of a randomized trial means that fewer participants are needed to obtain the same statistical power in a randomized trial. In addition, by having fewer participants, the randomized trial accumulates less data. This accumulation of less data improves the efficiency of the device that will accumulate, analyze, store, and / or otherwise process these data, since there is less data resulting from a randomized trial with a small sample size. This improves the efficiency of the device by reducing the computational and storage load on the device.

[0045] II. Parametric Simulations for Covariate Adjustment and Sample Size Reduction The Fleiss formula below describes the number of events (N) required for a given statistical power in an unadjusted analysis for the correlation (denoted as γ) between the outcome and the adjusted covariates in a continuous outcome setting. original ) and the number of events required for a given statistical power in the adjusted analysis (N adjusted This represents the relationship between the N adjusted =N original (1-R 2 )

[0046] Parametric simulations were performed to evaluate the observed sample size reduction R in the continuous outcome setting as well as the time-to-event outcome setting. 2 obsA simulation is performed to calculate the C-index of a single adjusted covariate as a function of the outcome incidence. As used herein, "C-index" is a measure of the predictive ability of the model. The C-index is controlled by the covariate coefficients. As used herein, "outcome incidence" means (number of events / sample size). In the case of survival data, the outcome incidence can be estimated using a Kaplan-Meier curve. Other parameters of interest are the size of the treatment effect, the Weibull shape of the baseline hazard function, and the dropout rate. These simulations are performed under the proportional hazards assumption as in the Cox model.

[0047] Survival times are generated according to a Weibull distribution with shape w and scale that depend on the treatment hazard ratio hr and standard Gaussian covariates x. Censoring times are delineated from an exponential distribution with a specified dropout rate d. Denoting the treatment assignment variable by z, this generative model can be summarized as a formula for patient i as follows: JPEG2025512702000005.jpg17157

[0048] All patients remaining at risk at 5 years are censored at this point. Treatment assignment is independent of covariates and the arms of the trial are balanced. For each input parameter set, the auxiliary parameters are numerically optimized to reach the target outcome incidence λ at 5 years in the placebo arm, and κ and θ are numerically optimized, and the C-index C is evaluated for the entire trial population.

[0049] Figure 4 shows the R 2 obsThe figure illustrates the process for the calculation of . Once survival times are simulated, the statistical power for the unadjusted model and the model with the X-adjusting covariates added is estimated as the percentage of the treatment effect detected with 10e+3 resamples on a grid of sample sizes (Heo et al., 1998, Mech Ageing Dev 102:45-53). The resulting statistical power curves are the number of events N needed to reach 80% statistical power for both models. adjusted and N original From there, R 2 obs These simulations explore a wide range of parameter values, shown in Table 1, and estimate R as a function of λ and C for various settings of proportional hazards. 2 obs Allows for extensive behavioral testing.

[0050] (Table 1) Table 1. Description of simulation parameters JPEG2025512702000006.jpg48153

[0051] Parametric simulations show that the sample size reduction obtained by covariate adjustment improves with both the C-index and outcome incidence of the model. Figures 5A-5C show the R 2 obs(Figures 5A and 5C: hr = 0.7, Figure 5B: hr ≈ 0.4). The impact of outcome incidence increases with C-index. For example, comparing outcome incidence of λ = 10% to outcome incidence of 90% in Figure 5C, which is the same as the upper left panel of Figure 5A (hr = 0.7, w = 0.5, d = 0.01), a C-index of 0.55 results in a difference of 5 percentage points in sample size reduction, and a C-index of 0.85 results in a difference of 60 percentage points in sample size reduction. Furthermore, Figures 5A-5B show that the relationship of sample size reduction to C-index and outcome incidence does not depend on the treatment effect, Weibull shape parameter, or dropout rate. The values ​​chosen for dropout rate, d=0.01 or d=0.1, yielded a median of censored patients before end of follow-up of 3.7% (range 0.7%-4.8%) and 28.1% (range 5.4%-38.4%), respectively.

[0052] The parametric simulations provided herein demonstrate that the main determinants of sample size reduction required for a given statistical power are the C-index of the adjusting covariates and the outcome incidence in the clinical trial. Other parameters such as Weibull shape, dropout rate, or effect size do not affect the increased precision achieved by adjusting covariates (Hernandez et al., 2006, Ann Epidemiol 16:41-48). The present disclosure reveals the strong dependence of sample size reduction on outcome incidence in a simulation setting with a finite time horizon (i.e., follow-up is discontinued at 5 years).

[0053] Identification of key parameters (e.g., outcome incidence, C-index) that determine sample size reduction by covariate adjustment can help prioritize indications to focus on with covariate adjustment. For example, in diseases with high prognostic outcome incidence, e.g., metastatic cancer, aggressive cancers, e.g., mesothelioma (Kerr et al., 2017, Clin Trials 14:629-638), covariate adjustment becomes more influential and every additional point of C-index is a significant increase in precision. In diseases with low prognostic outcome incidence, e.g., in secondary prevention of cardiovascular disease, prognostic enrichment can be performed using prognostic signatures in addition to or instead of covariate adjustment.

[0054] III. Sample size design by adjusting for covariates based on time-to-event data Covariate adjustment increases statistical power and leads to more accurate estimation of treatment effects. FIG. 1 shows an illustrative flow diagram for designing a sample size for a clinical trial, such as a randomized controlled clinical trial (RCT) according to the method of the present disclosure. Process 100 begins with selecting an adjustment covariate at block 101. As used herein, a "adjustment covariate" can be any variable that is correlated with an outcome other than the treatment assignment being tested in the clinical trial. The adjustment covariate can be associated with a continuous outcome. Alternatively, the adjustment covariate can be associated with a discrete (e.g., time to event) outcome. Sample size calculation under covariate adjustment can be performed based on the magnitude of the adjustment covariate with respect to the outcome. Thus, at block 103, process 100 checks whether the magnitude of the correlation between the adjustment covariate and the outcome is known. If this determination is true, process 100 proceeds to block 105, where the sample size is designed based on the known correlation. Block 105 is described in further detail in Figure 2. Based on the sample size designed in block 105, process 100 proceeds to initiating the clinical trial in block 109.

[0055] FIG. 2 shows an illustrative flow diagram for designing a clinical trial sample size based on known correlations between adjusting covariates and outcomes. Block 105 is depicted as process 200. Process 200 examines whether the outcome associated with the adjusting covariate in 201 is a continuous outcome or a time-to-event outcome. For continuous outcomes, the number of events required for a given statistical power (N) for an unadjusted analysis for the correlation between the outcome and the adjusting covariate (denoted as R) is determined. original ) and the number of events required for a given statistical power in the adjusted analysis (N adjusted ) there is the following equation that connects N adjusted =N original (1-R 2 )

[0056] For example, a correlation of 0.5 between a baseline covariate and an outcome translates into a 25% reduced sample size requirement in the adjusted analysis compared to the unadjusted analysis. This formula is sometimes called the Fleiss formula. The process 200 is 2 and obtaining the number of events N based on the above Fleiss formula in 205. adjusted This is followed by the design stage.

[0057] In contrast, there has been no unique definition for the proportion of variation explained by a covariate when the outcome associated with the adjusted covariate in 201 is a time-to-event outcome, e.g., survival data. As provided herein in parametric simulations, the observed sample size reduction (R 2 obs ) improves with increasing predictive power of the prognostic variable (C-index) and / or increasing outcome incidence (λ), with the effect of λ being higher for intermediate C-index values ​​(e.g., between 0.7 and 0.8) than for other ranges of C-index values. However, there has been no clear method to account for the association between covariates and outcomes when calculating sample size in the time-to-event outcome setting. The present disclosure provides a novel method for calculating the Cox·Snell R 2 CSIn other words, when the outcome associated with the adjusted covariates in 201 was a time-to-event outcome, the process 200 calculates the R 2 CS Calculate R using the Fleiss formula in 209. 2 Instead of R 2 CS Using the number of events N adjusted This is followed by the design stage.

[0058] In arriving at the methods described herein, such as those illustrated in FIG. 2 Different ways of generalizing Fleiss's formula are implemented and parametric simulations are used to investigate which measures could be considered to generalize Fleiss' formula for time-to-event settings over a finite time horizon.

[0059] Conventional R 2 Among other proposed categories of measures that generalize to time-to-event data, we considered the Explained Variance (EV) and Explained Randomness (ER) measures. The Explained Variance (EV) measures are an extension of the traditional measures of the proportion of variance explained by a set of covariates used in linear regression. The Explained Randomness (ER) measures, on the other hand, are based on the concept of entropy, which compares the amount of information contained in a model with and without the covariate of interest. In this disclosure, we consider three EV measures [R 2 D (Royston and Sauerbrei, 2004, Stat Med 23:723-748), R 2 PM (Kent and O'Quigley, 1988, Biometrika 75:525-534), R 2 R (Royston, 2006, Stata J Promot Commun Stat Stata 6:83~96)] and five ER scales [R 2 i(Royston and Sauerbrei, 2004, Stat Med 23:723-748), ρ 2 k (OQuigley et al., 2005, Stat Med 24:479~489), ρ 2 WA (Royston and Sauerbrei, 2004, Stat Med 23:723-748), ρ 2 XO (Ronghui and O'Quigley, 1999, J Nonparametric Stat 12:83-107), and R 2 CS (Cox et al., 1989, Analysis of binary data. 2.ed., 1. CRC Press reprint. Boca Raton, Fla., Chapman & Hall)]. 2 The "Scale Proposal" is 2 D , R 2 PM , and R 2 i And, R 2 i , ρ 2 k , ρ 2 WA , ρ 2 XO , and R 2 CS Surveyed prospects including R 2 For example, if the log-likelihood of the basic model and the model adjusted for auxiliary covariates are denoted as l0 and l1, the following formula is given: JPEG2025512702000007.jpg13150

[0060] 6A to 6C show all R 2 Collect the scale values ​​and use them for each Weibull shape in R 2 obs What stands out is that only two of the eight measures actually show increasing values ​​with the incidence of the outcome (R 2 CS and ρ2 XO ) while the others only increase with C-index value and remain generally constant with respect to outcome incidence, resulting in steps. The occurrence of these steps is due to the fact that most of the measures were developed to be robust to censoring directly related to outcome incidence in a finite time horizon setting. Cox·Snell R 2 CS is another R 2 It outperforms the proposed scale and most accurately captures the sample size reduction observed in all our simulations, and R 2 CS The median absolute error is 2.1% (1st and 3rd quartiles are 0.8% and 3.9%, respectively).

[0061] Therefore, we use the Cox Snell R metric, which generalizes the Fleiss formula to discrete data (e.g., time to event) settings. 2 CS Such a metric is of practical importance since it can help design clinical trials without the aid of simulation. Herein, we provide a method for designing randomized controlled trials (RCTs), which can be used to measure the N original is the number of events required to obtain statistical power without covariate adjustment, and N adjusted is the number of events required to obtain statistical power using covariate adjustment, N adjusted =N original (1-R 2 CS ) In some embodiments, the methods of the present disclosure allow for designing clinical trials without the use of simulation. An illustrative method is depicted in FIG. 2 at 201, 207, and 209. In some embodiments, the RCT is based on event count calculations.

[0062] IV. Blinded Sample Size Reestimation for Covariate Adjustment Based on Time-to-Event Data The metrics and methods of the present disclosure can also be useful in the case of blinded sample size re-estimation during a clinical trial when uncertainty arises in the predictive performance of the adjusting covariate at the start of the trial. Generally, sample size calculation under covariate adjustment requires knowledge of the magnitude of the correlation between the adjusting covariate and the clinical outcome. However, when planning the sample size of a clinical trial, it may be difficult to obtain a good estimate of this correlation from previous trials, especially when new endpoints and / or indications are used. An exemplary workflow of blinded sample size re-estimation during a clinical trial, such as an RCT, is depicted in FIG. 1. In block 103, the process 100 checks whether the magnitude of the correlation between the adjusting covariate and the outcome is known. If this determination is false, the process 100 proceeds to block 107, where it sets a preliminary sample size based on known information in the absence of correlation information. In some embodiments, a provisional sample size is calculated based on an initial guess of the value of the covariate. Based on the sample size designed based on the preliminary sample size set in block 107 without any correlation information, process 100 continues to block 111 to start the clinical trial.

[0063] As the trial progresses, data is obtained from subjects in a blinded manner, i.e., without the investigator knowing whether the subject is in the treatment or control group. Process 100 continues to block 113, where observations during the trial are obtained in a blinded manner. When observations are available for a pre-specified portion of this sample size, the correlations between the adjusted covariates and the outcome are re-estimated and, if necessary, the sample size is adapted accordingly. From a regulatory perspective, it is important that the sample resizing is performed so that all involved in the trial remain blinded to the treatment group assignment and that the procedure does not inflate the Type I error rate. At 115, process 100 performs a blinded sample size re-estimation based on the observations, including the adjusted covariates and outcome data obtained during the ongoing trial. Block 115 is described in further detail in FIG. 3.

[0064] 3 shows an illustrative flow diagram for blinded sample size re-estimation, for example, during a clinical trial. Block 113 is depicted as process 300. Process 300 begins by examining whether the outcome associated with the adjusting covariate at 301 is a continuous outcome or a time-to-event outcome. In a continuous outcome setting, process 300 continues to block 303, where the correlation (R 2 ) is obtained based on data obtained during the clinical trial. Process 300 continues to block 305, where R 2 The sample size is re-estimated using the method described above. Sample size can be re-estimated according to certain procedures published by, for example, Friede and Kieser, 2011, Pharmaceut. Statist. 10:8-13 and Zimmermann et al., 2020, Univ. Kentucky, Statistics Faculty Publications 28, for blinded sample size re-estimation, the entire contents of each of which are incorporated herein by reference.

[0065] For example, N original is the original number of events required to obtain statistical power without covariate adjustment, and N adjusted where R is the re-estimated number of events required to obtain statistical power and R is the correlation between the outcome and the adjusted covariates based on blinded RCT data. N adjustedl =N original (1-R 2 ) The re-estimated number of events can be obtained by the formula: RCT can be further performed using blinded sample size re-estimation. In some embodiments, N original teeth, N original is the estimated number of events required to obtain statistical power based on the Schoenfeld formula (Schoenfeld, 1983, Biometrics, 39:2:499-503), α is the Type I error level, β is the Type II error level, P1 and P2 are the proportions of the trial sample included in the treatment and control groups, respectively (e.g., both equal 1 / 2 in the case of balanced treatment allocation), hr is the specified hazard ratio, z p When is the pth quantile of the standard normal distribution, It is evaluated based on the formula JPEG2025512702000008.jpg9103.

[0066] Additionally or alternatively, the adjusted number of events (N A (hereinafter written as N A and N GS are the re-estimated number of events required to obtain statistical power, respectively, and α is the Type I error level, β is the Type II error level, γ=n2 / n1 is the allocation ratio, n1 and n2 are the sample sizes of groups 1 and 2, respectively; δ 2 Y is the variance of the outcome, Δ is the specified difference in adjusted means, z p When is the pth quantile of the standard normal distribution, It can be obtained using the basic approximation formula: JPEG2025512702000009.jpg1484.

[0067] Alternatively or additionally, the adjusted number of events (N GS ), i.e. JPEG2025512702000010.jpg1543 or the adjusted number of events (N DF ), i.e. JPEG2025512702000011.jpg1344 or the adjusted number of events (N GS,DF ), i.e. You can get JPEG2025512702000012.jpg1547.

[0068] In contrast, there has previously been no unique definition for the proportion of variance explained by a covariate when the outcome associated with the adjusted covariate is a time-to-event outcome, e.g., survival data. There has previously been no clear method for blinded sample size re-estimation when uncertainty arises about the magnitude of association between a prognostic covariate and the outcome in the time-to-event outcome setting. The present disclosure provides a method for the blinded sample size re-estimation of a covariate that is based on the Cox·Snell R 2 CS That is, if the outcome associated with the adjustment covariate in block 301 is not a continuous outcome, process 300 provides a new method for performing blinded sample size re-estimation using R 2 CS This is followed by the step of calculating R 2 CS teeth, R 2 CS However, Cox·Snell R 2 and n is the number of participants, l0 is the log-likelihood of the model without adding adjustment covariates, where l1 is the log likelihood of the model including adjustment covariates, This can be calculated as provisional data based on the formula JPEG2025512702000013.jpg13150.

[0069] Process 300 continues to block 309, where R 2 CS The blinded sample size re-estimation is performed using the number of events, N adjustedl is the number of re-estimated events required to obtain statistical power, Nadjustedl =N original (1-R 2 CS ) RCTs can be further performed using blinded sample size reestimation.

[0070] In some embodiments, N original teeth, N original is the estimated number of events required to obtain statistical power based on the Schoenfeld formula, α is the Type I error level, β is the Type II error level, P1 and P2 are the proportions of the trial sample included in the treatment and control groups, respectively (e.g., both equal 1 / 2 in the case of balanced treatment allocation), hr is the specified hazard ratio, z p When is the pth quantile of the standard normal distribution, It can be evaluated based on the formula JPEG2025512702000014.jpg999.

[0071] In block 117, process 100 adjusts the sample size based on the results of the blinded sample size re-estimation in block 115. Process 100 then continues the trial in block 119 based on the adjusted sample size in block 117.

[0072] V. Covariate adjustment based on deep learning covariates Provided herein is a method for improving the statistical power of a study, e.g., a clinical trial, and / or reducing the number of events required for a target statistical power by incorporating new prognostic sources obtained using a deep learning prognostic model as adjustment covariates. The deep learning prognostic model can be based on histopathological slides of cancer patients. In some embodiments, the covariate adjustment is performed based on deep learning covariates. Histological slides have already been used to obtain the covariates used for adjustment, which are obtained by anatomic pathologists to determine histological subtypes (e.g., MAPS trial). By using deep learning techniques, in addition to the previous clinical evaluation of subtypes, the quantified slides are automatically processed using the deep learning model.

[0073] Prognosis prediction based on deep learning model Provided herein is a method for predicting patient outcomes based on deep learning of histological slides for use in the context of covariate adjustment. Deep learning-based survival prediction for histological slides can be performed using deep learning models described, for example, in Saillard et al., 2020, Hepatology 72:6;2000-2013 (HCCnet) and Courtiol et al., 2019, Nat Medicine 25:10;1519-1525 (MesoNet), the entire contents of each of which are incorporated herein by reference.

[0074] Histology is the field of study of the microscopic characteristics of biological samples. Histopathology refers to the microscopic examination of a sample, e.g., tissue, obtained or otherwise harvested from a subject, e.g., a patient, to assess a disease state. Generally, a histopathology sample results from processing the sample, e.g., tissue, in a manner that fixes the sample, or a portion thereof, to a microscope slide. For example, a microtome or other suitable device can be used to obtain thin sections of the tissue sample, and these thin sections can be fixed to a slide. To aid in visualization of the sample, the sample can optionally be further processed, e.g., by adding a dye. Many dyes have been developed to visualize cells and tissues. These dyes include, but are not limited to, hematoxylin and eosin (H&E), methylene blue, Masson's trichrome, Congo red, Oil Red O, and safranin. H&E is routinely used by pathologists to aid in visualizing cells within a tissue sample. Hematoxylin stains the nuclei of cells blue, and eosin stains the cytoplasm and extracellular matrix pink. A pathologist visually inspecting an H&E stained slide can use this information to evaluate the morphological characteristics of the tissue. However, H&E stained slides typically contain insufficient information to evaluate the presence or absence of a particular biomarker by visual inspection. Visualization of a particular biomarker (e.g., protein or RNA biomarker) can be achieved using additional staining techniques, such as immunofluorescence, immunohistochemistry, in situ hybridization, etc., that rely on labeled detection reagents that specifically bind to the marker of interest. Such techniques are useful for determining the expression of individual genes or proteins, but are impractical for evaluating complex expression patterns involving multiple biomarkers. Global expression profiling can be achieved by genomic and proteomic methods using separate samples taken from the same tissue source as the sample used for histopathological analysis.Nevertheless, such methods are costly and time-consuming, requiring the use of specialized equipment and reagents, and do not provide any information correlating biomarker expression to specific regions within a tissue sample, e.g., specific regions within an H&E stained image.

[0075] As used herein, the term "digital image" refers to an electronic image represented by a set of pixels that can be displayed, processed, and / or analyzed by a computer. In some aspects of the present disclosure, a digital image of a histology slide, e.g., an H&E stained slide, allows for computerized evaluation of a tissue sample in addition to or instead of visual inspection by a pathologist. In some embodiments, the digital image can be acquired using a digital camera or other optical device capable of capturing a digital image from a slide or a portion thereof. In other embodiments, the digital image can be acquired by scanning a non-electronic image of a slide or a portion thereof. In some embodiments, the digital image used in the applications provided herein is a whole slide image. As used herein, "whole slide image (WSI)" refers to an image that includes all or nearly all of a portion of a tissue section, e.g., a tissue section present on a histology slide. In some embodiments, a WSI includes an image of the entire slide. In other embodiments, the digital image used in the applications provided herein is a selected portion of a tissue section, e.g., a tissue section present on a histology slide. In some embodiments, the digital image is acquired after the tissue section has been treated with a stain, e.g., H&E.

[0076] In some aspects, a computer-implemented method for predicting the likelihood of a certain prognosis, e.g., recurrence, overall survival, or disease-free survival, comprises: accessing a digital histology image of a histology section obtained from the subject; extracting a plurality of feature vectors of the histology image by applying a first convolutional neural network, each of the features representing a local descriptor of the histology image; classifying the histology image using at least the plurality of feature vectors and a classification model trained using a training set of known histology images and known prognostic information; determining a prognostic likelihood for the subject based on at least the classification of the histology image; Includes.

[0077] In some aspects, a computer-implemented method for predicting the likelihood of a certain prognosis, e.g., recurrence, overall survival, or disease-free survival, comprises: obtaining a digital image of a histological section from a subject; Segmenting the digital image into a set of tiles; extracting a plurality of feature vectors from the tile set or a subset thereof; calculating an artificial intelligence (AI) risk score based on the histological image using a machine learning model trained by processing a plurality of training images to predict the likelihood of a certain prognosis, such as the likelihood of recurrence, overall survival, or disease-free survival; and the AI ​​risk score represents the prognostic likelihood of the subject.

[0078] In some embodiments, the method further comprises: obtaining subject-derived clinical attributes; calculating a clinical risk score using a clinical model trained using one or more subject attributes; calculating, for the subject, a final risk score representing the subject's prognostic likelihood from the machine learning risk score and the clinical risk score; It may further include.

[0079] In some embodiments, the digital image of the method is a whole slide image (WSI). In some embodiments, the histology section of the disease (e.g., cancer) sample has been stained with a dye, such as hematoxylin and eosin (H&E), to visualize the underlying tissue structure. Other common dyes that can be used to visualize tissue structures in the input image include, for example, Masson's trichrome dye, periodic acid Schiff dye, Prussian blue dye, Gomori trichrome dye, Alcian blue dye, or Ziehl-Nielsen dye.

[0080] The machine learning model can be a self-supervised machine learning model by using tissue structure data (such as whole slide images) to extract features from tiles of whole slide images. In one embodiment, the feature extractor is trained on in-domain tissue structure tiles without annotation. In one embodiment, to apply the self-supervised framework to tissue structure data, tiles from all WSIs extracted in the tile splitting stage are concatenated to form a training dataset. A feature extractor is then trained on this set of unlabeled tile images using MoCo v2. The tile set can first be split into two batches of tiles. The first and second batches of tiles can be modified by, for example, adding a 90° rotation and a vertical flip, and further performing color enhancement. Since tissue structure tiles contain the same information regardless of their orientation, rotation is a reasonable enhancement to perform. Since tissue structure tiles are images containing cells or tissues and are orientation independent, such tiles can be displayed properly regardless of rotation, horizontal flip, etc. Thus, rotating the image provides a beneficial enhancement without losing any important characteristics of the image. By applying a batch of tiles to its respective feature extractor, a tile embedding representation can be generated.

[0081] In one embodiment, the tile embedding representation is the output of the feature extractor and serves as a signature for each tile and contains semantic information about each tile, in other words, the tile embedding representation is a representation of the tile including the semantic information for that tile.

[0082] In some embodiments, the self-supervised learning algorithm uses a contrastive loss to shape the tiled embedding representations such that different augmented views of the same image have close or similar tiled embedding representations to each other. In other words, the contrastive loss can compare two tiled embedding representations, and based on this comparison, the first feature extractor can be adjusted so that the tiled embedding representation of the first feature extractor is similar to the tiled embedding representation of the second feature extractor. A gradient is back-propagated through the first feature extractor. In some embodiments, the weights of the second feature extractor are updated using an exponentially smoothed moving average (EMA) of the weights of the first extractor. In some embodiments, the use of EMA can avoid overfitting. Thus, the output of the system is a trained feature extractor that has been trained with intra-domain organizational structure tiles such that the tiled embedding representations of different augmentations of the same image are similar. This type of specially trained feature extractor can provide significant improvements in downstream performance, as described below. In some embodiments, the trained feature extractor can be achieved after training for a certain number of learning rounds. In some embodiments, training is performed until the accuracy is at or near 1 (or 100%), the AUC is at or near 1 (or 100%), or the loss is near zero. In some embodiments, during training of the feature extractor, many useful metrics are not accessible. Therefore, one of the available metrics of the downstream task, such as AUC, can be monitored to see how well the feature extractor is performing. In one example, the trained feature extractor can be used to train a weakly supervised task to evaluate the performance. Additional training can be justified if it is believed that additional training can result in improved downstream performance.

[0083] In some embodiments, the second feature extractor may be optional, and a single feature extractor may be used to generate tiled embedding representations from the two batches of tiles. In such embodiments, a feature extractor is used to generate tiled embedding representations from the two batches of tiles, the two tiled embedding representations are compared using a contrastive loss as described above, and the first feature extractor is adjusted so that the first tiled embedding representation is similar to the second tiled embedding representation.

[0084] In some embodiments, the machine learning algorithm extracts a plurality of feature vectors from the digital image, and the step of extracting the plurality of feature vectors is performed using a first convolutional neural network, for example a ResNet50 neural network.

[0085] In some embodiments, the computer-implemented method further comprises removing background from the image. In some embodiments, removing background regions from the image is performed using a second convolutional neural network. In some embodiments, the second convolutional neural network is a semantic segmentation deep learning network.

[0086] In some embodiments, the computer-implemented method further comprises selecting a subset of tiles suitable for application to the machine learning model, hi some embodiments, the subset of tiles is selected by random sampling.

[0087] In some embodiments, the machine learning model is trained using a plurality of training images, the plurality of training images including digital images of histology sections of samples of a disease, e.g., cancer, taken from a plurality of control subjects having the disease, e.g., cancer. In some embodiments, the plurality of training images includes images lacking local annotations. In some further embodiments, the plurality of training images includes images associated with one or more global labels indicative of one or more disease characteristics of the control patients from whom the samples were taken.

[0088] In one embodiment of self-supervised learning on tissue structure images to train a feature extractor, the process begins with accepting a set of tissue structure images. In some embodiments, each image in the training image set is an annotation-free whole slide image. The process then continues with tiling and expanding the training image set into a tile set. In one embodiment, the digital image can be divided into a tile set. Tiling the image can include dividing the original image into smaller images, called tiles, that are easier to manage. In one embodiment, the tile division operation is performed by applying a fixed grid to the whole slide image and using a segmentation mask generated by the segmentation method to select tiles that contain tissue or any other region of interest. To further reduce the number of tiles to be processed, in one embodiment, additional or alternative selection methods can be used, such as random subsampling to keep only a given number of slides.

[0089] In one embodiment, the augmentation can be applied to each of the tile sets. The process continues with generating a processed tile set by performing the following operations on each tile batch selected from the tile set: A first feature set is extracted from the first augmented tile batch, and a second feature set is extracted from the second augmented tile batch. In some embodiments, the augmented tiles include zoomed-in views, rotated views, or views provided with color augmentation. For example, for histology slides, orientation is not important, so the slides can be rotated to various angles. The slides can also be enlarged or zoomed in. The process then uses a contrast loss between the first and second sets of extracted feature pairs to bring matching tiles closer and dissimilar tiles farther apart. The contrast loss is applied to focus on positive pairs rather than negative pairs obtained from the first and second feature sets.

[0090] The process continues with training a feature extractor using the processed tile set generated by the operation of the above embodiment. In some embodiments, the trained feature extractor disclosed herein can be used to improve classification of tissue structure images. The process continues with outputting a trained feature extractor trained using a self-supervised ML algorithm. In some embodiments, the feature extractor can be trained for a certain number of learning times such that the training images are seen a certain number of times.

[0091] A trained machine learning model can be used to calculate the risk score. The process begins with accepting an input histology image. In some embodiments, the input histology image is a WSI, which may be derived from a patient tissue sample. In some embodiments, the patient tissue sample is known or suspected to contain a tumor.

[0092] In some embodiments, the process includes removing background regions from the input image. In some embodiments, object detection can be used to recruit only tiles that originate from tissue regions of the input image. In some embodiments, the background can be removed using Otsu's method applied to the hue and saturation channels after conversion of the input image to a hue, saturation, value (HSV) color space.

[0093] The process continues with tiling the histology image into a set of tiles. In one embodiment, the process enhances the ability to pre-process the image using tiling. For example, in one embodiment, using tiling is useful for histopathology analysis due to the large size of whole slide images. More broadly, when dealing with specialized images such as histopathology slides, satellite imagery, or other types of large images, the amount of random access memory attached to the image sensors used in these fields increases, but the resolution of the sensor can also increase just as quickly. With these large image sizes, it is difficult to store image batches, or even single images, inside the random access memory of a computer. This difficulty is exacerbated when trying to store these large images in the dedicated memory of a graphics processing unit (GPU). This situation makes it difficult for a computer to process image slides or any other images of similar size as a whole.

[0094] In one embodiment, tiling the image (or image minus background) addresses the above problem by dividing the original image (or image minus background) into smaller images called tiles that are easier to manage. In one embodiment, the tiling operation is performed by applying a fixed grid to the entire slide image and using a segmentation mask generated by the segmentation method to select tiles containing tissue or any other area of ​​interest for the subsequent classification process. As used herein, it is understood that an "area of ​​interest" of an image can be any area that is semantically associated with the task to be performed, particularly in the context of histopathology, areas corresponding to tissues, organs, bones, cells, body fluids, etc. To further reduce the number of tiles to be processed, additional or alternative selection methods can be used, such as random subsampling to keep only a given number of slides.

[0095] For example, in one embodiment, the process divides the image (or image minus background) into tiles of a fixed size (e.g., each tile has a size of 774x774 pixels). Alternatively, the tile size can be smaller or larger. In this example, the number of tiles generated depends on the size of the detected object and can vary from a few hundred to 50,000 or more. In one embodiment, the number of tiles is limited to a fixed number (e.g., 10,000), which can be set based at least on computation time and memory requirements.

[0096] For each tile, the process continues with extracting one or more features of that tile. In one embodiment, each of the features is extracted by applying a trained feature extractor trained with a contrastive loss ML algorithm using a training image set. In one embodiment, the training image set is an annotation-free image set. In one embodiment, the input images and the training image set are from the same domain, i.e., the images are of the same category or type. For example, the input images and the training image set can both be histology images. This is in contrast to embodiments in which the training image set includes out-of-domain images or images that are not histology images or are not of the same category or type as the images being analyzed. In one embodiment, the contrastive loss ML algorithm is Momentum Contrast or Momentum Contrast v7 (MoCo v7). In some embodiments, the trained feature extractor is an ImageNet-type feature extractor. In one embodiment, the trained machine learning model is a machine learning model described herein.

[0097] In some embodiments, the machine learning model is a Deep Multiple Instance Learning model. In some embodiments, the machine learning model is a Weldon model. In some embodiments, the machine learning model is applied to the entire set of tiles. In some embodiments, the machine learning model is applied to a subset of tiles. The training images can include digital images of histological sections of disease (e.g., cancer) samples taken from several control subjects. In some cases, the training images lack local annotations. The training images can include images associated with one or more global labels indicative of one or more disease features of the control patients from whom the samples were taken. The disease features can include, in some embodiments, duration of survival, duration of disease-free survival, or duration of disease (e.g., cancer) recurrence. In some embodiments, the disease is cancer, the one or more disease features can include one or more of histological subtype, tumor stage, tumor size, number of positive lymph nodes, biomarker status, pathological stage, clinical stage, patient age, and / or treatment history, or a combination thereof. In some embodiments, one or more disease features of the patient can be obtained, and the machine learning model is applied to both the extracted features and the disease features of the patient. The example disease features for the patient may be the same as the disease features represented in the global labels associated with the training images.

[0098] The process then uses the machine learning model to calculate an AI risk score for the patient. The AI ​​risk score can represent a certain prognosis, such as overall survival, disease-free survival, early recurrence, or likelihood of recurrence after treatment, and can be expressed as a categorical or continuous range. In one embodiment, the AI ​​risk score represents the predicted time from diagnosis to death (overall survival) for the subject.

[0099] In some embodiments, a final risk score can be calculated as a weighted average of the AI ​​risk score and the clinical risk score, where the weights of these risk scores can be the same or different. Using the trained machine learning model and the trained clinical model, the process can accept the WSI and clinical attributes to determine a risk score for the subject. In one embodiment, the WSI is a digital image of the subject, and the one or more clinical attributes are attributes used as inputs to the clinical model. The process can then use the trained machine learning model to calculate an AI risk score, and the trained clinical model to calculate a clinical risk score.

[0100] In one embodiment, the final risk score is the average of the machine learning risk score and the clinical risk score. f is the final risk score, R m is the AI ​​risk score, and R c where is the machine clinical risk score, Alternatively, the final risk score can be a weighted average of the machine learning risk score and the clinical risk score. m and a m are R m and R c When the weight is TIFF2025512702000016.tif7150 holds. In yet another embodiment, the process may calculate the final risk score differently (e.g., as the square root of the sum of the squares of the two inputs or another function).

[0101] The final risk score can represent a certain prognosis, such as overall survival, disease-free survival, early recurrence, or likelihood of recurrence after treatment, and can be expressed as a categorical or continuous range. In one embodiment, the final risk score represents the predicted time from diagnosis to death (overall survival) for the subject.

[0102] Deep learning models can outperform traditional biomarkers or histological classifications traditionally used by pathologists in prognostic prediction and can provide improved C-indexes when predicting clinical outcomes such as overall survival (OS). For example, a deep learning model (HCCnet) was trained on 390 whole slide images (WSIs) from 194 patients. HCCnet was then tested on 342 WSIs from 328 patients from The Cancer Genome Atlas (TCGA). In this validation set, the deep learning model outperformed the composite score. HCCnet obtained a C-index of 0.70 when the composite score only had a C-index of 0.63. The composite score was based on American Joint Committee on Cancer (AJCC) stage, age at diagnosis, sex, serum AFP, alcohol consumption, HBV (Hepatitis B virus) infection or HCV infection, other etiology, unknown etiology, tumor differentiation according to World Health Organization (WHO) criteria, macrovascular and microvascular invasion, positive resection margins, and non-neoplastic liver fibrosis (cirrhosis).

[0103] VI. Semi-synthetic simulation of covariate adjustment for deep learning covariates based on clinical trial data A. Simulation of hepatocellular carcinoma (HCC) data using HCCnet Patients with early-stage hepatocellular carcinoma (HCC) are suitable for localized treatment (resection or local ablation). Despite poor survival outcomes in HCC, there is no adjuvant treatment after resection or local ablation. Sorafenib, the standard treatment for advanced HCC in the STORM trial, failed to demonstrate superiority over placebo in the adjuvant setting (Bruix et al. 2015, Lancet Oncol 2015;16:1344-1354). Currently, large pharmaceutical sponsored trials are investigating whether immunotherapy can improve outcomes for HCC patients after localized treatment (clinicaltrials.gov NCT03383458, NCT04102098, NCT03867084, NCT03847428). HCCnet, a deep learning model on histological slides, captures important prognostic signatures for overall survival for HCC after curative treatment (Saillard et al. 2020, Hepatology 72:6;2000-2013). We investigated the potential sample size reduction that could be achieved by adjusting for HCCnet in adjuvant trials in a semi-synthetic simulation based on the master external validation set from TCGA-HCC (Cancer Genome Atlas Research Network 2017 Cell 169:1327-1341).

[0104] Eligibility criteria in clinical trials are overly restrictive, leading to limited generalizability and difficulty in enrolling participants (Kim et al., 2017, J Clin Oncol Off J Am Soc Clin Oncol 35:3737-3744; FDA 2020 https: / / www.fda.gov / regulatory-information / search-fda-guidance-documents / enhancing-diversity-clinical-trial-populations-eligibility-criteria-enrollment-practices-and-trial). Although sometimes used to ensure patient safety, strict eligibility restrictions may also be used to ensure homogeneity in clinical trial populations (FDA, 2018, Workshop Rep 12). In non-small cell lung cancer, observational cohorts have shown that many inclusion criteria were redundant, since they restricted prospective trial enrollment, even though the treatment was as effective for excluded patients as for included patients (Liu et al., 2021, Nature 592:629-633). Since covariate adjustment allows analytically compensating for heterogeneity in the patient population, we investigated whether moderate covariate adjustment could be considered to allow expanding eligibility criteria while maintaining statistical power. For that purpose, we used both parametric and semi-synthetic (HCC adjuvant) simulation settings.

[0105] We applied HCCnet on 328 patients with early-stage HCC from the TCGA HCC dataset (Saillard et al., 2020, Hepatology 72:6; 2000–2013; Cancer Genome Atlas Research Network 2017 Cell 169:1327–1341). Based on this dataset with HCCnet predictions, semi-synthetic simulations were performed. All missing values ​​in 73 clinical variables with fewer than 50% missing variables and more than 1 modality were imputed to have a dataset with no missing data. For mixed data, we used a method relying on factor analysis and for data containing both continuous and categorical variables, we used a principal component analysis method (Josse and Husson, 2016, J Stat Softw 70). Imputed variables used as adjustments were tumor stage (1% missing values) and ECOG score, which had 20% missing values. Imputed variables used as eligibility criteria were ECOG score, Child-Pugh classification (33% missing), macrovascular invasion (15% missing), and hepatitis B or C infection status (15% and 5% missing values, respectively). Due to the large proportion of early censoring in the HCC-TCGA dataset, new event times were assigned while preserving the observed survival curves and the dependence on covariates. In doing so, a Cox model of overall survival was fitted against the available prognostic variables (tumor stage, ECOG score, and HCCnet variables). For each simulated patient, the patient's clinical covariates were sampled from TCGA. θ is the coefficient vector from the Cox regression. We define the hazard rate as before, except for substituting JPEG2025512702000017.jpg7150. Hazard rate, Weibull distribution, and baseline survival function from Cox regression The Weibull distribution was replaced by an empirical survival function that depends on JPEG2025512702000018.jpg6150 as follows: JPEG2025512702000019.jpg20150

[0106] As before, kappa was numerically optimized to set the outcome incidence in the placebo group to the observed outcome incidence. Finally, all patients with an event occurring after 5 years were censored at this time point. A sample size of 760 patients was chosen for the clinical trial simulation because this was the average sample size among four ongoing trials of adjuvant treatment in early-stage HCC (clinicaltrials.gov NCT03383458, NCT04102098, NCT03867084, NCT03847428). The treatment effect size hr was set such that the estimated statistical power reached by adjustment for clinical variables for this sample size was 80%. Randomization of treatment assignment was stratified based on tumor stage. The statistical power curves of the two options of variable adjustment were compared: the statistical power curves of adjustment for clinical variables (tumor stage and ECOG score) and the statistical power curves of adjustment for HCCnet and clinical variables. Statistical power was estimated with 5e+3 resamples.

[0107] Figure 7 shows the statistical power curves for adding HCCnet to tumor stage and ECOG. HCCnet is R 2 obs This allows for a reduction in sample size of 10.3% and an increase in statistical power of 4.4%.

[0108] The fit of this result with the results of the parametric simulation. The mortality rate in the HCC-TCGA population was 32.3% at 5 years. The tumor stage and ECOG score had a C-index of 0.65 in the original population, whereas adding HCCnet produced a C-index of 0.72.

[0109] As used herein, the quantities associated with adjustments for the clinical variables (tumor stage and ECOG) are labeled as 1, and adjustments for the HCCnet model (tumor stage, ECOG, and HCCnet) are labeled as 2. The R 2 obs,iApplying the Fleiss formula using: JPEG2025512702000020.jpg11157

[0110] Therefore, the semi-synthetic simulation is consistent with the findings of the parametric simulation. Furthermore, the R 2 CS Calculating the scale gives a value of 9.0%.

[0111] B. Simulations on mesothelioma data using MesoNet Deep learning models have been developed to predict overall survival in patients with malignant mesothelioma using whole slide (histology) images of tumor tissue (Courtiol et al., 2019, Nat Medicine 25:10;1519-1525). Deep learning survival predictions can outperform existing subtype classifications utilized by pathologists. As shown in Figure 8A-C, the use of MesoNet risk scores as an adjustment covariate in addition to histological subtype in the primary analysis of the clinical trial may enable clinical trial participants to reduce the sample size requirements of three large phase 3 mesothelioma clinical trials (PROMISE-meso, BEAT-meso, and CheckMate743) by 6-13%. In these simulations, we assumed that the current clinical trials stratified patients, adjusted analyses for histological subtype (epithelioid vs. sarcomatoid vs. mixed), and had a sample size corresponding to 80% statistical power. All of these simulations were performed using the MesoNet training dataset from Mesobank.

[0112] Furthermore, assuming gradual patient enrollment, the reduced sample size achieved using MesoNet could potentially reduce trial turnaround times by 2-8 months compared to current implementations. Thus, MesoNet could reduce sample sizes, lower costs, and shorten trial durations for mesothelioma clinical trials.

[0113] VII. Achieving target statistical power using less stringent eligibility criteria The present disclosure provides a method to obtain target statistical power with less stringent eligibility criteria in clinical trials by using adjustment covariates that can compensate for the high heterogeneity associated with less restrictive inclusion criteria. Parametric and semi-synthetic simulations were used to explore the interaction between the degree of restrictiveness of the inclusion criteria and adjustment for covariates. In the parametric simulation case, the inclusion restriction criteria were based on the value of X, a covariate that summarizes prognostic information, and the most restrictive eligibility criteria included only patients with X below 0, i.e., low-risk patients (X was associated with a hazard ratio of 2 in this experiment). In this case, there are four scenarios combining the two possible eligibility criteria (all patients or with inclusion restrictions) and the two adjustment options (no adjustment or adjustment for X). Other parameters are the Weibull shape and the treatment hazard ratio, which were set to 1 and 0.7, respectively. The observed outcome incidence was 96.5% when all patients were included and 93.3% with inclusion restrictions.

[0114] In the case of the HCC semi-synthetic simulation, two additional levels of eligibility restriction criteria were defined as shown in Table 2. The less restrictive eligibility level had two inclusion criteria present in all four ongoing large clinical trials of adjuvant treatment in early stage HCC (clinicaltrials.gov NCT03383458, NCT04102098, NCT03867084, NCT03847428) and included only patients with a Child-Pugh score of A and an ECOG status of 0 or 1. The most restrictive eligibility criteria further restricted ECOG status to 0 as in the STORM trial (Bruix et al., 2015, Lancet Oncol 16:1344-1354), excluded patients with hepatitis B and C co-infection as in the KEYNOTE-937 trial (clinicaltrials.gov NCT03867084), and excluded patients with macrovascular invasion as in the IMBRAVE050 trial (clinicaltrials.gov NCT04102098). Only eligibility criteria that were available in the TCGA HCC dataset were considered. From the above, there are six different scenarios combining the two adjustment options considered (clinical, i.e., adjustment for tumor stage and adjustment for ECOG or HCCnet) and the three eligibility levels. In the scenario with the most restrictive eligibility level, all patients had an ECOG of 0, and therefore, no analysis was adjusted for ECOG.

[0115] In both cases, changing the inclusion criteria altered the number of events, which directly impacts the statistical power. Thus, the statistical power of the various scenarios is expressed as a function of the number of events. In both cases, no additional dropouts were added and 5000 replicates were performed.

[0116] (Table 2) Table 2. Eligibility level definitions from the HCC-TCGA dataset JPEG2025512702000021.jpg61153

[0117] FIG. 9A depicts the interaction between eligibility criteria and adjustment options in a parametric simulation where the lowest risk patients were selected with a 50% inclusion rate. FIG. 9B depicts the interaction between eligibility criteria and adjustment options in a semi-synthetic simulation based on the HCC-TCGA dataset. FIG. 9A and FIG. 9B show that for the same number of events in the unadjusted analysis in both parametric and semi-synthetic simulations, it is possible to have higher statistical power by restricting the population. This is because the population becomes more homogeneous. However, when the analysis is adjusted using all prognostic information, the same statistical power curve can be obtained regardless of the inclusion (eligibility) criteria. Patients at lower risk of events (e.g., death) are selected for enrollment when more restrictive criteria are considered. Therefore, focusing on sample size rather than the number of events (i.e., including subjects with and without events) may be advantageous for more inclusive criteria.

[0118] The adjusted analyses using different eligibility criteria have the same statistical power, but the different eligibility criteria mean that the sizes of the selection populations are very different. For example, in the HCC example, the required size of the selection population is 604 for the less restrictive inclusion, but 1729 for the most restrictive population. The selection population is therefore reduced by 65% ​​while obtaining the same statistical power. This difference in the selection population is explained by the smaller proportion of included patients in the case of the restrictive eligibility criteria, as well as the smaller proportion of events (20.5% at 5 years compared to 32.3% in the total population).

[0119] Restrictive eligibility criteria ensure a homogeneous population, but they lead to difficulties in enrollment and questionable generalizability of the trial results. This has led to calls for less restrictive eligibility criteria (FDA, 2020, https: / / www.fda.gov / regulatory-information / search-fda-guidance-documents / enhancing-diversity-clinical-trial-populations-eligibility-criteria-enrollment-practices-and-trial, FDA, 2018, Workshop Rep 12). The present disclosure provides that moderate covariate adjustments eliminate any motivation to homogenize the population using restrictive eligibility criteria. In fact, the adjusted analyses in both parametric simulations and semi-synthetic simulations based on real clinical trial datasets are equally powerful regardless of the stringency of the eligibility criteria. Thus, by performing covariate adjustment, it is possible to significantly reduce the size of the population that needs to be screened for inclusion by using less restrictive eligibility criteria while maintaining the same statistical power.

[0120] As noted in the draft FDA guidance, covariate adjustment causes a shift in the estimated target, a phenomenon called non-mergerability (FDA, 2021, https: / / www.fda.gov / regulatory-information / search-fda-guidance-documents / adjusting-covariates-randomized-clinical-trials-drugs-and-biological-products). When marginal estimates are favorable, adjusted marginal estimators can be devised that target the estimates of the unadjusted analysis while taking advantage of the increased precision afforded by the covariates (Daniel et al., 2020, Biom J 63:528-557; Permutt, 2020, Stat Biopharm Res 12:45-53).

[0121] The methods described herein encompass adjustment covariates associated with relative measures of treatment effect (e.g., hazard ratios), as well as absolute measures, such as restricted mean survival time or absolute risk reduction for absolute risk effects. Both relative and absolute measures of treatment effect can be refined using the prognostic value of covariates according to the present disclosure.

[0122] VIII. Computer System and Machine-Readable Medium As shown in Figure 10, a computer system 1000 in the form of a data processing system includes a bus 1003 coupled to a microprocessor 1005, a ROM (read only memory) 1007, a volatile RAM 1009, and a non-volatile memory 1013. The microprocessor 1005 may include one or more CPUs, GPUs, special purpose processors, and / or combinations thereof. The microprocessor 1005 may be in communication with a cache 1004 and may retrieve instructions from the memories 1007, 1009, 1013 and execute those instructions to perform the operations described above. The bus 1003 interconnects these various components and further interconnects these components 1005, 1007, 1009, and 1013 to a display controller and display device 1015 and to peripheral devices such as input / output (I / O) devices 1011, which may be mice, keyboards, modems, network interfaces, printers, and other devices known in the art. Typically, input / output devices 1011 are coupled to the system through an input / output controller 1017. Volatile RAM (random access memory) 1009 is typically implemented as dynamic RAM (DRAM), which requires constant power to refresh or maintain the data in the memory.

[0123] The non-volatile memory 1013 may be, for example, a magnetic hard drive, a magneto-optical drive, an optical drive, a DVD RAM, a flash memory, or other type of memory system that retains data (e.g., large amounts of data) even after power is removed from the system. Typically, the non-volatile memory 1013 will also be a random access memory, although this is not required. While FIG. 10 illustrates that the non-volatile memory 1013 is a local device that is directly coupled to the rest of the components within the data processing system, it will be appreciated that the present invention may utilize non-volatile memory that is remote from the system, such as a network storage device that is coupled to the data processing system through a network interface, such as a modem, an Ethernet interface, or a wireless network. The bus 1003 may include one or more buses connected together through various bridges, controllers, and / or adapters as are known in the art.

[0124] Portions of the above description may be implemented using logic circuitry, such as dedicated logic circuitry, or using a microcontroller or other form of processing core executing program code instructions. Thus, the process taught by the above discussion may be implemented by program code, such as machine-executable instructions, which cause the machine executing it to perform certain functions. In this context, a "machine" may be a machine that converts intermediate form (or "abstract") instructions into processor-specific instructions (e.g., abstract execution environments such as "virtual machines" (e.g., Java Virtual Machines), interpreters, common language runtimes, high-level language virtual machines, etc.), and / or electronic circuitry (e.g., "logic circuitry" implemented with transistors) located on a semiconductor chip and designed to execute instructions, such as general-purpose processors and / or special-purpose processors. The process taught by the above discussion may also be implemented by electronic circuitry designed to execute the process (or a portion thereof) without executing program code (either in the alternative to or in combination with a machine).

[0125] The present invention also relates to an apparatus for performing the operations described herein. This apparatus may be specially constructed for the required purposes, or may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs, and magneto-optical disks, read-only memory (ROM), RAM, EPROM, EEPROM, magnetic or optical cards, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0126] A machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer). For example, machine-readable media include read-only memory ("ROM"), random access memory ("RAM"), magnetic disk storage media, optical storage media, flash memory devices, etc.

[0127] An article of manufacture can be used to store the program code. The article of manufacture storing the program code can be embodied as, but is not limited to, one or more memories (e.g., one or more flash memories, random access memories (static, dynamic, or other)), optical disks, CD-ROMs, DVD ROMs, EPROMs, EEPROMs, magnetic or optical cards, or other types of machine-readable media suitable for storing electronic instructions. The program code can also be downloaded from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by a data signal embodied in a propagation medium (e.g., via a communications link (e.g., a network connection)).

[0128] The preceding detailed description has been described in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the tools used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is conceived, and generally considered, herein to be a self-consistent sequence of operations leading to a desired result. The operations require physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0129] However, it must be borne in mind that all of these and similar terms are associated with the appropriate physical quantities and that these terms are merely convenient labels applied to these quantities.Unless otherwise expressly stated, and as will be apparent from the above discussion, discussions utilizing terms such as "segment," "tile," "receive," "calculate," "extract," "process," "apply," "enhance," "normalize," "pretrain," "screen," "select," "aggregate," or "sort" throughout this specification are recognized to refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities in the registers and memory of the computer system into other data also represented as physical quantities in the memory or registers of the computer system or other such information storage, transmission, or display devices.

[0130] The processes and displays presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used in conjunction with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the operations described. The required structure for a variety of these systems will be apparent from the description herein. In addition, the present invention is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages ​​may be used to implement the teachings of the present invention as described herein.

[0131] The above description merely describes some exemplary embodiments of the present invention. Those skilled in the art will readily recognize that various modifications can be made from such discussions, the accompanying drawings, and the claims without departing from the spirit and scope of the present invention. Furthermore, where feasible, any of the aspects disclosed herein can be combined with each other (e.g., features from one aspect may be added to or substitute for equivalent features of another aspect) or combined with features known in the art, unless circumstances indicate otherwise.

[0132] All references to references, including, for example, references to patents, published patent applications, and articles, are incorporated herein by reference in their entireties.

[0133] The section headings used herein are for organizational purposes only and are not to be construed as limiting the subject matter described in any way. [Explanation of symbols]

[0134] 100 Process for Designing Sample Size for Randomized Controlled Trials (RCTs) 101 Selecting Adjustment Covariates 105 Design sample size based on known correlations 109, 111 Start clinical trials 113 Observations in clinical trials are obtained in a blinded manner

Claims

1. A method for designing a randomized controlled trial (RCT) using the outcome of time to an event, The step of selecting covariates for regulation, formula: N adjusted =N original (1-R 2 CS ) The first step is to calculate the number of events required to obtain statistical power based on the above, Includes, Here, the RCT is performed using the number of events calculated above. Here, the N original This is the original number of events required to obtain the aforementioned statistical power without covariate adjustment. Here, the N adjusted This is the adjusted number of events required to obtain the aforementioned statistical power using covariate adjustment. Here, R 2 CS The formula is: Based on this, calculations are performed for data outside the RCT, Here, the said R 2 CS is the Cox·Snell R 2 and is The above n is the number of participants, Said l 0 This is the log-likelihood of the Cox model that explains the outcome of the time to the event using only the intercept. Said l 1 This is the log-likelihood of the Cox model that explains the outcome of the time to the event through the intercept and covariate adjustment. method.

2. A method for evaluating sample size in the interim stage of an ongoing randomized controlled trial (RCT), The step of selecting covariates for regulation, The stage of acquiring blinded RCT data, In the aforementioned intermediate stage, R 2 CS And formula: N adjusted =N original (1-R 2 CS ) The steps include performing a blinded sample size re-estimation using and Includes, Here, the RCT is further performed using the blinded sample size reestimation, Here, the N original This is the original number of events required to obtain statistical power without covariate adjustment. Here, the N adjusted This is the re-estimated number of events required to obtain the aforementioned statistical power, Here, R 2 CS In the aforementioned intermediate stage, the formula is: Based on this, the data is calculated for the blinded RCT data. Here, R 2 CS Cox Snell R 2 And, The above n is the number of participants, Said l 0 This is the log-likelihood of the Cox model that explains the outcome of the time to the event using only the intercept. Said l 1 This is the log-likelihood of the Cox model that explains the outcome of the time to the event through the intercept and covariate adjustment. method.

3. formula: The number of original events (N) required to obtain the statistical power without covariate adjustment based on the above. original ) at the stage of evaluation, It further includes, Here, the N original This is the estimated number of events required to obtain the statistical power based on the Schoennfeld formula, The aforementioned α is a Type I error level, The aforementioned β is a Type II error level, The aforementioned P 1 and P 2 These represent the proportion of the clinical trial sample included in the treatment group and the control group, respectively (for example, if the treatment allocation is balanced, both are equal to 1 / 2). The aforementioned hr is a specified hazard ratio, Said z p This is the p-quantile of the standard normal distribution. The method according to claim 1 or claim 2.

4. The method according to claim 1 or claim 2, wherein the outcome of the time to the aforementioned event is overall survival, disease-free survival, or time to disease recurrence.

5. The method according to claim 1 or 2, wherein the RCT is performed to evaluate the therapeutic effect in cancer patients.

6. The method according to claim 5, wherein the cancer is hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.

7. Covariate adjustment is performed for the clinical risk score. The method is The stage of obtaining clinical attributes derived from the subjects, A step of calculating a clinical risk score that quantifies the prognosis of the subject using a clinical model trained with one or more subject attributes, including, The method according to claim 1 or claim 2.

8. The method according to claim 1 or claim 2, wherein covariate adjustment is performed on covariates obtained by a deep learning model.

9. The method according to claim 8, wherein the deep learning model is based on histopathological slides obtained from cancer subjects, and the covariates are prognostic covariates.

10. The aforementioned covariates are obtained by a computer-implemented method for determining the prognosis likelihood of subjects with the disease, and the method is The steps include: accessing digital tissue structure images of tissue structure sections obtained from the aforementioned subjects; A step of extracting a plurality of feature vectors of the tissue structure image by applying a first convolutional neural network, wherein each of the features of the plurality of feature vectors represents a local descriptor of the tissue structure image, A step of classifying the tissue structure image using at least the plurality of feature vectors and a classification model, wherein the classification model is trained using a training set of known tissue structure images and known prognostic information, The steps include determining the likelihood of the prognosis of the subject based on the classification of the tissue structure images, The method according to claim 8, including the method described in claim 8.

11. The aforementioned covariates are obtained by a computer-implemented method for determining the prognosis of subjects with a disease, and the method is The steps include obtaining a digital tissue structure image of the tissue structure section from the subject, The steps include dividing the aforementioned digital image into a tile set, The steps include: extracting multiple feature vectors from the aforementioned tile set or a subset thereof; A step of calculating an artificial intelligence (AI) risk score that quantifies the prognosis of the subject based on the tissue structure image, using a machine learning model trained by processing multiple training images to predict the prognosis, The method according to claim 8, including the method described in claim 8.

12. The steps include obtaining clinical attributes derived from the aforementioned subjects, A step of calculating a clinical risk score using a clinical model trained with one or more subject attributes, A step of calculating a final risk score for the subject from the AI ​​risk score and the clinical risk score, wherein the final risk score quantifies the subject's prognosis. The method according to claim 11, further comprising:

13. The method according to claim 10, wherein the digital tissue structure image is a whole slide image (WSI).

14. The method according to claim 10, wherein the tissue structure section is stained with a dye.

15. The method according to claim 14, wherein the dye is hematoxylin and eosin (H&E).

16. The method according to claim 10, wherein the disease is cancer.

17. The method according to claim 16, wherein the cancer is hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.

18. The method according to claim 1 or 2, wherein subject enrollment based on restrictive eligibility criteria does not improve statistical power compared to subject enrollment based on less restrictive eligibility criteria.

19. The method according to claim 1 or 2, wherein the targeted statistical power is achieved using less stringent eligibility criteria in a clinical trial.

20. The method according to claim 1 or claim 2, performed by a computer.

21. The step of selecting covariates for regulation, formula: N adjusted =N original (1-R 2 CS ) The steps include: calculating the sample size required to obtain statistical power based on the above; Includes, Here, the RCT is performed using the calculated sample size. Here, the N original However, this is the original number of events required to obtain the aforementioned statistical power without covariate adjustment. Here, the N adjusted However, this is the adjusted number of events required to obtain the aforementioned statistical power using covariate adjustment. Here, R 2 CS However, the formula is: Based on this, calculations are performed for data outside the RCT, Here, R 2 CS However, Cox Snell R 2 And, The aforementioned n is the number of participants, Said l 0 However, this is the log-likelihood of the Cox model that explains the outcome of the time up to the aforementioned event using only the intercept. Said l 1 However, the log-likelihood of the Cox model that explains the outcome of the time to the aforementioned event through the intercept and covariate adjustment is A machine-readable medium having executable instructions that cause one or more processing units to execute a method for designing a randomized controlled trial (RCT).

22. The step of selecting covariates for regulation, The stage of acquiring blinded RCT data, In the intermediate stage, R 2 CS And formula: N adjusted =N original (1-R 2 CS ) The steps include performing a blinded sample size re-estimation using and Includes, Here, the RCT is further performed using the blinded sample size reestimation, Here, the N original However, this is the original number of events required to obtain statistical power without covariate adjustment. Here, the N adjusted However, this is the re-estimated number of events required to obtain the aforementioned statistical power, Here, R 2 CS However, in the intermediate stage, the formula is: Based on this, the data is calculated for the blinded RCT data. Here, R 2 CS However, Cox Snell R 2 And, The aforementioned n is the number of participants, Said l 0 However, this is the log-likelihood of the Cox model that explains the outcome of the time up to the aforementioned event using only the intercept. Said l 1 However, the log-likelihood of the Cox model that explains the outcome of the time to the aforementioned event through the intercept and covariate adjustment is A machine-readable medium having executable instructions that cause one or more processing units to execute a method for evaluating the sample size required to obtain statistical power in an ongoing randomized controlled trial (RCT).

23. The aforementioned method, formula: The number of original events (N) required to obtain the statistical power without covariate adjustment based on the above. original ) at the stage of evaluation, It further includes, Here, the N original This is the estimated number of events required to obtain the statistical power based on the Schoennfeld formula, The aforementioned α is a Type I error level, The aforementioned β is a Type II error level, The aforementioned P 1 and P 2 These represent the proportion of the clinical trial sample included in the treatment group and the control group, respectively (for example, if the treatment allocation is balanced, both are equal to 1 / 2). The aforementioned hr is a specified hazard ratio, Said z p This is the p-quantile of the standard normal distribution. A machine-readable medium according to claim 21 or claim 22.

24. The machine-readable medium according to claim 21 or claim 22, wherein the outcome of the time to the aforementioned event is overall survival, disease-free survival, or time to disease recurrence.

25. The machine-readable medium according to claim 21 or 22, wherein the RCT is performed to evaluate the therapeutic effect in cancer patients.

26. The machine-readable medium according to claim 25, wherein the cancer is hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.

27. Covariate adjustment is performed for the clinical risk score. Machine-readable media, The steps include obtaining clinical attributes derived from the aforementioned subjects, A step of calculating a clinical risk score that quantifies the prognosis of the subject using a clinical model trained with one or more subject attributes, including, A machine-readable medium according to claim 21 or claim 22.

28. The machine-readable medium according to claim 21 or claim 22, wherein covariate adjustment is performed on covariates obtained by a deep learning model.

29. The machine-readable medium according to claim 28, wherein the deep learning model is based on histopathological slides obtained from cancer subjects, and the covariates are prognostic covariates.

30. The aforementioned covariates are obtained by a computer-implemented machine-readable medium for determining the prognosis likelihood of subjects with the disease, and the machine-readable medium is, The steps include: accessing digital tissue structure images of tissue structure sections obtained from the aforementioned subjects; A step of extracting a plurality of feature vectors of the tissue structure image by applying a first convolutional neural network, wherein each of the features of the plurality of feature vectors represents a local descriptor of the tissue structure image, A step of classifying the tissue structure image using at least the plurality of feature vectors and a classification model, wherein the classification model is trained using a training set of known tissue structure images and known prognostic information, The steps include determining the likelihood of the prognosis of the subject based on the classification of the tissue structure images, A machine-readable medium according to claim 28, including the following:

31. The aforementioned covariates are obtained by a computer-implemented machine-readable medium for determining the prognosis of subjects with the disease, and the machine-readable medium is The steps include obtaining a digital tissue structure image of the tissue structure section from the subject, The steps include dividing the aforementioned digital image into a tile set, The steps include: extracting multiple feature vectors from the aforementioned tile set or a subset thereof; A step of calculating an artificial intelligence (AI) risk score that quantifies the prognosis of the subject based on the tissue structure image, using a machine learning model trained by processing multiple training images to predict the prognosis, A machine-readable medium according to claim 28, including the following:

32. The steps include obtaining clinical attributes derived from the aforementioned subjects, A step of calculating a clinical risk score using a clinical model trained with one or more subject attributes, A step of calculating a final risk score for the subject from the AI ​​risk score and the clinical risk score, wherein the final risk score quantifies the subject's prognosis. The machine-readable medium according to claim 31, further comprising:

33. The machine-readable medium according to claim 30, wherein the digital organizational structure image is a whole slide image (WSI).

34. The machine-readable medium according to claim 30, wherein the tissue structure section is stained with a dye.

35. The machine-readable medium according to claim 34, wherein the dye is hematoxylin and eosin (H&E).

36. The machine-readable medium according to claim 30, wherein the disease is cancer.

37. The machine-readable medium according to claim 36, wherein the cancer is hepatocellular carcinoma, mesothelioma, pancreatic cancer, lung cancer, or breast cancer.

38. The machine-readable medium according to claim 21 or 22, wherein subject enrollment based on restrictive eligibility criteria does not improve statistical power compared to subject enrollment based on less restrictive eligibility criteria.

39. The machine-readable medium according to claim 21 or 22, wherein the target statistical power is achieved using less stringent eligibility criteria in a clinical trial.

Images