Model-based featurization and classification

Abstract

To provide model-based featurization and classifiers for predicting a disease state from nucleic acid samples.SOLUTION: In various embodiments, an analytics system uses models to determine features and classification of a disease state. The disease state can indicate presence or absence of cancer, a cancer type, or a cancer tissue of origin. The models can include a binary classifier and a tissue of origin classifier. The analytics system can process sequence reads from test biological samples to generate data for training the classifiers. The analytics system can also use combinations of machine learning techniques to train the models, which can include a multilayer perceptron. In some embodiments, the analytics system uses methylation information to train the models to determine predictions regarding the disease state.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates generally to model-based feature quantification and classifiers for predicting disease states from nucleic acid samples. [Background technology]

[0002] DNA methylation plays a role in regulating gene expression. Many disease processes, including cancer, involve aberrant DNA methylation. DNA methylation profiling using methylation sequencing (e.g., whole genome bisulfite sequencing (WGBS)) is increasingly recognized as a valuable diagnostic tool for the detection, diagnosis, and / or monitoring of cancer. For example, specific patterns of differentially methylated regions may be useful as molecular markers for various disease states. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2010 / 037001 Brochure [Patent Document 2] International Publication No. 2011 / 127136 Brochure [Patent Document 3] US Patent Application Publication No. 2019 / 0287652 [Patent Document 4] U.S. Patent Application Serial No. 16 / 352,602 [Patent Document 5] International Publication No. 2019 / 195268 Brochure [Patent Document 6] PCT / US Patent Application Publication No. 2019 / 053509 [Patent Document 7] PCT / US Patent Application Publication No. 2020 / 015082 [Non-patent literature]

[0004] [Non-Patent Document 1] Clinical Trial.gov Identifier: NCT02889978 (https: / / www.clinicaltrials.gov / ct2 / show / NCT02889978) [Non-Patent Document 2] Clinical Trail.gov Identifier: NCT03085888( / / clinicaltrials.gov / ct2 / show / NCT03085888) [Non-Patent Document 3] The Annals of Oncology journal article titled "Sensitive and specific multi-cancer detection and localization using methylation signatures in cell-free DNA" was published online on March 30, 2020 (https: / / www.annalsofoncology.org / article / S0923-7534(20)36058-0 / fulltext). [Non-Patent Document 4] Riedmiller M, Braun H. RPROP - A Fast Adaptive Learning Algorithm. Proceedings of the International Symposium on Computer and Information Science VII, 1992 Summary of the Invention

[0005] Disclosed herein are methods for training and applying models for generating features and / or for classifying disease states (e.g., presence or absence of cancer, cancer type, and / or tissue of origin of cancer) using a nucleic acid sample. In one aspect, the disclosure provides a method for analyzing sequence reads to generate a plurality of features, comprising: generating a first plurality of reference sequence reads from a first reference sample, the first sample being from a subject having a first disease state; generating a second plurality of reference sequence reads from a second reference sample, the second sample being from a subject having a second disease state; training a first probability model using the first plurality of reference sequence reads, the first probability model being associated with the first disease state; and training a second probability model using the second plurality of reference sequence reads, the second probability model being associated with the first disease state. The model is associated with a second disease state; generating a plurality of training sequence reads from a training sample, where for each sequence read of the plurality of training sequence reads, applying the sequence read to a first probabilistic model to determine a first probability value, the first probability value being a probability that the sequence read is derived from a sample associated with the first disease state, and applying the sequence read to a second probabilistic model to determine a second probability value, the second probability value being a probability that the sequence read is derived from a sample associated with the second disease state; and identifying one or more features by comparing, for each sequence read, the first probability value and the second probability value.

[0006] In another aspect, the present disclosure provides a system comprising a computer processor and a memory, which when executed by the computer processor, includes the steps of: accessing a first plurality of reference sequence reads from a first reference sample, the first sample being from a subject having a first disease state; accessing a second plurality of reference sequence reads from a second reference sample, the second sample being from a subject having a second disease state; training a first probability model using the first plurality of reference sequence reads, the first probability model being associated with the first disease state; and training a second probability model using the second plurality of reference sequence reads, the second probability model being associated with the first disease state. The method includes the steps of: accessing a plurality of training sequence reads from a training sample, the model being associated with a second disease state; for each sequence read of the plurality of training sequence reads, applying the sequence read to a first probabilistic model to determine a first probability value, the first probability value being a probability that the sequence read is from a sample associated with the first disease state, and applying the sequence read to a second probabilistic model to determine a second probability value, the second probability value being a probability that the sequence read is from a sample associated with the second disease state; and identifying, for each sequence read, one or more features by comparing the first probability value to the second probability value.

[0007] In another aspect, the disclosure provides a method, when executed by one or more processors, for detecting a first plurality of reference sequence reads from a first reference sample, the first sample being from a subject having a first disease state; accessing a second plurality of reference sequence reads from a second reference sample, the second sample being from a subject having a second disease state; training a first probabilistic model using the first plurality of reference sequence reads, the first probabilistic model being associated with the first disease state; and training a second probabilistic model using the second plurality of reference sequence reads, the second probabilistic model being associated with the second disease state. accessing a plurality of training sequence reads from a training sample, where, for each sequence read of the plurality of training sequence reads, applying the sequence read to a first probability model to determine a first probability value, the first probability value being a probability that the sequence read is from a sample associated with a first disease state, and applying the sequence read to a second probability model to determine a second probability value, the second probability value being a probability that the sequence read is from a sample associated with a second disease state; and identifying, for each sequence read, one or more features by comparing the first probability value to the second probability value.

[0008] In some embodiments, the first disease state is cancer and the second disease state is non-cancer. In some embodiments, the first disease state is a first type of cancer and the second disease state is a second type of cancer, and the first type of cancer and the second type of cancer are different.

[0009] In some embodiments, the method, system, or non-transitory computer-readable medium further includes generating a plurality of reference sequence reads from a third, fourth, fifth, sixth, seventh, eighth, ninth, and / or tenth reference sample, wherein each of the third, fourth, fifth, sixth, seventh, eighth, ninth, and / or tenth reference samples has a different disease state, each of the different disease states being a different type of cancer; and training a third, fourth, fifth, sixth, seventh, eighth, ninth, and / or tenth probabilistic model using the third, fourth, fifth, sixth, seventh, eighth, ninth, and / or tenth plurality of reference sequence reads, wherein each of the third, fourth, fifth, sixth, seventh, eighth, ninth, and / or tenth probabilistic models are each associated with a different type of cancer.

[0010] In some embodiments, the cancer or type of cancer is selected from the group including breast cancer, uterine cancer, cervical cancer, ovarian cancer, bladder cancer, urothelial cancer of the renal pelvis and ureter, non-urothelial renal cancer, prostate cancer, anorectal cancer, colorectal cancer, squamous cell carcinoma of the esophagus, non-squamous esophageal cancer, gastric cancer, hepatobiliary cancer arising from hepatocytes, hepatobiliary pancreatic cancer arising from cells other than hepatocytes, pancreatic cancer, head and neck cancer associated with human papillomavirus, head and neck cancer not associated with human papillomavirus, lung adenocarcinoma, small cell lung cancer, squamous cell lung cancer and lung cancer other than adenocarcinoma or small cell lung cancer, neuroendocrine cancer, melanoma, thyroid cancer, sarcoma, multiple myeloma, lymphoma, and leukemia. In some embodiments, the type of cancer is further selected from the group including brain tumor, vulvar cancer, vaginal cancer, testicular cancer, pleural mesothelioma, peritoneal mesothelioma, and gallbladder cancer.

[0011] In some embodiments, the first disease state comprises a first primary tissue and the second disease state comprises a second primary tissue. The first primary tissue or the second primary tissue can be selected from the group comprising breast tissue, thyroid tissue, lung tissue, bladder tissue, cervical tissue, small intestine tissue, colorectal tissue, esophageal tissue, stomach tissue, tonsil tissue, liver tissue, ovarian tissue, fallopian tube tissue, pancreatic tissue, prostate tissue, kidney tissue, and uterine tissue. In some embodiments, the first primary tissue or the second primary tissue is further selected from the group comprising brain tissue and cells, endocrine tissue and cells, vascular endothelial tissue and cells, head and neck tissue and cells, exocrine pancreatic tissue and cells, endocrine pancreatic tissue and cells, lymphatic tissue and cells, mesenchymal tissue and cells, bone marrow tissue and cells, pleural tissue and cells, muscle tissue and cells, bone marrow tissue and cells, adipose tissue and cells, gallbladder tissue and cells.

[0012] In some embodiments, the first probability model or the second probability model is a constant model, a binomial model, an independent sites model, a neural net model, or a Markov model.

[0013] In some embodiments, the method, system, or non-transitory computer-readable medium of the present disclosure further includes determining a methylation ratio for each of a plurality of CpG sites in the first plurality of reference sequence reads or the second plurality of reference sequence reads, wherein the first probability model or the second probability model is parameterized by a product of methylation ratios.

[0014] In some embodiments, the method, system, or non-transitory computer-readable medium of the present disclosure further includes, for each sequence read of the first plurality of reference sequence reads or the second plurality of sequence reads, determining whether the sequence read is aberrantly methylated, and filtering the first plurality of reference sequence reads or the second plurality of sequence reads using p-value filtering by removing sequence reads from the first plurality of reference sequence reads or the second plurality of sequence reads that have a p-value below a threshold.

[0015] In some embodiments, the disclosed method, system, or non-transitory computer readable medium further includes determining, for each sequence read of the first plurality of reference sequence reads, the second plurality of sequence reads, or the plurality of training sequence reads, whether the sequence read is hypomethylated or hypermethylated by determining whether at least a threshold number of CpG sites, having at least a threshold percentage of CpG sites, are unmethylated or methylated, respectively.

[0016] In some embodiments, the method, system, or non-transitory computer-readable medium of the present disclosure further includes, for each sequence read of the first plurality of reference sequence reads, the second plurality of sequence reads, or the plurality of training sequence reads, determining whether the sequence read is aberrantly methylated, and filtering the first plurality of reference sequence reads using p-value filtering by removing sequence reads from the first plurality of reference sequence reads that have a p-value below a threshold.

[0017] In some embodiments, the first probabilistic model or the second probabilistic model is parameterized by a sum of a plurality of mixture components, each associated with a product of methylation ratios. In some embodiments, each mixture component of the plurality of mixture components is associated with a proportion assignment, the proportion assignments summing to 1.

[0018] In some embodiments, training the first or second probabilistic model includes determining a set of parameters for the probabilistic model that maximizes the total log-likelihood of the first or second plurality of reference sequence reads derived from a subject associated with the first or second disease state associated with the probabilistic model.

[0019] In some embodiments, the method, system, or non-transitory computer-readable medium of the present disclosure further includes, for each of a plurality of windows, selecting a plurality of the first plurality of reference sequence reads taken from the window and utilizing the sequence reads taken from the window to train a first probability model for the window, and selecting a plurality of the second plurality of reference sequence reads taken from the window and utilizing the sequence reads taken from the window to train a probability model for each window.

[0020] In some embodiments, the method, system, or non-transitory computer-readable medium of the present disclosure further comprises: for each of the plurality of windows, selecting a subset of the plurality of training sequence reads taken from the window; and for each sequence read of the subset, identifying one or more features by comparing the first probability value to the second probability value. In some embodiments, each of the windows is separated by at least a threshold number of base pairs between CpG sites. In some embodiments, each of the plurality of windows comprises about 200 base pairs (bp) to about 10 kilobase pairs (kbp).

[0021] In some embodiments, the one or more features include a count of outlier sequence reads of the plurality of training sequence reads where the first probability value is greater than the second probability value. In some embodiments, the one or more features include a binary count. In some embodiments, the one or more features include a total count of outlier sequence reads. In some embodiments, the one or more features include a total count of anonymously methylated sequence reads. In some embodiments, the one or more features include a count of fragments that include one or more particular methylation patterns. In some embodiments, the one or more features are identified using an output of a discrimination classifier trained within a single genomic region. In some embodiments, the discrimination classifier is a multi-layer perceptron or a convolutional neural net model. In some embodiments, comparing the first probability value to the second probability value includes determining a ratio of the first probability value to the second probability value, and the one or more features include a sequence read count of sequence reads that exceed a ratio threshold. In some embodiments, the first probability value or the second probability value is a log-likelihood value. In some embodiments, the one or more features include ranking informative sequence reads based on rarity of the sequence read in the first disease state.

[0022] In some embodiments, identifying one or more features includes determining, for each sequence read among the plurality of training sequence reads, a log-likelihood ratio of a first probability value to a second probability value, and determining, for one or more thresholds, a count of sequence reads having a log-likelihood ratio that exceeds the threshold.

[0023] In some embodiments, the disclosed method, system, or non-transitory computer readable medium further includes, for each of the one or more features, determining a decision measure for the feature in distinguishing between the first disease state and the second disease state.

[0024] In some embodiments, determining a decision measure for each of the one or more features comprises determining mutual information between the features and the probability of the presence of the first disease state and the second disease state. In some embodiments, the method of the present disclosure further comprises filtering the one or more features for training a classifier by ranking the features based on the decision measure.

[0025] In some embodiments, the disclosed method, system, or non-transitory computer-readable medium further comprises training a classifier from the one or more features, the classifier being trained to predict one or more disease states for a plurality of sequence reads from a test sample of a test subject, the one or more disease states including presence or absence of disease, disease type, and / or disease tissue of origin. In some embodiments, the classifier is a logistic regression, a multinomial logistic regression, a generalized linear model (GLM), a support vector machine, a multilayer perceptron, a random forest, or a neural net classifier. In some embodiments, the classifier is a multilayer perceptron model. In some embodiments, the classifier is generated using L1 or L2 regularized logistic regression. In some embodiments, the disclosed method further comprises determining a vector of probabilities for the test sample, and determining a label of the test sample based on the vector of probabilities.

[0026] In some embodiments, the method, system, or non-transitory computer readable medium of the present disclosure further includes determining the accuracy of the classifier using a confusion matrix, the confusion matrix including information describing the success rate of the classifier in identifying each of a plurality of disease states.

[0027] In some embodiments, the first reference sample or the second reference sample is a cell-free or tissue nucleic acid sample from a subject with a known disease state.

[0028] In some embodiments, the known disease state is the presence or absence of disease, the type of disease, and / or the tissue of disease origin.

[0029] In some embodiments, the training sample comprises a cell-free nucleic acid sample or a tissue sample. In some embodiments, the test sample comprises a cell-free nucleic acid sample.

[0030] In some embodiments, the first plurality of reference sequence reads, the second plurality of reference sequence reads, the plurality of training sequence reads, or the plurality of sequence reads from the test sample are generated from methylation sequencing (or methylation-aware sequencing). In some embodiments, the methylation sequencing comprises whole genome bisulfite sequencing. In some embodiments, the methylation sequencing comprises targeted sequencing.

[0031] In another aspect, the disclosure provides a method for generating a classifier for predicting a tissue of origin associated with a disease state, the method comprising: generating a first plurality of reference sequence reads from a reference sample having one of a plurality of disease states each associated with the tissue of origin; training a plurality of probability models using the first plurality of reference sequence reads, each associated with a different one of the plurality of disease states; applying the probability model to the sequence reads to determine, for each probability model of the plurality of probability models, a value for each of a second plurality of sequence reads based at least on a first probability that the sequence read originates from a sample associated with the disease state associated with the probability model; identifying features by determining a count of the second plurality of sequence reads having a value above a threshold; and using the features to generate a classifier, wherein the classifier is trained to predict the disease state and / or the tissue of origin associated with the disease state of the plurality of disease states for input sequence reads from a test sample of a test subject. In some embodiments, the multiple disease conditions include at least 2, at least 3, at least 4, at least 5, or at least 10 different disease conditions.

[0032] In some embodiments, the method further comprises determining a methylation ratio for each of a plurality of CpG sites in the first plurality of reference sequence reads, wherein each of the plurality of probabilistic models is parameterized by a product of the methylation ratios.

[0033] In some embodiments, each probabilistic model of the plurality of probabilistic models is parameterized by a sum of a plurality of mixture components, each associated with a product of methylation ratios. In some embodiments, each mixture component of the plurality of mixture components is associated with a proportion assignment, the proportion assignments summing to 1.

[0034] In some embodiments, training the plurality of probability models comprises determining, for a probability model of the plurality of probability models, a set of parameters that maximizes a total log-likelihood of a first plurality of reference sequence reads derived from a subject associated with a disease state associated with the probability model. In some embodiments, the method further comprises determining a vector of probabilities for the test sample and determining a label for the test sample based on the vector of probabilities.

[0035] In some embodiments, the step of determining the value comprises determining a first probability that the sequence read is derived from a sample associated with a disease state associated with the probabilistic model, where the disease state is associated with the presence or type of cancer; determining a second probability that the sequence read is derived from a healthy sample; and determining a log-likelihood ratio of the first probability to the second probability.

[0036] In some embodiments, identifying features comprises determining, for a plurality of thresholds, a count of a second plurality of sequence reads having a log-likelihood ratio that exceeds the threshold.

[0037] In some embodiments, the method further includes determining, for each of the features, a decision measure for the feature in distinguishing between a first disease state and a second disease state of the plurality of disease states.

[0038] In some embodiments, determining a decision measure for the feature comprises determining mutual information between the feature and the probability of the presence of the first disease state and the second disease state.

[0039] In some embodiments, the first probability of the first disease state is equal to the second probability of the second disease state. In some embodiments, the method further comprises filtering the features for training the classifier by ranking the features based on a decision measure.

[0040] In some embodiments, the method further includes determining the accuracy of the classifier using a confusion matrix, the confusion matrix including information describing the success rate of the classifier in identifying each of a plurality of disease states.

[0041] In some embodiments, the method further comprises determining a plurality of blocks of the reference genome, each of the blocks being separated by at least a threshold number of base pairs between CpG sites, and the first plurality of reference sequence reads are generated using the plurality of blocks. In some embodiments, a count of the second plurality of sequence reads having a value above the threshold is determined for the plurality of CpG sites.

[0042] In some embodiments, the reference sample comprises one or more of a cell-free nucleic acid sample and a tissue sample.

[0043] In some embodiments, the multiple disease states include one or more of a cancer type, a disease type, and a healthy state.

[0044] In some embodiments, the classifier is a logistic regression, a multinomial logistic regression, a generalized linear model (GLM), a multilayer perceptron, a support vector machine, a random forest, or a neural net model classifier. In some embodiments, the classifier is generated using L1 or L2 regularized logistic regression. In some embodiments, the classifier is a multilayer perceptron model.

[0045] In some embodiments, the method further comprises binarizing the features to indicate the presence or absence of one of a plurality of disease states, and the classifier is generated using the binarized features, each of which can have a value of 0 or 1.

[0046] In some embodiments, the method further comprises determining a metric of uncertainty in the localization for the reference sample and labeling at least one prediction of the classifier as an uncertain tissue of origin according to the metric.

[0047] In other aspects, the disclosure provides a method including generating a plurality of sequence reads from one or more biological samples; for each location among a plurality of chromosomal locations, determining using the plurality of sequence reads a count of nucleic acid fragments of the one or more biological samples within the location that have at least a threshold similarity to fragments associated with a disease state; training a machine learning model using the counts of the plurality of locations as features; and determining a probability that a test sample has the disease state using the trained machine learning model.

[0048] In some embodiments, the method further includes binarizing the features to indicate the presence or absence of one of the disease states at each of the plurality of locations, wherein a count of at least one nucleic acid fragment at a location indicates the presence of one of the disease states at that location.

[0049] In some embodiments, the method further comprises filtering the plurality of sequence reads according to a p-value score of the plurality of sequence reads, the p-value score of the sequence read indicating a probability of observing methylation in a nucleic acid fragment of the one or more biological samples corresponding to the sequence read.

[0050] In some embodiments, the machine learning model is a multi-layer perceptron model. In some embodiments, the machine learning model uses logistic regression. In some embodiments, each of the plurality of positions represents a plurality of consecutive base pairs of a chromosome.

[0051] In some embodiments, the plurality of sequence reads are processed for a plurality of regions of the genome. In some embodiments, the plurality of sequence reads represent nucleic acid fragments of a target subset of regions of the genome. In some embodiments, the plurality of sequence reads represent nucleic acid fragments of the entire genome. In some embodiments, the disease state is associated with at least one type of cancer. In some embodiments, the disease state is associated with a stage of at least one type of cancer. In some embodiments, the method further comprises using a probability that the test sample has the disease state to determine a treatment.

[0052] In another aspect, the disclosure provides a method comprising generating a plurality of sequence reads from nucleic acid fragments of a plurality of biological samples; determining a first set of training data by processing the plurality of sequence reads; and training a first classifier using the first set of training data, where the first classifier is trained to predict, for first input sequence reads from the first test biological sample, the presence or absence of at least one disease state in the first test biological sample; determining a subset of the plurality of biological samples having the presence of one or more disease states using a prediction of the first classifier; determining a second set of training data using a subset of the plurality of sequence reads corresponding to the nucleic acid fragments of the subset of the plurality of biological samples; and training a second classifier using the second set of training data, where the second classifier is trained to predict, for second input sequence reads from the second test biological sample, a tissue of origin associated with a disease state present in the second test biological sample.

[0053] In some embodiments, the second classifier is a multi-layer perceptron including at least one hidden layer. In some embodiments, the first classifier does not include a hidden layer. In some embodiments, the multi-layer perceptron includes a hidden layer of 100 units, or a hidden layer of 200 units. In some embodiments, the multi-layer perceptron is fully connected and uses a regularized linear unit activation function. In some embodiments, the second classifier is a logistic regression or multinomial logistic regression model. In some embodiments, the first classifier is a multi-layer perceptron including at least one hidden layer. In some embodiments, the multi-layer perceptron (first classifier) ​​includes a hidden layer of 100 units or more, and the multi-layer perceptron is fully connected and uses a regularized linear unit activation function. In some embodiments, the second classifier is a second multi-layer perceptron including at least one hidden layer. In some embodiments, the first classifier is a logistic regression or multinomial logistic regression model.

[0054] In some embodiments, the method further includes performing a first cross-validation on the first classifier, retraining the first classifier using first hyperparameters selected based on an output of the first cross-validation, performing a second cross-validation on the second classifier, and retraining the second classifier using second hyperparameters selected based on an output of the second cross-validation. In some embodiments, the first hyperparameters and the second hyperparameters are selected using aggregate results from all folds in the first cross-validation and the second cross-validation, respectively. In some embodiments, the second hyperparameters are selected to optimize the tissue of origin accuracy of the second classifier.

[0055] In some embodiments, the first classifier and the second classifier are trained without early truncation. In some embodiments, the second classifier is trained using one or more of the following machine learning techniques: stochastic gradient descent, weight decay, dropout regularization, Adam optimization, He initialization, learning rate scheduling, regularized linear unit activation function, leaky regularized linear unit activation function, sigmoid activation function, and boosting.

[0056] In some embodiments, determining the first set of training data by processing the plurality of sequence reads comprises determining a probability of observing methylation in the nucleic acid fragments of the plurality of biological samples, in some embodiments, the probability of observing methylation is determined for each of a plurality of CpG sites in the plurality of sequence reads.

[0057] In some embodiments, determining the first set of training data by processing the plurality of sequence reads comprises determining whether the plurality of sequence reads are hypomethylated or hypermethylated by determining, for each of the plurality of sequence reads, whether at least a threshold number of CpG sites, having at least a threshold percentage of CpG sites, respectively, are unmethylated or methylated.

[0058] In some embodiments, determining the first set of training data by processing the plurality of sequence reads comprises determining that one or more of the plurality of sequence reads are hypomethylated by determining a threshold number or percentage of CpG sites corresponding to one or more of the plurality of sequence reads are unmethylated. In some embodiments, determining the first set of training data by processing the plurality of sequence reads comprises determining that one or more of the plurality of sequence reads are hypermethylated by determining a threshold number or percentage of CpG sites corresponding to one or more of the plurality of sequence reads are methylated.

[0059] In some embodiments, determining a first set of training data by processing the plurality of sequence reads includes determining that one or more of the plurality of sequence reads are aberrantly methylated; and filtering the plurality of sequence reads with p-value filtering to generate the first set of training data, where the p-value filtering includes removing sequence reads having a p-value less than a threshold p-value.

[0060] In some embodiments, the method further comprises determining a score indicative of the probability that the primary tissue associated with the disease state is present in the second test biological sample by the second classifier, and calibrating the score. In some embodiments, calibrating the score comprises performing a k-nearest neighbor operation in association with the score using the feature space output by the second classifier. In some embodiments, the feature space comprises predicted labels indicative of at least a first primary tissue and a second primary tissue associated with the first disease state and the second disease state, respectively, present in the second test biological sample. In some embodiments, the feature space further comprises an indication that the correct primary tissue prediction for the second test biological sample is different from the first primary tissue and the second primary tissue.

[0061] In some embodiments, calibrating the score comprises normalizing the probability using a different probability of presence that at least one disease state is present in the second test biological sample, the different probability being determined by the first classifier.

[0062] In some embodiments, the method further comprises determining, by the first classifier, a probability that at least one disease state is present in the first test biological sample, and predicting the presence of the at least one disease state in the first test biological sample in response to determining that the probability is greater than a binary threshold. In some embodiments, the binary threshold is a specificity between 90% and 99.9%. In some embodiments, the second test biological sample has a probability predicted by the first classifier that is greater than the binary threshold.

[0063] In some embodiments, the first test biological sample is a second test biological sample.

[0064] In some embodiments, the method further comprises determining, by a second classifier, a probability that a primary tissue associated with the disease state is present in the second test biological sample, and predicting that a primary tissue associated with the disease state is present in the second test biological sample in response to determining that the probability is greater than a primary tissue threshold. In some embodiments, the method further comprises determining, by the second classifier, different probabilities that different primary tissues associated with different disease states are present in the second test biological sample, and predicting that different primary tissues associated with different disease states are present in the second test biological sample in response to determining that the different probabilities are greater than a second primary tissue threshold.

[0065] In some embodiments, the method further comprises determining a primary tissue threshold associated with a given disease state for the second classifier by determining a sensitivity rate at a given specificity rate of the second classifier for a plurality of different probabilities of the candidate primary tissue threshold. In some embodiments, the sensitivity rate is determined using the score output by the first classifier. In some embodiments, the sensitivity rate is determined using the score output by the second classifier to stratify the samples.

[0066] In some embodiments, the method further comprises optimizing a trade-off between the sensitivity rate and specificity rate of the second classifier for a given disease state. In some embodiments, a subset of the plurality of biological samples is labeled as having the presence of cancer of known tissue of origin according to information from a reference sample.

[0067] In various embodiments, the system includes a computer processor and a memory storing computer program instructions that, when executed by the computer processor, cause the processor to perform any of the methods described herein. In various embodiments, a non-transitory computer readable medium stores one or more programs including instructions that, when executed by an electronic device including the processor, cause the device to perform any of the methods described herein. [Brief description of the drawings]

[0068] [Figure 1] 1 is a flowchart of a method for generating a classifier for predicting a disease state, according to various embodiments. [Figure 2A] 1 is a flow chart of a device for sequencing a nucleic acid sample, according to one embodiment. [Figure 2B] FIG. 1 is a block diagram of a processing system for processing sequence reads, in accordance with various embodiments. [Diagram 3] 1 is a flow chart illustrating a process for sequencing a nucleic acid, according to various embodiments. [Figure 4A] FIG. 4 illustrates a portion of the process of FIG. 3 for sequencing nucleic acids to obtain methylation information and methylation state vectors, according to various embodiments. [Figure 4B] FIG. 1 illustrates the generation of a data structure for a control group, according to various embodiments. [Figure 4C] 1 is a flow chart illustrating a process for determining aberrantly methylated fragments from a sample, according to various embodiments. [Diagram 5] FIG. 1 illustrates a block of a reference genome, according to various embodiments. [Figure 6] FIG. 1 illustrates a process for determining features for training a classifier, according to various embodiments. [Figure 7A] FIG. 1 illustrates a confusion matrix showing the accuracy of a classifier, according to various embodiments. [Figure 7B] FIG. 1 illustrates a confusion matrix showing the accuracy of a classifier, according to various embodiments. [Figure 7C] FIG. 1 illustrates a confusion matrix showing the accuracy of a classifier, according to various embodiments. [Figure 8] 1 is a flowchart of a method for model-based featurization, in accordance with various embodiments. [Figure 9A] FIG. 13 illustrates the sensitivity of a primary tissue classifier, according to an embodiment. [Figure 9B] FIG. 13 illustrates the sensitivity of a primary tissue classifier, according to an embodiment. [Figure 10A] FIG. 1 illustrates the sensitivity of the tissue of origin classifier at different cancer stages, according to an embodiment. [Figure 10B] FIG. 1 illustrates the sensitivity of the tissue of origin classifier at different cancer stages, according to an embodiment. [Figure 11] FIG. 1 illustrates a performance grid representing the accuracy of primary tissue localization, according to an embodiment. [Figure 12] FIG. 1 illustrates the accuracy and sensitivity of the primary tissue classifier at different cancer stages, according to an embodiment. [Figure 13A] FIG. 1 illustrates an ROC curve for a primary tissue classifier, according to an embodiment. [Figure 13B] FIG. 1 illustrates an ROC curve for a primary tissue classifier, according to an embodiment. [Figure 14] FIG. 1 is a dataflow diagram for training a model, according to various embodiments. [Figure 15] FIG. 2 illustrates example precision-recall curves for an indeterminate call threshold, according to various embodiments. [Figure 16] 1 is a flowchart of a method for determining the probability that a sample has a disease state, according to various embodiments. [Figure 17] 1 illustrates a performance gain in sensitivity of a multi-layer perceptron model, according to an embodiment. [Figure 18] 14 illustrates experimental results of a multi-layer perceptron model in determining tissue of origin, according to an embodiment. [Figure 19] 1 illustrates experimental results of a multi-layer perceptron model in determining tissue of origin by cancer stage, according to an embodiment. [Figure 20] 1 illustrates experimental results of a multi-layer perceptron model across cancer types, according to an embodiment. [Figure 21] 13 is a graph of the cancer type likelihood for non-cancer samples above 95% specificity. [Figure 22] 1 is a graph of methylation sequencing data of non-cancer samples and hematological subtype cancer samples. [Figure 23A] 1 is a flowchart illustrating a process of determining a binary threshold cutoff for binary cancer classification, according to one or more embodiments. [Figure 23B] 1 is a flowchart illustrating the process of thresholding tissue of origin labels to determine a binary threshold cutoff for binary cancer classification, according to one or more embodiments. [Figure 24A] FIG. 13 illustrates a confusion matrix showing the performance of a trained cancer tissue of origin classifier with additional hematological cancer subtypes. [Figure 24B] FIG. 13 illustrates a confusion matrix showing the performance of a trained cancer tissue of origin classifier with additional hematological cancer subtypes. [Figure 25A] 1 is a graph showing cancer prediction accuracy for cancer classifiers with and without adjustment of threshold cutoffs across cancer stages for a number of cancer types. [Figure 25B] 1 is a graph showing cancer prediction accuracy for cancer classifiers with and without adjustment of threshold cutoffs across cancer stages for a number of cancer types. [Figure 26A] FIG. 1 shows receiver operator curves (ROC) showing sensitivity and specificity of cancer detection using methylation data for the targeted genomic regions of assay panel A. [Figure 26B] FIG. 1 shows a confusion matrix illustrating the accuracy of cancer type classification for subjects determined to have cancer using methylation data for the target genomic regions of assay panel A. [Figure 27A] FIG. 13 shows receiver operator curves (ROC) showing sensitivity and specificity of cancer detection using methylation data for the targeted genomic regions of assay panel B. [Figure 27B] FIG. 13 shows a confusion matrix illustrating the accuracy of cancer type classification for subjects determined to have cancer using methylation data for the target genomic regions of assay panel B. [Figure 28] FIG. 1 shows classifier performance for a proprietary cancer assay panel (Assay Panel C), according to an embodiment. [Figure 29] FIG. 13 shows a tissue of origin (TOO) confusion matrix representing the accuracy of cancer tissue of origin localization for assay panel C, according to an embodiment. [Diagram 30] FIG. 13 shows classifier sensitivity performance by stage in individual tumors for assay panel C, according to embodiments. [Diagram 31] FIG. 13 illustrates tissue of origin accuracy for multiple iterations of a trained model, in accordance with various embodiments. [Diagram 32] FIG. 1 illustrates a process for stratifying hematological signals into two layers, according to various embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0069] Reference will now be made in detail to several embodiments, examples of which are illustrated in the accompanying figures. It should be noted that wherever practicable, similar or similar reference numbers may be used in the figures and may indicate similar or similar functionality. It should also be noted that the contents of all published materials (such as patent applications, patents, papers, and conference proceedings) mentioned herein are incorporated herein by reference in their entirety.

[0070] I. Definition Unless otherwise defined, all technical and scientific terms used herein have the meaning commonly understood by one of ordinary skill in the art to which this description belongs. As used herein, the following terms have the meanings ascribed to them below.

[0071] The term "individual" refers to a human individual. The term "healthy individual" refers to an individual who is presumed to be cancer or disease free.

[0072] The term "subject" refers to an individual whose DNA is being analyzed. The subject may be a test subject whose DNA is evaluated using whole genome sequencing or a targeted panel as described herein to evaluate whether or not the subject has a disease state (e.g., cancer, cancer type, or cancer tissue of origin). The subject may also be a member of a control group known to not have cancer or another disease. The subject may also be a member of a cancer or other disease group known to have cancer or another disease. The control group and the cancer / disease group may be used to assist in the design or validation of a targeted panel.

[0073] The term "reference sample" refers to a sample obtained from a subject with a known disease state.

[0074] The term "training sample" refers to a sample obtained from a known disease state that can be used to generate sequence reads. The training sample can be applied to a probabilistic model to generate features that can be utilized for disease state classification.

[0075] The term "test sample" refers to a sample that may have an unknown disease state.

[0076] The term "sequence read" refers to a nucleotide sequence read from a sample obtained from an individual. A sequence read may be generated from a nucleic acid fragment in a sample. A sequence read may be a collapsed sequence read generated from multiple sequence reads taken from multiple amplicons from a single original nucleic acid molecule. In some embodiments, a sequence read may be a de-duplicated sequence read. A sequence read may be obtained through various methods known in the art.

[0077] The term "disease state" refers to the presence or absence of a disease, the type of disease, and / or the tissue of disease origin. For example, in one embodiment, the present disclosure provides methods, systems, and non-transitory computer-readable media for detecting cancer (i.e., the presence or absence of cancer), the type of cancer, or the tissue of cancer origin.

[0078] The term "tissue of origin" or "TOO" refers to an organ, group of organs, body region, or cell type from which a disease state may arise or originate. For example, identification of the tissue of origin or cancer cell type generally allows one to identify appropriate next steps to further diagnosis, stages, and determine treatment.

[0079] The term "methylation," as used herein, refers to the chemical process by which a methyl group is added to a DNA molecule. Two of the four bases of DNA, cytosine ("C") and adenine ("A"), can be methylated. For example, the hydrogen atom on the pyrimidine ring of the cytosine base can be converted to a methyl group to form 5-methylcytosine. Methylation tends to occur at the dinucleotides of cytosine and guanine, referred to herein as "CpG sites." In other instances, methylation can occur at cytosines that are not part of a CpG site, or at other nucleotides that are not cytosines, however, these occur more rarely. For simplicity in this disclosure, methylation is described with reference to CpG sites. However, the principles described herein are equally applicable to the detection of methylation in non-CpG contexts, including methylation of non-cytosines. For example, adenine methylation has been observed in bacterial, plant, and mammalian DNA, but has received much less attention.

[0080] In such embodiments, the wet lab assays used to detect methylation may differ from those described herein, as is well known in the art. Furthermore, the methylation state vector may include an element that is a vector of sites where methylation generally occurs or does not occur (even if those sites are not specifically CpG sites). With that substitution, the rest of the process described herein remains the same, so that the inventive concepts described herein are applicable to those other forms of methylation.

[0081] The term "CpG site" refers to a region of a DNA molecule that has, along its 5' to 3' linear sequence of bases, a cytosine nucleotide followed by a guanine nucleotide. "CpG" is an abbreviation for 5'-C-phosphate-G-3', in which the cytosine and guanine are separated by only one phosphate group, which links any two nucleotides together in the DNA. The cytosine in a CpG dinucleotide can be methylated to form 5-methylcytosine.

[0082] The term "methylation site" refers to a single site of a DNA molecule to which a methyl group can be added. Although "CpG" sites are the most common methylation sites, methylation sites are not limited to CpG sites. For example, DNA methylation can occur at cytosine in CHG and CHH, where H is adenine, cytosine, or thymine. Cytosine methylation in the form of 5-hydroxymethylcytosine and its characteristic amounts can also be evaluated using the methods and procedures disclosed herein (see, for example, U.S. Patent No. 5,399,233 and U.S. Patent No. 5,399,233, which are incorporated herein by reference). The terms "hypomethylated" or "hypermethylated" refer to the methylation status of a DNA molecule that contains a large number of CpG sites (e.g., more than 3, 4, 5, 6, 7, 8, 9, 10, etc.), where a high percentage of the CpG sites (e.g., greater than 80%, 85%, 90%, or 95%, or any other percentage within the range of 50%-100%) are unmethylated or methylated, respectively.

[0083] The terms "cell-free deoxyribonucleic acid," "cell-free DNA," or "cfDNA" refer to deoxyribonucleic acid fragments that circulate in bodily fluids, such as blood, sweat, urine, or saliva, and that are derived from one or more healthy cells and / or one or more cancer cells.

[0084] The term "circulating tumor DNA" or "ctDNA" refers to deoxyribonucleic acid fragments derived from tumor cells or other types of cancer cells that may be released into an individual's bodily fluids, such as blood, sweat, urine, or saliva, as a result of biological processes such as apoptosis or necrosis of dying cells, or that may be actively released by viable tumor cells.

[0085] II. Methodology Overview FIG. 1 is a flow chart of a method 100 for identifying a plurality of features for generating a classifier for predicting a disease state (e.g., presence or absence of disease, type of disease, and / or tissue of origin of disease) according to various embodiments. FIG. 2B is a block diagram of a processing system 200 for processing sequence reads according to various embodiments. In some embodiments, the processing system 200 executes the method 100 for processing sequence reads of fragments from a nucleic acid sample. The method 100 includes, but is not limited to, the following steps: generating sequence reads; training a probability model associated with each of a plurality of different disease states (e.g., different cancer types); applying the probability model to determine a value based on the probability that the sequence read originates from a sample associated with each of the plurality of disease states associated with each probability model; identifying features by determining a count of sequence reads having a value above a threshold; using the features to generate a classifier; and optionally applying the classifier to predict a disease state and / or tissue of origin associated with the disease state. Each of which is described with reference to FIGS. 2-6 with respect to the components of the processing system 200. In the embodiment shown in FIG. 2B, the processing system 200 includes a sequence processor 210, a machine learning engine 220, a probabilistic model 230, and a classifier 240.

[0086] In step 110, the sequence processor 210 generates a first set of sequence reads from a plurality of samples, each having a known or suspected disease state, such as the presence or absence of disease, the type of disease, and / or the disease tissue of origin. For example, in some embodiments, the plurality of samples can include any number of cancer samples from individuals known to have cancer, and / or non-cancer samples from healthy individuals. In addition, the samples can include any of cell-free nucleic acid samples (e.g., cfDNA), solid tumor samples, and / or other types of samples. As will be appreciated by those skilled in the art, next-generation sequencing procedures may generate multiple sequence reads from a single original nucleic acid molecule. Thus, in some embodiments, the sequence processor 210 can use known methods for removing duplicates and / or collapsing sequence reads to remove duplicate sequence reads and identify a single sequence read for a single original nucleic molecule from which one or more raw sequence reads were generated.

[0087] II.A. Assay Protocol 3 is a flow chart illustrating a process 300 for sequencing nucleic acids, according to an embodiment. In some embodiments, the process 300 is performed to generate sequence reads as part of step 110 of the method 100 of FIG.

[0088] In step 310, a nucleic acid sample (e.g., DNA or RNA) is extracted from a subject. In this disclosure, DNA and RNA can be used interchangeably unless otherwise indicated. That is, the embodiments described herein can be applicable to both DNA and RNA type nucleic acid sequences. However, the examples described herein can focus on DNA for purposes of clarity and explanation. The sample can include nucleic acid molecules taken from any subset of the human genome, including the entire genome. The sample can include blood, plasma, serum, urine, stool, saliva, other types of bodily fluids, or any combination thereof. In some embodiments, methods for taking blood samples (e.g., syringe or finger prick) can be less invasive than procedures for obtaining tissue biopsies, which can require surgical procedures. The extracted sample can include cfDNA and / or ctDNA. If the subject has a disease condition, such as cancer, the cell-free nucleic acid (e.g., cfDNA) in the sample extracted from the subject generally includes detectable levels of nucleic acid that can be used to assess the disease condition.

[0089] In step 315, the extracted nucleic acids (e.g., including cfDNA fragments) are treated to convert unmethylated cytosines to uracil. In some embodiments, the method 300 uses bisulfite treatment of the sample, which converts unmethylated cytosines to uracil without converting methylated cytosines. For example, a commercially available kit such as EZ DNA Methylation™-Gold, EZ DNA Methylation™-Direct or an EZ DNA Methylation™-Lightning kit (available from Zymo Research Corp, Irvine, Calif.) is used for bisulfite conversion. In another embodiment, the conversion of unmethylated cytosines to uracil is achieved using an enzymatic reaction. For example, the conversion can use a commercially available kit for the conversion of unmethylated cytosines to uracil, such as APOBEC-Seq (NEBiolabs, Ipswich, Mass.).

[0090] In step 320, a sequencing library is prepared. In some embodiments, the preparation includes at least two steps. In the first step, ssDNA adaptors are added to the 3'-OH ends of the bisulfite converted ssDNA molecules using a ssDNA ligation reaction. In some embodiments, the ssDNA ligation reaction uses CircLigase II (Epicentre) to ligate ssDNA adaptors to the 3'-OH ends of the bisulfite converted ssDNA molecules, the 5' ends of the adaptors are phosphorylated, and the bisulfite converted ssDNA is dephosphorylated (i.e., the 3' ends have hydroxyl groups). In another embodiment, the ssDNA ligation reaction uses Thermostable 5' AppDNA / RNA ligase (available from New England BioLabs, Ipswich, Massachusetts) to ligate ssDNA adaptors to the 3'-OH ends of the bisulfite converted ssDNA molecules. In this example, the first UMI adaptor is adenylated at the 5' end and blocked at the 3' end. In another embodiment, the ssDNA ligation reaction uses T4 RNA ligase (available from New England BioLabs) to ligate the ssDNA adaptor to the 3'-OH end of the bisulfite converted ssDNA molecule.

[0091] In the second step, the second strand DNA is synthesized in an extension reaction. For example, an extension primer that hybridizes with the primer sequence contained in the ssDNA adaptor is used in a primer extension reaction to form a double-stranded bisulfite converted DNA molecule. Optionally, in some embodiments, the extension reaction uses an enzyme that can read through uracil residues in the bisulfite converted template strand.

[0092] Optionally, in a third step, dsDNA adaptors are added to the double-stranded bisulfite converted DNA molecules. The double-stranded bisulfite converted DNA is then amplified to add sequencing adaptors. For example, PCR amplification using a forward primer containing a P5 sequence and a reverse primer containing a P7 sequence is used to add P5 and P7 sequences to the bisulfite converted DNA. Optionally, during library preparation, unique molecular identifiers (UMIs) can be added to nucleic acid molecules (e.g., DNA molecules) through adaptor ligation. UMIs are short nucleic acid sequences (e.g., 4-10 base pairs) that are added to the ends of DNA fragments during adaptor ligation. In some embodiments, UMIs are degenerate base pairs that function as unique tags that can be used to identify sequence reads derived from specific DNA fragments. During PCR amplification after adapter ligation, the UMI is replicated along with the attached DNA fragment, which provides a way to distinguish sequence reads derived from the same original fragment in downstream analyses.

[0093] In optional step 325, the nucleic acids (e.g., fragments) can be hybridized. Hybridization probes (also referred to herein as "probes") can be used to target and pull down nucleic acid fragments that are informative for a disease state. For a given workflow, a probe can be designed to anneal (or hybridize) with a target (complementary) strand of DNA or RNA. The target strand can be the "positive" strand (e.g., the strand that is transcribed into mRNA and then translated into protein) or the complementary "negative" strand. Probes can range in length from 10s, 100s, or 1000s of base pairs. Additionally, probes can cover overlapping portions of the target region.

[0094] In optional step 330, the hybridized nucleic acid fragments can be captured and enriched, e.g., amplified using PCR. In some embodiments, the target DNA sequence can be enriched from the library. This is used, for example, when a target panel assay is being performed on the sample. For example, the target sequence can be enriched to obtain enriched sequences that can be sequenced later. In general, any method known in the art can be used to separate and enrich the probe-hybridized target nucleic acid. For example, a biotin moiety can be added to the 5' end of the probe (i.e., biotinylated) to facilitate separation of the probe-hybridized target nucleic acid using a streptavidin-coated surface (e.g., streptavidin-coated beads), as is well known in the art.

[0095] In step 335, sequence reads are generated from the nucleic acid sample, e.g., the enriched sequences. Sequencing data can be obtained from the enriched DNA sequences by means known in the art. For example, the method can include next-generation sequencing (NGS) techniques, including synthesis technology (Illumina), pyrosequencing (454 Life Sciences), ion semiconductor technology (Ion Torrent sequencing), single molecule real-time sequencing (Pacific Biosciences), sequencing by ligation (SOLiD sequencing), nanopore sequencing (Oxford Nanopore Technologies), or paired-end sequencing. In some embodiments, massively parallel sequencing is performed using sequencing-by-synthesis with reversible dye terminators.

[0096] In step 340, the sequence processor 210 can generate methylation information using the sequence reads. A methylation state vector can then be generated using the methylation information determined from the sequence reads. FIG. 4B illustrates a process 360 that starts with the process 300 of FIG. 3 of sequencing a cfDNA molecule to obtain a methylation state vector 352, according to an embodiment. As an example, the analysis system receives a cfDNA molecule 312, which in this example includes three CpG sites. As shown, the first and third CpG sites of the cfDNA molecule 312 are methylated 314. During processing step 315, the cfDNA molecule 312 is converted to generate a converted cfDNA molecule 322. During processing 315, the second CpG site that was unmethylated has its cytosine converted to uracil. However, the first and third CpG sites are not converted.

[0097] After conversion, the sequencing library 330 is prepared and sequenced, generating sequence reads 342. The analysis system aligns the sequence reads 342 to a reference genome 344 (not shown), which provides context as to where in the human genome the fragment cfDNA originates. In this simplified example, the analysis system aligns the sequence reads 342 such that three CpG sites correlate to CpG sites 23, 24, and 25 (arbitrary reference identifiers used for convenience of illustration). Thus, the analysis system generates information regarding both the methylation status of all CpG sites on the cfDNA molecule 312 and the location in the human genome to which the CpG sites map. As shown, the CpG sites on the methylated sequence reads 342 are read as cytosines. In this example, cytosine appears only at the first and third CpG sites in sequence read 342, which allows one to infer that the first and third CpG sites in the original cfDNA molecule were methylated. Meanwhile, the second CpG site is read as thymine (U is converted to T during the sequence read process), and therefore one can infer that the second CpG site in the original cfDNA molecule was unmethylated. Using these two pieces of information, i.e., methylation status and position, analysis system 200 generates a methylation state vector 352 for fragment cfDNA 312. In this example, the resulting methylation state vector 352 is <M 23 , U 24 , M 25 >, where M corresponds to a methylated CpG site, U corresponds to an unmethylated CpG site, and the subscript numbers correspond to the position of each CpG site in the reference genome.

[0098] II.B. Identification of Aberrant Fragments In some embodiments, the analysis system uses the methylation state vector of the sample to determine abnormal fragments for the sample. For example, for each nucleic acid molecule or fragment in the sample, the analysis system uses the methylation state vector corresponding to the nucleic acid molecule to determine (through analysis of sequence reads extracted therefrom) whether the nucleic acid molecule or fragment is an abnormally methylated molecule or fragment compared to the expected methylation state vector from a healthy sample. In one embodiment, the analysis system calculates a p-value score for each methylation state vector (e.g., as described in U.S. Patent No. 6,399,323, incorporated herein by reference) that describes the probability of observing that methylation state vector or the probability of observing other methylation state vectors that are less likely in a healthy control group. The process for calculating the p-value score is also described below in Section II. BiP-Value Filtering. The analysis system may determine, and optionally filter out, sequence reads of nucleic acid molecules or fragments that have methylation state vectors with p-value scores below a threshold to be abnormal fragments. In another embodiment, the analysis system further labels fragments having at least some number of CpG sites with a percentage of methylation or unmethylation above some threshold as hypermethylated and hypomethylated fragments, respectively. Hypermethylated and hypomethylated fragments may also be referred to as unusual fragments with extreme methylation (UFXM). In other embodiments, the analysis system may implement a variety of other probabilistic models for determining abnormal molecules or fragments. Examples of other probabilistic models include mixture models, deep probabilistic models, and the like. In some embodiments, the analysis system may use any combination of the processes described below for identifying abnormal fragments. With the identified abnormal fragments, the analysis system may filter a set of methylation state vectors for the sample for use in other processes, for example, for use in training and deploying a cancer classifier.

[0099] II. BIP value filtering In one embodiment, the analysis system calculates a p-value score for each methylation state vector that is compared to the methylation state vectors from the fragments in the healthy control group. The p-value score describes the probability of observing a nucleic acid molecule in the healthy control group that has a methylation status consistent with that methylation state vector. To determine that a DNA fragment is abnormally methylated, the analysis system uses a healthy control group in which the majority of fragments are normally methylated. When performing this probabilistic analysis to determine abnormal fragments, the decision carries weight compared to the group of controls that make up the healthy control group. To ensure the robustness of the healthy control group, the analysis system may select some threshold number of healthy individuals from which to source samples containing DNA fragments. Figure 4B below describes how to generate a data structure for the healthy control group that the analysis system can use to calculate a p-value score. Figure 4C describes how to use the generated data structure to calculate a p-value score.

[0100] 4B is a flow chart illustrating a process 400 for generating a data structure for a healthy control group, according to an embodiment. To create the healthy control group data structure, the analysis system receives a plurality of DNA fragments (e.g., cfDNA) from a plurality of healthy individuals. A methylation state vector is identified for each fragment, e.g., via process 360.

[0101] Using the methylation state vector of each fragment, the analysis system subdivides 405 the methylation state vector into strings of CpG sites. In one embodiment, the analysis system subdivides 405 the methylation state vector such that the resulting strings are all smaller than a given length. For example, a methylation state vector of length 11, which can be subdivided into strings of lengths 3 or less, results in 9 strings of length 3, 10 strings of length 2, and 11 strings of length 1. In another example, a methylation state vector of length 7, which can be subdivided into strings of lengths 4 or less, results in 4 strings of length 4, 5 strings of length 3, 6 strings of length 2, and 7 strings of length 1. If the methylation state vector is shorter than or equal to the specified string length, the methylation state vector can be converted to a single string that contains all of the CpG sites in the vector.

[0102] For each possible CpG site and possible methylation state in the vector, the analysis system 200 tabulates 410 the strings by counting the number of strings present in the control group that have the specified CpG site as the first CpG site in the string and have that possible methylation state. For example, given a string length of 3 at a given CpG site, 3 For each of the eight possible string configurations at that given CpG site, the analysis system tallies 410 how many times each occurrence of each methylation state vector possibility occurred in the control group. Continuing with the example, this translates to the following quantities for each starting CpG site x in the reference genome: <M x ,M x+1 ,M x+2 >, <M x ,M x+1 ,U x+2 >,..., x ,U x+1 ,U x+2 ​The analysis system creates 415 a data structure that stores the tabulated counts for each starting CpG site and string possibility.

[0103] There are several benefits to placing an upper limit on string length. First, depending on the maximum length for a string, the size of the data structures created by the analysis system can increase dramatically in size. For example, a maximum string length of 4 means that every CpG site must account for at least 2 4 Increasing the maximum string length to 5 means that every CpG site has an additional 2 4 or 16 numbers, doubling the number of counts (and computer memory required) compared to the previous string length. Reducing the string size helps keep data structure creation and execution (e.g., use for later access as described below) reasonable in terms of computation and storage. Second, a statistical consideration in limiting the maximum string length is to avoid overfitting downstream models that use string counts. If long strings of CpG sites do not have a strong biological effect on the outcome (e.g., predicting abnormalities that predict the presence of cancer), calculating a probability based on a large string of CpG sites can be problematic because it requires a significant amount of data that may not be available, and therefore the data becomes too sparse for the model to work properly. For example, calculating the probability of abnormality / cancer conditional on the previous 100 CpG sites requires counting strings in a data structure of length 100, ideally some that match exactly the methylation state of the previous 100. If only a sparse count of strings of length 100 is available, there is insufficient data to determine whether a given string of length 100 in a test sample is abnormal.

[0104] 4C is a flow chart illustrating a process 420 for identifying aberrantly methylated fragments from an individual, according to an embodiment. In process 420, the analysis system generates methylation state vectors 352 from the subject's cfDNA fragments. The analysis system processes each methylation state vector as follows.

[0105] For a given methylation state vector, the analysis system enumerates 430 all possibilities for methylation state vectors that have the same starting CpG site and the same length (i.e., set of CpG sites) as in the methylation state vector. Since each methylation state is generally either methylated or unmethylated, there are effectively two possible states for each CpG site, and therefore the count of different possibilities for a methylation state vector is calculated by dividing the n possible states by the n possible states for a methylation state vector of length n. n The probability of a methylation state vector depends on a power of 2 so that it is associated with a probability of 1. If one or more CpG sites has a methylation state vector that includes an uncertain state, the analysis system may enumerate 430 the possibilities of the methylation state vector considering only CpG sites that have an observed state.

[0106] The analysis system 200 calculates 440 the probability of observing each possible methylation state vector for the identified starting CpG site and methylation state vector length by accessing the healthy control data structure. In one embodiment, calculating the probability of observing a given possibility uses Markov chain probability to model the joint probability calculation. In other embodiments, a calculation method other than Markov chain probability is used to determine the probability of observing each possible methylation state vector.

[0107] The analysis system uses the calculated probability for each possibility to calculate 450 a p-value score for the methylation state vector. In one embodiment, this involves identifying a calculated probability that corresponds to the likelihood of matching the methylation state vector in question. Specifically, this is the likelihood of having the same set of CpG sites as the methylation state vector, or also the same starting CpG site and length. The analysis system sums the calculated probabilities of all possibilities that have a probability less than or equal to the identified probability to generate a p-value score.

[0108] This p-value represents the probability of observing the methylation state vector of the fragment, or other methylation state vectors that are less likely in the healthy control group. Thus, a low p-value score generally corresponds to a methylation state vector that is rare in healthy individuals, causing the fragment to be labeled as abnormally methylated compared to the healthy control group. A high p-value score generally relates to a methylation state vector that is expected to be present in healthy individuals in a relative sense. For example, if the healthy control group is a non-cancer group, a low p-value indicates that the fragment is abnormally methylated compared to the non-cancer group, and therefore likely indicates the presence of cancer in the test subject.

[0109] As described above, the analysis system calculates a p-value score for each of a plurality of methylation state vectors, each representing a cfDNA fragment in a test sample. To identify which of the fragments are abnormally methylated, the analysis system may filter 460 the set of methylation state vectors based on their p-value scores. In one embodiment, the filtering is performed by comparing the p-value scores to a threshold and retaining only the fragments that are below the threshold. This threshold p-value score can be on the order of 0.1, 0.01, 0.001, or 0.0001, etc.

[0110] According to an exemplary result from the process, the analysis system yielded a median of 2800 fragments with abnormal methylation patterns (range 1500-12000 fragments) for participants without cancer who participated in the training, and a median of 3000 fragments with abnormal methylation patterns (range 1200-220000 fragments) for participants with cancer who participated in the training. These filtered sets of fragments with abnormal methylation patterns can be used for downstream analysis as described below.

[0111] In one embodiment, the analysis system uses a sliding window 455 to determine the probabilities of the methylation state vector and to calculate p-values. Instead of enumerating the probabilities and calculating p-values ​​for the entire methylation state vector, the analysis system enumerates the probabilities and calculates p-values ​​only over a window of contiguous CpG sites, where the window is shorter in length (of CpG sites) than at least some fragment (otherwise the window serves no purpose). The window length may be static, user determined, dynamic, or otherwise selected.

[0112] When calculating p-values ​​for a methylation state vector larger than a window, the window identifies a set of contiguous CpG sites from the vector within the window, starting from the first CpG site in the vector. The analysis system calculates a p-value score for the window that includes the first CpG site. The analysis system then "slides" the window to a second CpG site in the vector and calculates another p-value score for the second window. Thus, for a window size of l and a methylation vector length of m, each methylation state vector generates m-l+1 p-value scores. After completing the p-value calculations for each portion of the vector, the lowest p-value score from all sliding windows is taken as the overall p-value score for the methylation state vector. In another embodiment, the analysis system aggregates the p-value scores for the methylation state vector to generate an overall p-value score.

[0113] The use of a sliding window helps to reduce the number of possible methylation state vectors to be enumerated and their corresponding probability calculations that would otherwise need to be performed. To give a realistic example, a fragment can have more than 54 CpG sites. To generate a single p-score, 2 54 (approx. 1.8×10 16 Instead of calculating probabilities for 20 possibilities, the analysis system could instead use a window of size 5 (for example), which would result in 50 p-value calculations for each of the 50 windows of the methylation state vector for that fragment. 5 List (32) possibilities, the total of which is 50 x 2 5 (1.6×10 3 ) probability calculations. This results in a significant reduction in the calculations performed without having a meaningful hit for correct identification of the anomalous fragment.

[0114] In embodiments with uncertain states, the analysis system may calculate a p-value score that sums out the CpG sites with uncertain states in the methylation state vector of the fragment. The analysis system identifies all possibilities that have a match with all methylation states in the methylation state vector, excluding the uncertain states. The analysis system may assign a probability to the methylation state vector as the sum of the probabilities of the identified possibilities. By way of example, the analysis system may<M1,M2,U3> and<M1,U2,U3> The methylation state vector<M1,I2,U3> The probability of 1 being the most likely candidate for CpG sites 1 and 3 is calculated because the methylation states for CpG sites 1 and 3 are observed and match the methylation states of the fragments at CpG sites 1 and 3. The method for summing out CpG sites with uncertain states is to iThe method uses a calculation of the probability of the possibility of i = 1, where i denotes the number of uncertain states in the methylation state vector. In a further embodiment, a dynamic programming algorithm may be implemented to calculate the probability of the methylation state vector having one or more uncertain states. Advantageously, the dynamic programming algorithm operates in linear computation time.

[0115] In one embodiment, the computational load of calculating the probability and / or p-value scores may be further reduced by caching at least some of the calculations. For example, the analysis system may cache in temporary or persistent memory the calculation of the probability for the likelihood of a methylation state vector (or a window thereof). Caching the likelihood probabilities allows for efficient calculation of p-score values ​​without needing to recalculate the underlying likelihood probabilities when other fragments have the same CpG sites. Similarly, the analysis system may calculate a p-value score for each of the possibilities of the methylation state vector associated with a set of CpG sites from the vector (or a window thereof). The analysis system may cache the p-value scores for use in determining the p-value scores of other fragments containing the same CpG sites. In general, the p-value scores of the possibilities of methylation state vectors with the same CpG sites may be used to determine the p-value scores of different ones of the possibilities from the same set of CpG sites.

[0116] II.B.II. Hypermethylated and Hypomethylated Fragments In some embodiments, the analysis system determines an aberrant fragment as one having more than a threshold number of CpG sites, more than a threshold percentage of CpG sites being methylated, or more than a threshold percentage of CpG sites being unmethylated, and the analysis system identifies such fragments as hypermethylated or hypomethylated. Exemplary thresholds for fragment (or CpG site) length include greater than 3, greater than 4, greater than 5, greater than 6, greater than 7, greater than 8, greater than 9, greater than 10, etc. Exemplary percentage thresholds for methylation or unmethylation include greater than 80%, greater than 85%, greater than 90%, or greater than 95%, or any other percentage in the range of 50%-100%.

[0117] II.C. Exemplary Sequencer and Analysis System 2A and 2B are flow charts of a system and device for sequencing a nucleic acid sample, according to one embodiment. This exemplary flow chart includes devices such as a sequencer 270 and an analysis system 200. The sequencer 270 and the analysis system 200 may operate in conjunction to perform one or more steps in the processes described herein.

[0118] In various embodiments, the sequencer 270 receives the enriched nucleic acid sample 260. As shown in FIG. 2A, the sequencer 270 can include a graphical user interface 275 that allows user interaction with a particular task (e.g., start sequencing or end sequencing), as well as one or more loading stations 280 for loading a sequencing cartridge containing the enriched fragment sample and / or loading the necessary buffers for performing a sequencing assay. Thus, once a user of the sequencer 270 provides the necessary reagents and sequencing cartridge to the loading station 280 of the sequencer 270, the user can initiate sequencing by interacting with the graphical user interface 275 of the sequencer 270. Once initiated, the sequencer 270 performs sequencing and outputs sequence reads of the enriched fragments from the nucleic acid sample 260.

[0119] In some embodiments, sequencer 270 is communicatively coupled to analysis system 200. Analysis system 200 includes a number of computing devices used to process sequence reads for various applications, such as evaluation of methylation status at one or more CpG sites, variant calling, or quality control. Sequencer 270 may provide sequence reads to analysis system 200 in BAM file format. Analysis system 200 can be communicatively coupled to sequencer 270 through wireless, wired, or a combination of wireless and wired communication techniques. In general, analysis system 200 is configured to include a processor and a non-transitory computer-readable storage medium that stores computer instructions that, when executed by the processor, cause the processor to process sequence reads or perform one or more steps of any of the methods or processes disclosed herein.

[0120] In some embodiments, the sequence reads may be aligned to a reference genome using methods known in the art to determine alignment position information. The alignment positions may generally describe the start and end positions of a region in the reference genome that corresponds to the start and end nucleotide bases of a given sequence read. In response to methylation sequencing, the alignment position information may be generalized to indicate the first and last CpG sites included in the sequence read according to the alignment to the reference genome. The alignment position information may further indicate the methylation status and the positions of all CpG sites in a given sequence read. Regions in the reference genome may be associated with genes or segments of genes, such that the analysis system 200 may label the sequence read with one or more genes that are aligned to the sequence read. In one embodiment, the length (or size) of the fragment is determined from the start and end positions.

[0121] In various embodiments, for example, when a paired-end sequencing process is used, the sequence read is composed of a read pair, referred to as R_1 and R_2. For example, the first read R_1 may be sequenced from a first end of a double-stranded DNA (dsDNA) molecule, while the second read R_2 may be sequenced from a second end of the double-stranded DNA (dsDNA). Thus, the nucleotide base pairs of the first read R_1 and the second read R_2 may be consistently (e.g., in opposite directions) aligned with the nucleotide bases of the reference genome. The alignment position information retrieved from the read pair R_1 and R_2 may include a start position in the reference genome corresponding to the end of the first read (e.g., R_1) and an end position in the reference genome corresponding to the end of the second read (e.g., R_2). In other words, the start position and the end position in the reference genome represent the likely positions in the reference genome to which the nucleic acid fragment corresponds. In one embodiment, the read pair R_1 and R_2 can be assembled into fragments, which are used for subsequent analysis and / or classification. An output file having a SAM (Sequence Alignment Map) format or a BAM (Binary) format can be generated and output for further analysis.

[0122] 2B, which is a block diagram of an analysis system 200 for processing DNA samples, according to one embodiment. The analysis system implements one or more computing devices for use in analyzing DNA samples. Analysis system 200 includes a sequence processor 210, a sequence database 215, a model database 225, one or more probabilistic models 230 and / or one or more classifiers 240, and a parameter database 235. In some embodiments, analysis system 200 performs one or more steps in a method or process disclosed herein.

[0123] The sequence processor 210 generates methylation state vectors for fragments from the sample. For each CpG site on the fragment, the sequence processor 210 generates a methylation state vector for each fragment via process 360 of FIG. 4B that specifies the location of the fragment in the reference genome, the number of CpG sites in the fragment, and the methylation state of each CpG site in the fragment, i.e., methylated, unmethylated, or uncertain. The sequence processor 210 may store the methylation state vectors for the fragments in a sequence database 215. The data in the sequence database 215 may be organized such that the methylation state vectors from the samples are associated with each other.

[0124] Additionally, multiple different models 230 may be stored in model database 225 or retrieved for use with test samples. In one example, the model is a trained cancer classifier 240 to determine a cancer prognosis for a test sample using feature vectors derived from abnormal fragments. The training and use of cancer classifiers is described elsewhere herein. Analysis system 200 may train one or more models 230 and / or one or more classifiers 240 and store the various trained parameters in parameter database 235. Analysis system 200 stores models 230 and / or classifiers along with functions in model database 225.

[0125] During inference, the machine learning engine 220 uses one or more models 230 and / or classifiers 240 to return outputs. The machine learning engine accesses the models 230 and / or classifiers 240 in the model database 225 along with trained parameters from the parameter database 235. According to each model, the machine learning engine 220 receives inputs appropriate for the model and calculates outputs based on the received inputs, parameters, and functions of each model that link the inputs and outputs. In some use cases, the machine learning engine 220 further calculates metrics that correlate with confidence for the calculated outputs from the models. In other use cases, the machine learning engine 220 calculates other intermediate values ​​for use in the models.

[0126] II.B. Reference Genome Blocks 5 is a diagram of blocks of a reference genome according to one embodiment. The sequence processor 210 can partition the reference genome (or a subset of the reference genome) in one or more stages, for use cases including, for example, targeted methylation assays. For example, the sequence processor 210 separates the reference genome into blocks of CpG sites. Each block is defined when there is a separation between two adjacent CpG sites that exceeds a threshold value, for example, 200 base pairs (bp), 300 bp, 400 bp, 500 bp, 600 bp, 700 bp, 800 bp, 900 bp, or more than 1,000 bp, among other values. Thus, the blocks can vary in size in base pairs. For each block, the sequence processor 210 can subdivide the block into windows of a certain length, e.g., 500 bp, 600 bp, 700 bp, 800 bp, 900 bp, 1,000 bp, 1,100 bp, 1,200 bp, 1,300 bp, 1,400 bp, or 1,500 bp, among other values. In other embodiments, the windows can be 200 bp to 10 kilobase pairs (kbp) in length, 500 bp to 2 kbp, or about 1 kbp. The windows (e.g., adjacent ones) can overlap by a number of base pairs or a percentage of their length, e.g., 10%, 20%, 30%, 40%, 50%, or 60%, among other values. The window can be divided between two adjacent CpG sites that exceed a threshold value, for example, 200 base pairs (bp), 300 bp, 400 bp, 500 bp, 600 bp, 700 bp, 800 bp, 900 bp, or more than 1,000 bp, among other values.

[0127] The sequence processor 210 can analyze sequence reads derived from DNA fragments using windowing. In particular, the sequence processor 210 scans the block window by window and reads the fragments within each window. The fragments can originate from tissues and / or high signal cfDNA. High signal cfDNA samples can be determined by a binary classification model, by cancer stage, or by another metric. By partitioning the reference genome (e.g., using blocks and windows), the sequence processor 210 can facilitate computational parallelization. Furthermore, the sequence processor 210 can reduce computational resources for processing the reference genome by targeting sections of base pairs that contain CpG sites while skipping other sections that do not contain CpG sites.

[0128] III. Model-Based Feature Engineering and Classification III.A. Model-Based Feature Engineering According to one embodiment, as shown in Figure 8, the present disclosure is directed to model-based feature engineering to derive features useful for classifying disease states. As described elsewhere herein, the disease state can be the presence or absence of a disease, a type of disease, and / or a tissue of origin. For example, as described herein, the disease state can be the presence or absence of cancer, a type of cancer, and / or a tissue of origin of the cancer. The cancer type and / or cancer tissue of origin may be selected from the group including breast cancer, uterine cancer, cervical cancer, ovarian cancer, bladder cancer, urothelial carcinoma of the renal pelvis, non-urothelial renal cancer, prostate cancer, anorectal cancer, colorectal cancer, esophageal cancer, gastric cancer, hepatobiliary carcinoma arising from hepatocytes, hepatobiliary carcinoma arising from cells other than hepatocytes, pancreatic cancer, squamous cell carcinoma of the upper gastrointestinal tract, non-squamous upper gastrointestinal tract cancer, head and neck cancer, lung cancer such as lung adenocarcinoma, small cell lung cancer, squamous cell lung cancer, and cancer other than adenocarcinoma or small cell lung cancer, neuroendocrine carcinoma, melanoma, thyroid cancer, sarcoma, multiple myeloma, lymphoma, and leukemia, among other cancer types.

[0129] In step 810, a first plurality of sequence reads is generated from a first reference sample having a first disease state, and a second plurality of sequence reads is generated from a second reference sample having a second disease state, as described elsewhere herein. The first plurality of sequence reads and / or the second plurality of sequence reads can be more than 10,000, more than 50,000, more than 100,000, more than 200,000, more than 500,000, more than 1,000,000, more than 2,000,000, more than 5,000,000, or more than 10,000,000 sequence reads. As used herein, a "reference sample" is a sample obtained from a subject having a known disease state. In some embodiments, one or more reference samples having one or more known disease states can be used to train one or more probability models, which can then be used to derive features for classifying the disease state of an unknown test sample. The sample can be a genomic DNA (gDNA) sample or a cell-free DNA (cfDNA) sample. The reference sample can be a blood, plasma, serum, urine, feces, and saliva sample. Alternatively, the reference sample can be a whole blood, blood fraction, tissue biopsy sample, pleural effusion, pericardial fluid, cerebrospinal fluid, and peritoneal fluid. In some embodiments, the first reference sample is obtained from a subject known to have cancer, and the second reference sample is obtained from a healthy or non-cancer subject. In some embodiments, the first reference sample is obtained from a subject known to have a first type of cancer (e.g., lung cancer), and the second reference sample is obtained from a subject known to have a second type of cancer (e.g., breast cancer). In yet other embodiments, the first reference sample is obtained from a subject known to have a first diseased tissue of origin (e.g., lung disease), and the second reference sample is obtained from a second diseased tissue of origin (e.g., liver disease).

[0130] In step 815, the machine learning engine 220 trains a first probabilistic model 230 and a second probabilistic model 230 from the first and second plurality of sequence reads (generated in step 110), respectively, where each probabilistic model is associated with a different disease state of one or more possible disease states. As described above, the disease state can be the presence or absence of cancer, the type of cancer, and / or the cancer tissue of origin. In various embodiments, the training data is split into K subsets (folds) for K-fold cross-validation. The folds can be balanced for cancer / non-cancer status, tissue of origin, cancer stage, age (e.g., grouped into 10-year buckets), sex, ethnicity, and smoking status, among other factors. Data from K-1 of the folds can be used as training data for the probabilistic models, and the folds that hold out can be used as test data.

[0131] The machine learning engine 220 trains the first and second probabilistic models 230 for the first and second disease states, respectively, by fitting each of the probabilistic models 230 to the first and second plurality of sequence reads, respectively. For example, in one embodiment, the first probabilistic model is fitted using a first plurality of sequence reads derived from one or more samples from a subject known to have cancer, and the second probabilistic model is fitted using a second plurality of sequence reads derived from one or more samples from a healthy or non-cancer subject. In other embodiments, the first probabilistic model can be trained for a first type of cancer or a first primary tissue, and the second probabilistic model can be trained for a second type of cancer or a second primary tissue. As one skilled in the art will appreciate, any number of disease state probabilistic models can be trained using sequence reads derived from one or more samples taken from subjects having any one of several possible disease states. For example, in some embodiments, as described elsewhere herein, additional cancer-specific probability models (i.e., for additional types of cancer and / or tissue of origin models) can be trained on a third, fourth, fifth, sixth, seventh, eighth, ninth, tenth, etc. (e.g., up to 20, 30 or more) specific types of cancer and used to determine the probability that a sequence read from the training set, or an unknown cancer type, is more likely to be derived from one cancer type (or cancer tissue of origin) rather than another cancer type (or cancer tissue of origin).

[0132] As used herein, a "probability model" is any mathematical model capable of assigning a probability to a sequence read based on the methylation status at one or more sites on the read. During training, the machine learning engine 220 can be used to fit sequence reads derived from one or more samples from subjects with known disease and utilize methylation information or methylation state vectors (e.g., as described above in connection with Figures 3-4) to determine sequence read probabilities indicative of a disease state. In particular, in one embodiment, the machine learning engine 220 determines an observed ratio of methylation for each CpG site in the sequence read. The methylation ratio represents the proportion or percentage of base pairs that are methylated within a CpG site. The trained probability model 230 can be parameterized by a product of methylation ratios. In general, any known probability model for assigning probabilities to sequence reads from a sample can be used. For example, the probability model can be a binomial model, in which every site (e.g., a CpG site) on a nucleic acid fragment is assigned a probability of methylation, or an independent site model, in which the methylation of each CpG is specified by a distinct methylation probability, where methylation at one site on a nucleic acid fragment is assumed to be independent of methylation at one or more other sites.

[0133] In some embodiments, the probabilistic model 230 is a Markov model in which the probability of methylation at each CpG site depends on the methylation state at some number of preceding CpG sites in the sequence read or the nucleic acid molecule from which the sequence read is derived. See, e.g., U.S. Patent Application Publication No. 2019 / 0133634, entitled “Anomalous Fragment Detection and Classification,” filed March 13, 2019.

[0134] In some embodiments, the probabilistic model 230 is a "mixture model" that is fitted using a mixture of components from the underlying models. For example, in some embodiments, the mixture components can be determined using a multiple independent site model in which the methylation (e.g., methylation ratio) at each CpG site is assumed to be independent of the methylation at other CpG sites. Using the independent site model, the probability assigned to a sequence read or the nucleic acid molecule from which it is derived is the product of the methylation probability at each CpG site where the sequence read is methylated, and 1 minus the methylation probability at each CpG site where the sequence read is unmethylated. According to this embodiment, the machine learning engine 220 determines the methylation ratio of each of the mixture components. The mixture model is parameterized by the sum of the mixture components, each associated with a product of the methylation ratios. A probabilistic model Pr of n mixture components can be expressed as:

[0135]

number

[0136] For an input fragment, m i ∈{0,1} represents the observed methylation status of the fragment at position i in the reference genome, with 0 indicating unmethylated and 1 indicating methylated. The partial assignment for each mixture component k is k where f k ≧0 and

[0137]

number

[0138] f k = 1. The probability of methylation at position i within a CpG site in mixture component k is β ki Therefore, the probability of non-methylation is 1-β ki The number of mixture components n can be 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, etc.

[0139] In some embodiments, the machine learning engine 220 finds a set of parameters {β ki ,f k To identify {{overscore (N)}}, a probabilistic model 230 is fitted using maximum likelihood estimation. The maximized quantity for the N total fragments can be expressed as:

[0140]

number

[0141] As will be appreciated by those skilled in the art, other means can be used to fit a probabilistic model or to identify parameters that maximize the log likelihood of all sequence reads derived from a reference sample. For example, in one embodiment, Bayesian fitting (e.g., using Markov chain Monte Carlo) is used, in which each parameter is not assigned a single value, but instead is associated with a distribution. In another embodiment, gradient-based optimization is used, in which the gradient of the likelihood (or log likelihood) versus parameter value is used to step through parameter space toward an optimum. In another embodiment, expectation maximization is used, in which a set of latent parameters (such as the identification of the mixture components from which each fragment is derived) is set to their expected values ​​under previous model parameters, and then the parameters of the model are assigned to maximize the likelihood condition proposition for the assumed values ​​of these latent variables. This two-step process is then repeated until convergence.

[0142] In step 820, a plurality of training sequence reads are generated from the training sample. The plurality of training sequence reads can be more than 10,000, more than 50,000, more than 100,000, more than 200,000, more than 500,000, more than 1,000,000, more than 2,000,000, more than 5,000,000, or more than 10,000,000 sequence reads. As used herein, a "training sample" is a sample obtained from a known disease state that can be used to generate sequence reads and then applied to the first and / or second probability models to generate features that can be utilized for disease state classification. In step 825, the processing system 200 applies the first and second probability models 230 to determine a first probability value and a second probability value for each sequence read of the plurality of training sequence reads. The first and second probability values ​​are determined based on the probability that the sequence read originates from a sample associated with the first disease state and the second disease state, respectively. The processing system 200 can repeat step 130 for any additional probability models 230 (e.g., trained from sequence reads from third, fourth, fifth, etc. reference samples) (not shown).

[0143] In step 830, one or more features are identified by comparing the first probability value and the second probability value for each of the plurality of training sequence reads. In general, a wide variety of methods can be utilized to compare the first and second probability values ​​and identify the features. For example, in one embodiment, the one or more features include a count of outlier sequence reads of the plurality of training sequence reads where the first probability value is greater than the second probability value. The count can be a binary count, a total count of outlier sequence reads, or a total count of anonymously methylated sequence reads. In another embodiment, the one or more features include a count of sequence reads or fragments that include a particular methylation pattern. For example, the one or more features can be a count of sequence reads or fragments that are fully methylated at each CpG site, a count of sequence reads or fragments that are partially methylated (e.g., at least 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, or 95% methylated). In another embodiment, the one or more features are identified using the output of a discriminatory classifier trained within the single genomic region (e.g., the discriminatory classifier can be a multi-layer perceptron or a convolutional neural net model). In another embodiment, comparing the first probability value to the second probability value includes determining a ratio of the first probability value to the second probability value, and the one or more features include sequence read counts of sequence reads that exceed a ratio threshold.

[0144] In another embodiment, the first probability value or the second probability value is a log likelihood value. For example, the processing system 200 can calculate the log likelihood ratio R that the fitted probability model is associated with the first and second disease states, respectively. Specifically, the log likelihood ratio can be calculated using the probability Pr of observing the methylation pattern on the fragment for the sample associated with the first disease state and the second disease state.

[0145]

number

[0146] The processing system 200 can use multiple tiered thresholds to identify features. For example, the tiers include thresholds of 1, 2, 3, 4, 5, 6, 7, 8, and 9. In some embodiments, a smoothing function can be applied. For example, in response to determining that R is (e.g., significantly) less than the tier value, the processing system 200 assigns a feature value of about 0, in response to determining that R is equal to the tier value, the processing system 200 assigns a feature value of 0.5, and in response to determining that R is (e.g., significantly) greater than the tier value, the processing system 200 assigns a feature value of about 1. Each tier represents a varying threshold at which a fragment (from which the sequence read was generated) is more likely to be derived from a sample associated with a disease state than a healthy sample. The processing system 200 can use the thresholds to determine a count of outlier fragments, which can be used as a feature.

[0147] By filtering with a threshold, the processing system 200 can consider some fragments as outliers because they are less likely to be present in a healthy sample. Thus, outlier fragments can be considered more likely to be associated with (e.g., originate from) a disease state or a cancer sample. The number of features can vary between different tiers, e.g., one tier can have a different number of features than another tier based on corresponding thresholds. In other embodiments, the processing system 200 uses a different number of tiers or other thresholds. Other means for identifying features based on a measure of the feature's ability to distinguish between different disease states (e.g., using mutual information to determine a measure of the information content of the feature in distinguishing between two disease states) or for ranking the identified features are described elsewhere herein.

[0148] In other embodiments, the processing system 200 can identify multiple features using different types of ratios or formulas. The machine learning engine 220 can determine that a fragment is indicative of a disease state (e.g., cancer) based on whether at least one of the log-likelihood ratios attributed to various disease states is higher than a threshold.

[0149] The plurality of features can then be used to train a disease state classifier, as described in further detail elsewhere herein. For example, in some embodiments, the plurality of features can be used to train a classifier to classify the presence or absence of cancer, the type of cancer, and / or the tissue of cancer origin.

[0150] III.B. Disease Status Tissue of Origin Classification According to another embodiment, as shown in step 120 of FIG. 1, the machine learning engine 220 trains probabilistic models 230, each associated with a different disease state of a set of disease states. For ease of understanding, FIG. 1 depicts model-based feature quantification and training of a classifier for classifying disease state tissue of origin. However, as described above, in various embodiments, the disease state can be the presence or absence of cancer, the type of cancer, and / or the tissue of origin of the cancer. Additionally, the disease state can be associated with another type of disease (not necessarily associated with cancer) or a healthy state (absence of cancer or disease).

[0151] The machine learning engine 220 trains the probabilistic model 230 using the one or more sets of sequence reads, each of which is generated (according to step 110) from a different disease state of the set of multiple disease states. The disease states can include any number of cancer types or cancer primary tissues selected from the group including breast cancer, uterine cancer, cervical cancer, ovarian cancer, bladder cancer, urothelial carcinoma of the renal pelvis, non-urothelial renal cancer, prostate cancer, anorectal cancer, colorectal cancer, esophageal cancer, gastric cancer, hepatobiliary carcinoma arising from hepatocytes, hepatobiliary carcinoma arising from cells other than hepatocytes, pancreatic cancer, squamous cell carcinoma of the upper gastrointestinal tract, non-squamous upper gastrointestinal tract cancer, head and neck cancer, lung cancer, such as lung adenocarcinoma, small cell lung cancer, squamous cell lung cancer, and cancer other than adenocarcinoma or small cell lung cancer, neuroendocrine cancer, melanoma, thyroid cancer, sarcoma, multiple myeloma, lymphoma, and leukemia, among other cancer types.

[0152] The machine learning engine 220 trains the probabilistic model 230 for each of a plurality of disease states by fitting the probabilistic model 230 to sequence reads derived from each sample corresponding to each of the disease states. For example, in some embodiments, the probabilistic model may be trained for a specific type of cancer. According to this embodiment, a cancer-specific probabilistic model may be trained for a first, second, third, etc. specific type of cancer and used to assess the cancer type (e.g., of an unknown test sample). For example, a lung cancer-specific probabilistic model is fitted using a set of sequence reads derived from one or more samples associated with lung cancer. As another example, a breast cancer-specific probabilistic model is fitted using a set of sequence reads derived from one or more samples associated with breast cancer. In some embodiments, a tissue-specific probabilistic model may be trained for a first, second, third, etc. tissue type and used to assess the disease state tissue of origin. For example, a first tissue of origin probability model can be fitted using a set of sequence reads derived from a first tissue type (e.g., from a lung tissue sample, such as a lung biopsy sample), and a second tissue of origin probability model can be fitted using a set of sequence reads derived from a second tissue type (e.g., from a liver tissue sample, such as a liver biopsy sample). Alternatively, in some embodiments, a cancer probability model is fitted using a set of sequence reads derived from one or more samples from a subject known to have cancer, and a non-cancer specific probability model is fitted using a set of sequence reads derived from one or more samples from a healthy or non-cancer subject. As one skilled in the art will appreciate, any number of disease state probability models can be trained utilizing sequence reads derived from one or more samples taken from a subject having any one of a number of possible disease states.For example, in some embodiments, multiple sequence reads can be generated from 3, 4, 5, 6, 7, 8, 9, 10 or more reference samples obtained from one or more subjects, each having a different disease state (e.g., different types of cancer), and used to train 3, 4, 5, 6, 7, 8, 9, 10 or more probability models.

[0153] During training, the machine learning engine 220 can utilize methylation information or methylation state vectors (e.g., as described above in connection with Figures 3-4) to train on sequence reads indicative of a disease state. In particular, the machine learning engine 220 determines an observed ratio of methylation for each CpG site in the sequence read. The ratio of methylation represents the proportion or percentage of base pairs that are methylated within a CpG site. The trained probability model 230 can be parameterized by a product of methylation ratios. As described above, any known probability model for assigning probabilities to sequence reads from a sample can be used. For example, the probability model can be a binomial model in which every site (e.g., CpG site) on a nucleic acid fragment is assigned a probability of methylation, or an independent site model in which the methylation of each CpG is designated by a distinct methylation probability, where methylation at one site on a nucleic acid fragment is assumed to be independent from methylation at one or more other sites.

[0154] In some embodiments, a Markov model in which the probability of methylation at each CpG site depends on the methylation at some number of preceding CpG sites in the sequence read or in the nucleic acid molecule from which the sequence read is derived (see, e.g., U.S. Patent Application Publication No. 2019 / 0133634, entitled "Anomalous Fragment Detection and Classification," filed March 13, 2019).

[0155] In some embodiments, the probabilistic model 230 is a "mixture model" that is fitted using a mixture of components from the underlying models. For example, in some embodiments, the mixture components can be determined using a multiple independent site model in which the methylation (e.g., methylation ratio) at each CpG site is assumed to be independent of the methylation at other CpG sites. Using the independent site model, the probability assigned to a sequence read or the nucleic acid molecule from which it is derived is the product of the methylation probability at each CpG site where the sequence read is methylated, and 1 minus the methylation probability at each CpG site where the sequence read is unmethylated. According to this embodiment, the machine learning engine 220 determines the methylation ratio of each of the mixture components. The mixture model is parameterized by the sum of the mixture components, each associated with a product of the methylation ratios. A probabilistic model Pr of n mixture components can be expressed as:

[0156]

number

[0157] For an input fragment, m i ∈{0,1} represents the observed methylation status of the fragment at position i in the reference genome, with 0 indicating unmethylated and 1 indicating methylated. The partial assignment for each mixture component k is k where f k ≧0 and

[0158]

number

[0159] f k = 1. The probability of methylation at position i within a CpG site in mixture component k is β ki Therefore, the probability of non-methylation is 1-β ki The number of mixture components n can be 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, etc.

[0160] In some embodiments, the machine learning engine 220 finds a set of parameters {β ki ,f k To identify {{overscore (N)}}, a probabilistic model 230 is fitted using maximum likelihood estimation. The maximized quantity for the N total fragments can be expressed as:

[0161]

number

[0162] In step 130, the processing system 200 applies the probability model 230 to calculate a value for each sequence read of a second set of sequence reads that is different from the first set of sequence reads generated in step 110, for example. These values ​​are calculated based at least on the probability that the sequence read (and the corresponding fragment) originates from a sample associated with a disease state of the probability model 230. The processing system 200 can repeat step 130 for each of the different probability models 230. In some embodiments, the processing system 200 calculates the value using a log-likelihood ratio R with the fitted probability models associated with several disease states. Specifically, the log-likelihood ratio can be calculated using the probability Pr of observing a methylation pattern on a fragment for a sample associated with a disease state and a healthy sample.

[0163]

number

[0164] In other embodiments, the processing system 200 can use different types of ratios or formulas to calculate the values. The machine learning engine 220 can determine that a fragment is indicative of a disease state (e.g., cancer) based on whether at least one of the log-likelihood ratios attributed to various disease states is higher than a threshold.

[0165] III.C. Feature Selection FIG. 6 is a diagram of a process for determining features to train a classifier, according to one embodiment. As described above, the machine learning engine 220 trains a probabilistic model 230 associated with a disease state. In the example shown in FIG. 6, the probabilistic model 230 ("tissue model") is associated with non-cancer (healthy), breast cancer, and lung cancer. The processing system 200 processes one or more cfDNA and / or tumor samples to obtain fragments and uses the probabilistic model 230 to assign values ​​to the fragments associated with non-cancer (healthy), breast cancer, and lung cancer. The processing system 200 can use information from sequence reads from the cfDNA and / or tumor samples to identify features for the classifier. In some embodiments, the processing system 200 can obtain and assign fragments from each window of the partitioned reference genome, as shown in FIG. 5. The processing system 200 aggregates the fragments from the windows into sequences to determine features for the classifier.

[0166] In step 140, the processing system 200 identifies features by determining a count of sequence reads having a value above a threshold. In an embodiment where the value is based on a log-likelihood ratio R, the threshold is a threshold ratio. The processing system 200 can identify features using multiple tiers of thresholds. For example, the tiers include thresholds of 1, 2, 3, 4, 5, 6, 7, 8, and 9. Each tier represents a varying threshold at which the fragment (from which the sequence read was generated) is more likely to be from a sample associated with a disease state than a healthy sample. The processing system 200 can use the thresholds to determine a count of outlier fragments, which can be used as features.

[0167] By filtering with a threshold, the processing system 200 can consider some fragments as outliers because they are less likely to be present in a healthy sample. Thus, outlier fragments can be considered more likely to be associated with (e.g., originate from) a disease state or cancer sample. The number of features can vary between different strata. In other embodiments, the processing system 200 uses a different number of strata or other thresholds. In other embodiments, the processing system 200 can filter the fragments using other methods or scoring, such as p-values. In some embodiments, the processing system 200 calculates a p-value for a methylation state vector that represents the probability of observing a methylation state vector or other methylation state vector with a low probability in a healthy control group. To determine that a fragment is aberrantly methylated, the processing system 200 uses a healthy control group that has a majority of fragments that are normally methylated (see, for example, U.S. Patent Application Publication No. 2010 / 0133994, entitled "Anomalous Fragment Detection and Classification," filed March 13, 2019).

[0168] Processing system 200 may repeat steps 130-140 for each probabilistic model trained in step 120. As a result, processing system 200 may identify features for one or more disease states associated with the probabilistic models. In the example shown in Figure 6, processing system 200 identifies one or more features for breast cancer and lung cancer.

[0169] In some embodiments, the processing system 200 ranks the identified features based on a measure of the feature's ability to distinguish between different disease states. For example, a feature is informative if it can distinguish one type of cancer from another type of cancer or from a healthy sample. The processing system 200 can use mutual information to determine a measure of the feature's information content in distinguishing between two disease states. For each pair of distinct disease states, the processing system 200 can designate one disease state, e.g., cancer type A, as a positive type and the other disease state, e.g., cancer type B, as a negative type.

[0170] The mutual information can be calculated using the estimated proportion of samples of positive and negative types (e.g., cancer types A and B) that are expected to have a non-zero feature in the resulting assay. For example, if a feature occurs frequently in healthy cfDNA, the processing system 200 determines that the feature is unlikely to occur frequently in cfDNA associated with various types of cancer. Thus, the feature can be a weak decision measure in distinguishing between disease states. In calculating the mutual information I, the variable X is a feature (e.g., binary) and the variable Y represents a disease state, e.g., cancer type A or B.

[0171]

number

[0172] The joint probability mass function of X and Y is p(x,y), and the marginal probability mass functions are p(x) and p(y). The processing system 200 may assume that the absence of a feature is uninformative and that both disease states are equally likely a priori, e.g., p(Y=A)=p(Y=B)=0.5. The probability of observing a given binary feature of cancer type A (e.g., in cfDNA) is denoted by p(1|A), where f Ais the probability of observing a feature in a ctDNA sample from a tumor (or a high-signal cfDNA sample) associated with cancer type A, and f H is the probability of observing the feature in a healthy or non-cancer cfDNA sample.

[0173] In some embodiments, f A The value of is estimated by the proportion of cancer patients whose cfDNA is expected to contain a non-zero feature value. When the training data for cancer type A consists of cfDNA samples, this proportion can be estimated as simply as the proportion of cfDNA samples in which that feature is observed. When the training data includes tumor samples, a correction can be applied to compensate for the lower proportion of tumor-induced fragments in cfDNA compared to the tumor. For N fragments in the tumor sample determined (e.g., from step 140) to have a value greater than the threshold, the processing system 200 calculates the chance r of detecting each of those fragments in cfDNA from that patient as:

[0174]

number

[0175] The probability of observing at least one fragment in cfDNA from that patient is then calculated as p(N cfDNA >0)=1-(1-r) N It can be calculated as f A To estimate p(N cfDNA >0) can be averaged across all training samples of cancer type A, and the probability is 1 for cfDNA samples with features, 0 for cfDNA samples without features, and 1-(1-r) for tumor samples. N In some embodiments, these estimates are based on predefined assumptions for the tumor fraction in the cfDNA of early stage cancer patients (e.g., 0.1%), the cfDNA sequencing depth in the final assay to be applied to the patient (e.g., 1000x), and the tumor sequencing depth (e.g., 25x). HTo estimate , the processing system 200 uses the proportion of positive samples to determine how many additional samples at greater sequencing depth would result in a positive detection classification.

[0176] III.D.Classification In step 150, the processing system 200 uses the features to generate a classifier. The classifier is trained to predict tissue of origin associated with a disease state for input sequence reads from a test sample of a test subject. The processing system 200 can select a predetermined number (e.g., 1024) of top-ranked features for each pair of disease states to train the classifier, for example based on mutual information calculations or another calculated decision measure. The predetermined number can be treated as a hyperparameter selected based on performance in cross-validation. The processing system 200 can also select features from regions of the reference genome that are determined to be more informative in distinguishing between pairs of disease states. In various embodiments, the processing system 200 retains the best-performing hierarchies for each region and for each cancer type pair (including non-cancer as the negative type).

[0177] In some embodiments, the processing system 200 trains the classifier by inputting a set of training samples with their feature vectors into the classifier and adjusting the classification parameters so that the classifier's function accurately associates the training feature vectors with their corresponding labels. The processing system 200 can group the training samples into one or more sets of training samples for repeated batch training of the classifier. After inputting all the sets of training samples with their training feature vectors and adjusting the classification parameters, the classifier can be sufficiently trained to label test samples according to their feature vectors within some error bound. The processing system 200 can train the classifier according to any one of several methods, for example, L1 regularized logistic regression or L2 regularized logistic regression (e.g., log loss function), generalized linear model (GLM), random forest, multinomial logistic regression, multilayer perceptron, support vector machine, neural net, or any other suitable machine learning technique.

[0178] In various embodiments, the processing system 200 converts the feature values ​​by binarization. In particular, feature values ​​greater than 0 are set to 1, resulting in feature values ​​of 0 or 1 (indicating the presence or absence of a disease state). In other embodiments, instead of binarization to 0 or 1, a smoothing function may be implemented (e.g., to provide more granular values). As shown in FIG. 14, the processing system 200 may binarize the features in cross-validation before using the features to train a classifier.

[0179] In various embodiments, the processing system 200 trains a multinomial logistic regression classifier on the training data for a fold and generates predictions for the held up data. For each of the K folds, the processing system 200 trains one logistic regression for each combination of hyperparameters. An exemplary hyperparameter is L2 penalty, a form of regularization applied to the logistic regression weights. Another exemplary hyperparameter is topK, the number of highly ranked regions to retain for each tissue type pair (including non-cancer). For example, if topK=16, the processing system 200 retains the top 16 regions for each tissue type pair as ranked by the mutual information procedure described herein. By following this procedure, the processing system 200 can generate predictions for each sample in the training set while ensuring that the classifier is not trained on the data for which the predictions are generated.

[0180] In various embodiments, for each set of hyperparameters, the processing system 200 evaluates the performance against the cross-validated predictions of the full training set, and the processing system 200 selects the set of hyperparameters with the best performance to retrain against the full training set. Performance may be determined based on a log loss metric. The processing system 200 may calculate the log loss by taking the negative logarithm of the prediction for the correct label for each sample, then summing over the samples. For example, a perfect prediction of 1.0 for the correct label would result in a log loss of 0 (lower is more accurate). To generate predictions for new samples, the processing system 200 may calculate feature values ​​using the above method, but restricted to selected features (region / positive class combinations) under a chosen topK value. The processing system 200 may use the generated features and generate predictions using the trained logistic regression model.

[0181] In an optional step 160, the processing system 200 applies a classifier to predict the tissue of origin of the test sample, where the tissue of origin is associated with one of the disease states. In some embodiments, the classifier can return a prediction or likelihood for more than one disease state or tissue of origin. For example, the classifier can return a prediction that the test sample has a 65% likelihood of having breast cancer primary tissue, a 25% likelihood of having lung cancer primary tissue, and a 10% likelihood of having healthy primary tissue. The processing system 200 can further process the predictions to generate a single disease state determination.

[0182] III.E. Uncertain Location In various embodiments, tumor fraction can be a covariate of predictions made by a trained classifier or model across samples. As tumor fraction decreases, score assignments (e.g., based on the log-likelihood ratio R described above) can become less certain until the limit of classification detection is reached (i.e., 50% probability of cancer / cancer type detection). Samples with high cfDNA tumor fraction tend to be classified with certainty, while samples with low cfDNA tumor fraction tend to be more ambiguous. In instances with ambiguous signals, assignments become less reliable and may or may not be correct by chance. In a single localization use case, the processing system 200 can identify ambiguous signals and segregate these predictions into an "uncertain localization class."

[0183] For example, in some embodiments, the processing system 200 can determine a posteriori uncertain assignments from a set of tissue of origin localization vectors, particularly for individuals with cancer scores greater than a target threshold. The processing system 200 can determine the uncertain assignments under cross-validation. For each sample, the processing system 200 can calculate a metric to capture the uncertainty in localization for that sample. As one exemplary approach, the processing system 200 calculates this metric using the information entropy (bits) of the tissue of origin localization, where a bit value of 0 occurs when one prediction is certain. For the most ambiguous case (equal probability for all n classes), the processing system 200 calculates a bit value of log2(n). As another approach, the processing system 200 determines this metric using the difference (delta value) between the top-ranked score and the next top-ranked score. A delta value of 1 occurs when one prediction is certain. A delta value of 0 occurs for the most ambiguous case. By including indeterminate results, the processing system 200 can filter out weak calls that are only correct by chance and improve accuracy for unambiguous location calls (eg, percentage correction for tissue of origin assignment).

[0184] As an alternative to post-hoc uncertain assignment, the processing system 200 can use expectation maximization during training to determine assignments to uncertain classes. The processing system 200 can also add a second layer to the classifier output to classify cases into uncertain classes.

[0185] Given the metrics and a record of whether each sample was correctly located, the processing system 200 can calculate a precision-recall curve for an uncertain call threshold, as shown in Figure 18. For example, a cutoff point can be selected based on a target precision level, such as 90% in the example in Figure 18. The processing system 200 can calculate the cutoff point for the localization labels individually (e.g., for a cancer type) or for all cancer types as a whole. Tradeoffs are subject to optimization and can depend on the cost of an incorrect localization call versus the number of calls assigned an uncertain outcome (e.g., precision and recall).

[0186] III.F. Defending Against Class Imbalance In various embodiments, the element score vector s for each sample is i contains the posterior probability of signal localization for each predicted class (e.g., disease state). Each element is scaled by a prior probability proportional to the proportion of training examples for each class.

[0187]

number

[0188] When classes are imbalanced, samples with weak signals may be shifted to the inappropriate class. For example, a training set may contain 99% of samples with liver cancer detection results, but few detection results of different cancer types. As a result, a classifier trained on this set may be skewed toward predicting liver cancer (or always guessing that class). Furthermore, if the class proportions in the classifier training are inconsistent with the population frequencies to which the classifier is applied (e.g., if the class proportions are more balanced), incorrect predictions may be produced.

[0189] To assess the ability of the classifier to localize cfDNA samples from methylation and / or genomic and / or clinical features, the processing system 200 can target rate equivalence across classes. The processing system 200 can optionally calibrate the score to the incidence of disease conditions in the screening population to compensate for disease detectability through tumor rates. By modifying the a priori probabilities applied to the classifier trained using a general training set, the processing system 200 can customize the classifier to improve predictions for a particular population associated with the a priori probabilities (e.g., indicative of the distribution of disease conditions within that particular population). Different regions or countries may have different a priori probabilities based on the prevalence of a particular disease condition or type of cancer in a corresponding subpopulation of individuals.

[0190] As an exemplary approach, the processing system 200 performs a post-hoc recalibration of the model scores. Specifically, the processing system 200 corrects the scores for a class by dividing the assigned probability by the frequency of training set examples for that class. This correction can be optionally stabilized by adding pseudo-counts. The processing system 200 then sums each score vector s to 1. i can be normalized.

[0191] Alternatively, the processing system 200 can resample the low frequency training examples to a desired percentage. As yet another approach, the processing system 200 can reweight the loss function in classifier training.

[0192] IV. Multilayer Perceptron Model In some embodiments, a multi-layer perceptron model ("MLP") can be used as an alternative to logistic regression for classification. Similar to logistic regression-based classifiers, the MLP classifier can be a single multi-class classifier for detecting cancer and determining tissue of origin (TOO) or cancer type. For example, the multi-class classifier can be trained to distinguish between two or more, three or more, five or more, ten or more, fifteen or more, or twenty or more different types of cancer. In one embodiment, the multi-class cancer MLP model can also include a class label for non-cancer, and cancer detection can be determined (e.g., as 1-non-cancer). In another embodiment, the multi-layer perceptron model can be a two-stage classifier with a first stage for binary classification (e.g., cancer or non-cancer) and a second stage multi-layer perceptron model for multi-class classification (e.g., TOO), e.g., with one or more hidden layers.

[0193] In one embodiment, the multi-layer perceptron comprises a two-stage classifier: a first stage multi-layer perceptron (MLP) binary classifier with no hidden layer, and a second stage multi-layer perceptron (MLP) multi-class classifier with a single hidden layer. In one embodiment, samples determined to have cancer using the first stage classifier will be subsequently analyzed by the second stage classifier.

[0194] In the first stage of training, a binary (two-class) multi-layer perceptron model with no hidden layers for detecting the presence of cancer can be trained to distinguish cancer samples (regardless of TOO) from non-cancer. For each sample, the binary classifier outputs a prediction score indicating the likelihood of the presence or absence of cancer.

[0195] In the second stage of training, a parallel multi-class multi-layer perceptron model for determining cancer type or cancer tissue of origin can be trained. In one embodiment, only cancer samples that receive a score higher than a cutoff threshold (e.g., the 95th percentile of non-cancer samples in the first stage classifier) ​​can be included in the training of this multi-class MLP classifier. For each cancer sample used in training and testing, the multi-class MLP classifier outputs a prediction value for the cancer type classified, where each prediction value is the likelihood that a given sample has a certain cancer type. For example, the cancer classifier can return a cancer prediction for the test sample, including a prediction score for breast cancer, a prediction score for lung cancer, and / or a prediction score for no cancer.

[0196] 16 is a flow chart of a method 1600 for determining the probability that a sample has a disease state, according to various embodiments. In some embodiments, a processing system 200 performs the method 1600 to process fragment sequence reads from a nucleic acid sample. The method 1600 includes, but is not limited to, the following steps described with respect to components of the processing system 200:

[0197] In step 1610, the processing system 200 generates sequence reads from one or more biological samples. In some embodiments, the processing system 200 filters the sequence reads according to a p-value score of the sequence read. The p-value score of the sequence read indicates the probability of observing methylation in a nucleic acid fragment of the one or more biological samples corresponding to the sequence read.

[0198] In step 1620, the processing system 200 uses the sequence reads to determine, for each location of the set of chromosomal locations, a count of nucleic acid fragments of the one or more biological samples within that location that have at least a threshold similarity to fragments associated with a disease state, such as cancer. The disease state may be associated with at least one type of cancer, a stage of cancer, or another type of disease or condition.

[0199] Each of the locations may represent several consecutive base pairs of a chromosome. The number of base pairs may vary between different locations. The processing system 200 may generate sequence reads for multiple regions of a genome. There may be up to tens of thousands of regions or more. Each region may contain hundreds, thousands, or more base pairs. The method 1600 may be performed for whole genome bisulfite sequencing (WGBS) or for targeted panel assays.

[0200] In step 1630, the processing system 200 trains a machine learning model using the location counts as features. In some embodiments, the processing system 200 binarizes the features to indicate the presence or absence (e.g., a Boolean value) of one of the disease conditions at each of the locations. A count of at least one nucleic acid fragment at a location indicates the presence of one of the disease conditions at that location. A count of zero nucleic acid fragments at a location indicates the absence of one of the disease conditions at that location. In some embodiments, the machine learning model can be a logistic regression model. In some embodiments, the machine learning model can be a multi-layer perceptron model (neural network). One skilled in the art will readily appreciate that other machine learning models can be used, including, for example, a generalized linear model (GLM), a multi-layer perceptron, a support vector machine, a random forest, or a neural network classifier.

[0201] In step 1640, the trained machine learning model determines the probability that the test sample has a disease state. The test sample may be obtained from a patient and may include blood and / or tissue. In optional step 1650, a treatment is provided to the patient according to the probability. For example, the patient may be provided with a treatment (e.g., a medication or an intervention procedure) in response to determining that the probability is greater than a threshold. In another embodiment, in optional step 1650, a test report may be generated and the patient may be provided with those test results, including the probability that the test sample has the disease.

[0202] The experimental results shown in Figures 17-20 were obtained by training the model using samples from the CCGA study, which is further described below.

[0203] 17 illustrates the performance gain in sensitivity of a multi-layer perceptron model according to one embodiment. Compared to a logistic regression model, the multi-layer perceptron model (MLP) demonstrates performance gains in sensitivity of disease detection across cancer stages I, II, III, and IV.

[0204] Figure 18 shows experimental results of the multi-layer perceptron model in determining tissue of origin according to one embodiment. Compared to the logistic regression model (LR: 1803 and 1804), the multi-layer perceptron model (MLP: 1801 and 1802) has improved accuracy in determining tissue of origin. This improved accuracy is achieved when processing sequence reads associated with all cancer types in the training set, as well as when processing sequence reads of the training set that include more than 10 sequence read examples for each cancer type in the training set.

[0205] 19 shows experimental results of a multi-layer perceptron model in determining tissue of origin by cancer stage, according to one embodiment. Compared to a logistic regression (LR) model, the multi-layer perceptron model (MLP) demonstrates performance gains in accuracy of tissue of origin (TOO) detection across cancer stages I, II, III, and IV. Among cancer stages, the performance gains for the MLP model are greatest for stage I.

[0206] 20 shows experimental results of a multi-layer perceptron model across cancer types, according to one embodiment. For most cancer types shown in FIG. 20, the multi-layer perceptron model (MLP) achieves greater accuracy in tissue of origin (TOO) detection compared to the logistic regression model.

[0207] In some embodiments, the analysis system uses a two-stage model to determine the tissue of origin (TOO) of the cancer or another type of disease state. The analysis system generates sequence reads from nucleic acid fragments of a biological sample. The analysis system determines a first set of training data by processing the sequence reads, e.g., using any of the processes described in Section II.A. Assay Protocol. The analysis system can use methylation information to determine the first set of training data. For example, the analysis system determines sequence reads that are hypomethylated by determining that a threshold or percentage of CpG sites corresponding to the sequence reads are unmethylated. Further, the analysis system determines sequence reads that are hypermethylated by determining that a threshold or percentage of CpG sites corresponding to the sequence reads are methylated. The analysis system can also determine that the sequence reads are aberrantly methylated. In some embodiments, the analysis system filters the sequence reads by removing sequence reads that have a p-value below a threshold p-value.

[0208] The analysis system uses the first set of training data to train a binary classifier, which is trained to predict, for input sequence reads from a first test biological sample, a binary output, i.e., the presence or absence of at least one disease state in the first test biological sample.

[0209] Using the prediction of the binary classifier, the analysis system can determine that a subset of the biological samples has the presence of one or more disease conditions. The binary classifier can be used to train a tissue of origin classifier. In particular, the analysis system determines a second set of training data using sequence reads corresponding to nucleic acid fragments of the subset of biological samples. The analysis system trains the tissue of origin classifier using the second set of training data. The tissue of origin classifier is trained to predict, for input sequence reads from a second test biological sample, a tissue of origin associated with a disease condition present in the second test biological sample. The first and second test biological samples can be the same sample or different samples.

[0210] In some embodiments, the analysis system uses a primary tissue classifier to determine a score indicative of the probability that a primary tissue associated with a disease state is present in the second test biological sample. The analysis system can calibrate the score, for example to adjust the output of the model for overconfidence. For example, the analysis system performs a k-nearest neighbor (KNN) operation in relation to the score using the feature space output by the primary tissue classifier. In one embodiment, the feature space includes the top two predicted labels from the primary tissue classifier (e.g., lung cancer and prostate cancer) and an indication of whether the correct classification was a disease state different from the top two predictions. The analysis system can also calibrate the score by normalizing the probabilities using the output of a binary classifier indicating different probabilities of the presence of at least one disease state present in the second test biological sample.

[0211] In some embodiments, the primary tissue classifier is a multi-layer perceptron that includes at least one hidden layer. The primary tissue classifier can also include a 100-unit hidden layer or a 200-unit hidden layer, among other hidden layer sizes. The multi-layer perceptron can be fully connected and use a regularized linear unit activation function. In some embodiments, the binary classifier is a multi-layer perceptron that does not include a hidden layer. In a different embodiment, the binary classifier is a multi-layer perceptron that includes at least one hidden layer. In other embodiments, these classifiers can be logistic regression models, multinomial logistic regression models, or other types of machine learning models.

[0212] Furthermore, the analysis system can train the primary tissue classifier and the binary classifier using one or more machine learning techniques known to those skilled in the art, including, for example, no early truncation (instead selecting a given number of training epochs), stochastic gradient descent, weight decay, dropout regularization, Adam optimization, He initialization, and learning rate scheduling, regularized linear unit activation function, leaky regularized linear unit activation function, sigmoid activation function, and boosting, among others. As shown in Figure 31, the primary tissue accuracy of the primary tissue classifier improves through training iterations. Each iteration can include a different combination of machine learning techniques. Furthermore, the increase in primary tissue accuracy exists across different cancer stages, i.e., I, II, and III.

[0213] In some embodiments, the analysis system performs cross-validation on one or both of the primary tissue classifier and the binary classifier. The analysis system can retrain the classifier using hyperparameters selected based on the output of the cross-validation. The analysis system can select the hyperparameters by aggregating the results from all folds in the cross-validation. In one embodiment, the analysis system selects the hyperparameters to train the primary tissue classifier by optimizing for primary tissue accuracy instead of log-likelihood, because this classifier can be more reliable for samples with stronger signals.

[0214] In some embodiments, the analysis system determines the probability that a primary tissue associated with the disease state is present in the second test biological sample by the primary tissue classifier. In response to determining that the probability is greater than the primary tissue threshold, the analysis system predicts that a primary tissue associated with the disease state is present in the second test biological sample. The analysis system can determine different primary tissue thresholds associated with different primary tissues. Furthermore, the analysis system can determine the primary tissue threshold associated with a given disease state by iterating through a range of different probabilities of the candidate primary tissue thresholds. For each iteration, the analysis system determines a sensitivity rate at a given specificity rate of the primary tissue classifier. The analysis system can optimize the tradeoff between the sensitivity rate and the specificity rate of the primary tissue classifier for a given disease state. The analysis system can determine the sensitivity rate using the score output by the binary classifier or the primary tissue classifier. Furthermore, the analysis system can stratify the samples using the score from the primary tissue classifier.

[0215] In some embodiments, the analysis system trains the binary classifier and the tissue of origin classifier using binarized features, each of which has a value of 0 or 1. Values ​​greater than 1 are replaced with 1 during binarization.

[0216] V. Adjusting the binary classification threshold The analysis system may adjust the trained cancer classifier to remove samples that were used in training the cancer classifier. In particular, the analysis system may attempt to remove non-cancerous samples with high tissue signals that weaken the sensitivity of the cancer classifier in predicting cancer. High tissue signals refer to samples that have a significant proportion of cfDNA from tissue of origin (TOO) relative to the healthy distribution, as determined, for example, by a tissue of origin classifier, a multi-class cancer classifier, or other means. Non-cancerous samples with high tissue signals are outliers in the non-cancerous distribution, and they may be pre-cancer, early cancer, or undiagnosed cancer. The analysis system may identify non-cancerous samples with high tissue signals in at least one cancer type. In some embodiments, some cancer types are further separated into cancer subtypes. For example, hematological cancer types may be further separated into a combination of, for example, circulating lymphoid subtypes, non-Hodgkin's lymphoma (NHL) indolent subtypes, NHL aggressive subtypes, Hodgkin's lymphoma (HL) subtypes, myeloid subtypes, and plasma cell subtypes.

[0217] Referring to FIG. 21, FIG. 21 shows a graph of cancer type likelihood for non-cancer samples with specificity higher than 95%. A cancer score was calculated for each non-cancer sample from multiple non-cancer samples, i.e., samples from healthy individuals not currently diagnosed with cancer. The cancer score may be determined by a binary classifier as the likelihood that the sample has cancer given the methylation sequencing data of the sample. In other embodiments, the cancer score may be calculated according to other methods that input at least sequencing data (e.g., methylation, single nucleotide polymorphism (SNP), DNA, RNA, etc.) and output the likelihood of the sample having cancer based on the input sequencing data. One example of a classifier is a mixed model classifier. A distribution of non-cancer samples may be generated according to the cancer scores of the non-cancer samples. A binary threshold cutoff may be set to ensure some level of binary classification specificity, e.g., true negative rate. Typically, a high specificity cutoff is used in classifying cancer, e.g., specificity between 90% and 99.9%, or 99.5% or higher. However, many non-cancer samples just below the specificity cutoff used in training the cancer classifier may have high tissue signal, thereby biasing the binary threshold cutoff towards positivity.

[0218] To demonstrate, non-cancer samples with specificity above 95% were selected and then input into a multi-class cancer classifier to determine the probability for each cancer type, or tissue of origin (TOO). The cancer types or TOO labels used in this embodiment of the multi-class cancer classifier include: circulating lymph, bone marrow, NHL indolent, colorectal, NHL aggressive, lung, uterine, breast, prostate, pancreas and gallbladder, upper gastrointestinal, bladder and urothelium, plasma cell, head and neck, kidney, ovary, sarcoma, liver and bile duct, cervical, other tissues, HL, anorectal, melanoma, thyroid. The graph in FIG. 21 shows a number of non-cancer samples with high tissue signals from at least one tissue type. Each point in the column for a tissue type corresponds to the tissue of origin likelihood for the non-cancer sample above the 95% specificity threshold. Notably, a number of tissue types have multiple non-cancer sample outliers with significant tissue contributions that are not typical for non-cancer samples. This may occur when such non-cancer samples have cfDNA signals driven by cancer-like methylation, clonal proportions, and / or growth / turnover rates. It can be inferred that many non-cancer samples used in training the cancer classifier may be pre-cancer, early stage cancer, or undiagnosed cancer. However, these non-cancer samples with significant tissue contributions shift the binary classification cutoff threshold upwards, thereby decreasing the cancer classification sensitivity, especially for samples with significant tissue signals just below the pre-set binary classification cutoff threshold. In practice, such signals (e.g., corresponding to circulating lymph, bone marrow, and NHL indolent) may be the main attractors of false positive decisions. It is noted that circulating lymph, bone marrow, NHL indolent, colorectal, NHL aggressive, lung, uterus, breast, prostate, pancreas and gallbladder, upper gastrointestinal, plasma cell, head and neck, cervical, HL had at least one non-cancer sample with a probability of tissue of origin higher than 0.1. Notably, circulating lymphoid, myeloid, NHL indolent, and NHL aggressive (all hematological subtypes) had two or more non-cancerous samples with a probability of tissue of origin higher than 0.5.

[0219] Referring to FIG. 22, FIG. 22 shows a graph of hematological subtypes separated according to methylation sequencing data. The graph in FIG. 22 demonstrates the ability to model hematological subtypes. This can be beneficial in making multi-class cancer classification more granular (e.g., further classifying with hematological subtype labels) or as a way to adjust cancer classification through removing non-cancer samples with high hematological subtype signals before training a cancer classifier. As described above, methylation signals can cover multiple CpG sites, thereby generating a high vector space. Using the hematological subtype samples and non-cancer samples, the analysis system can perform principal component analysis. Principal component analysis identifies orthogonal principal components (or embeddings) of the vector space in order of variance of methylation signals among the samples. The first principal component, shown on the horizontal axis of the graph as V1, has the highest variance, and the second principal component, shown on the vertical axis of the graph as V2, has the next highest variance. Graph 900 is annotated with clusters of samples for each hematological subtype and non-cancer. The hematological subtypes shown include circulating lymphoid, solid lymphoid, plasma cell, and myeloid. The solid lymphoid subtype can be further divided into HL, NHL indolent, and NHL aggressive. The graph shows the possibilities for classifying according to the hematological subtype, either to add the hematological subtype to the multi-class cancer classification, or to model each of the hematological subtypes to tune the cancer classifier.

[0220] VA High Signal Non-cancer Sample Removal FIG. 23A shows a flow chart describing a process 1000 of determining a binary threshold cutoff for binary cancer classification according to one or more embodiments. Binary classification for predicting between cancer and non-cancer evaluates the cancer score of a sample against a determined binary threshold cutoff, and samples with a cancer score below the binary threshold cutoff are determined to be non-cancer and samples with a cancer score equal to or greater than the binary threshold cutoff are determined to be cancer. A trained multi-class cancer classifier evaluates the methylation signal (and / or other sequencing data) of the sample to determine the probability of a number of TOO labels classified by the multi-class cancer classifier. The TOO labels used in the multi-class cancer classifier can be cancer tissue types or cancer tissue subtypes (e.g., hematology subtypes described above). The process 1000 can be performed or accomplished by an analysis system.

[0221] The analysis system receives sequencing data of a plurality of biological samples containing cfDNA fragments, the biological samples including cancer samples and non-cancer samples 1010. The sequencing data can be methylation sequencing data, SNP sequencing data, other DNA sequencing data, RNA sequencing data, etc.

[0222] For each non-cancer sample, the analysis system classifies the non-cancer sample using a multi-class cancer classifier based on the sequencing-derived features 1020, where the multi-class cancer classifier predicts a probability of each of a plurality of TOO labels. The analysis system can generate a feature vector for the non-cancer sample that assigns, for each CpG site under consideration, an anomaly score based on at least one aberrantly methylated cfDNA fragment overlapping the CpG site.

[0223] For each non-cancer sample, the analysis system determines whether the predicted probability likelihood for one or more TOO labels exceeds a TOO threshold 1030. TOO threshold determination is further described below in FIG. 23B.

[0224] The analysis system determines 1040 a binary threshold cutoff for predicting the presence of cancer, where the binary threshold cutoff is determined based on the distribution of the non-cancer samples, excluding one or more non-cancer samples identified as having a probability likelihood above at least one TOO threshold. Non-cancer samples having at least one probability likelihood for that TOO label above the TOO threshold corresponding to that TOO label are excluded. The analysis system then calculates the distribution of the non-cancer samples according to the cancer score of each non-cancer sample, and then determines a binary threshold cutoff from the distribution at a desired specificity level (e.g., 99.4-99.9% specificity). It is noted that each cancer score can be determined according to the sequencing data, for example, the cancer score can be output by a binary cancer classifier that predicts the likelihood of cancer based on methylation sequencing data, as described herein. In other embodiments, the cancer score can be calculated according to other methods that input at least sequencing data (e.g., methylation, single nucleotide polymorphism (SNP), DNA, RNA, etc.) and output the likelihood that the sample has cancer based on the input sequencing data.

[0225] FIG. 23B shows a flow chart describing a process 1005 of thresholding TOO labels to determine a binary threshold cutoff for binary cancer classification, according to one or more embodiments. This process 1005 can be an embodiment of process 1000. Binary classification to predict between cancer and non-cancer evaluates the cancer score of the sample against a determined binary threshold cutoff, and samples with a cancer score below the binary threshold cutoff are determined to be non-cancer and samples with a cancer score equal to or greater than the binary threshold cutoff are determined to be cancer. A trained multi-class cancer classifier evaluates the methylation signal (and / or other sequencing data) of the sample to determine the probability of a number of TOO labels classified by the multi-class cancer classifier. The TOO labels can be cancer tissue types, or more specifically cancer tissue subtypes (e.g., hematology subtypes described above). Process 1005 can be performed or accomplished by an analysis system.

[0226] The analysis system obtains 1015 a plurality of samples labeled as cancer or non-cancer, i.e., a training set including either cancer or non-cancer samples, and a held-up set including a plurality of samples labeled as cancer or non-cancer, respectively. Each sample in the training set includes methylation sequencing data, for example, generated according to process 300 of FIG. 3. In other embodiments, each training sample has other sequencing data used in tandem or in place of the methylation sequencing data. Additionally, each sample from the training set and the held-up set has a cancer score. As described above, the cancer score can be determined by a binary classifier as the likelihood that a sample has cancer, given the methylation sequencing data of the sample. In other embodiments, the cancer score is calculated according to other methods, exemplified by the mixture model described herein, that input at least sequencing data (e.g., methylation, single nucleotide polymorphism (SNP), DNA, RNA, etc.) and output the likelihood that the sample has cancer according to the input sequencing data.

[0227] The analysis system determines a feature vector for each non-cancer training sample based on the methylation sequencing data 1025. The analysis system may determine the feature vector for each non-cancer training sample, for example, by determining an anomaly score for each CpG site in the set of CpG sites considered. In some embodiments, the analysis system defines the anomaly score of the feature vector using a binary score based on whether there is an anomaly fragment in the set of anomaly fragments that encompasses the CpG site. Once all anomaly scores have been determined for the sample, the analysis system determines the feature vector as a vector of anomaly scores associated with each CpG site considered. The analysis system may further normalize the anomaly scores of the feature vector based on the coverage of the sample.

[0228] The analysis system inputs the feature vector of each non-cancer training sample into a multi-class cancer classifier to generate a TOO prediction 1035. The multi-class cancer classifier is trained on multiple TOO labels, including cancer type, cancer subtype, non-cancer, or any combination thereof. The multi-class cancer classifier can be trained as described herein. The trained multi-class cancer classifier determines multiple probabilities for the TOO labels as cancer predictions, where the probabilities of the TOO labels indicate the likelihood of having cancer corresponding to the TOO labels.

[0229] In some examples, the analysis system sweeps or iterates 1045 over a range of probabilities of the TOO label as a candidate TOO threshold that calculates a specificity rate and a sensitivity rate over the range of probabilities of the TOO label. The analysis system may sweep over the range of probabilities incrementally, for example, at 0.01, 0.02, 0.03, 0.04, 0.05, etc. As the analysis system sweeps over the range of probabilities, the analysis system filters non-cancer training samples with a probability of the TOO label equal to or greater than the candidate TOO threshold according to the output of the multi-class cancer classifier. As a numerical example, the analysis system considers a candidate TOO threshold of 0.35. Non-cancer training samples with a probability of the TOO label equal to or greater than 0.35 are filtered out from the training set. The analysis system determines an adjusted binary threshold cutoff based on the filtered training set. The analysis system calculates a specificity rate of prediction using the adjusted binary threshold cutoff for the holdout set. Specificity refers to the accuracy of identifying non-cancer samples as non-cancer labels. The analysis system also calculates the sensitivity rate of the prediction using the adjusted binary threshold cutoff for the held-out set. Sensitivity refers to the accuracy of identifying the cancer sample as a cancer label. In practice, the specificity rate and / or sensitivity rate may be defined according to a true positive rate, a false positive rate, a true negative rate, a false negative rate, another statistical calculation, etc.

[0230] The analysis system determines 1055 a TOO threshold for the TOO label. The analysis system selects a TOO threshold from the candidate TOO thresholds by optimizing the calculated specificity rate and / or sensitivity rate over a range of candidate TOO thresholds. In some examples, TOO thresholds are determined or otherwise applied for several TOO tissue type classes or subtype classes, such as a hematology class. By way of example only, an algorithm for calculating and applying TOO-specific probability thresholds can be used to remove non-cancer samples with excess signals of hematological disorders. The algorithm can include first searching over a grid of probability values ​​for each pre-specified TOO label, and evaluating the clinical specificity and clinical sensitivity of the held-out set using a binary detection threshold that is calculated after removing non-cancer samples with equal to or greater than the probability of the specified TOO label for every value. By iterating over the probability grid, the algorithm identifies a combination of TOO thresholds for the pre-specified TOO labels that optimizes the trade-off between clinical specificity and clinical sensitivity of the held-out set. The final optimized TOO probability threshold is used to filter out non-cancer samples that exceed any of the values, given the TOO label. The cleaned set of non-cancer samples is used to calculate the cancer-non-cancer detection threshold. Nevertheless, in some instances, the TOO-specific thresholding can be manually set at some cut point, such as a desired specificity level (e.g., 99.4-99.9% specificity).

[0231] The analysis system adjusts the binary cancer classification by removing non-cancer training samples that exceed the TOO thresholding before determining the binary threshold cutoff 1065. The analysis system filters out the non-cancer training samples from the training set according to the determined TOO threshold for the TOO label. The analysis system sets the binary threshold cutoff according to the filtered training set. For example, the analysis system determines a new binary threshold cutoff based on the filtered distribution of scores. In additional embodiments, the analysis system can determine a TOO threshold for any of the TOO labels according to steps 1010, 1020, 1030, and 1040 to adjust the binary cancer classification.

[0232] Stratification of sample distribution by VBTOO signal In one or more embodiments, the analysis system tunes the cancer classifier by stratifying the sample distribution according to the TOO signal and determining a binary threshold cutoff for each stratum. The analysis system may stratify the sample distribution according to a signal for one or more TOO labels that are determined according to the TOO predictions output by the multi-class cancer classifier.

[0233] As used herein, "high tissue signal" refers to a sample with a tissue signal above some threshold, for example, for any type of tissue in general, or for a specific cancer type, also referred to as a TOO label. The tissue signal may be determined by a multi-class cancer classifier or other approach, compared to a healthy distribution. Non-cancer samples with high tissue signal are outliers in the non-cancer distribution. Some of these non-cancer samples may be pre-cancer, early cancer, or undiagnosed cancer. The analysis system can identify non-cancer samples with high tissue signal in at least one TOO label. In one approach to determine high tissue signal, the predicted value of the TOO label output by the multi-class cancer classifier is compared against a tissue signal threshold. Samples with predicted values ​​above the tissue signal threshold are considered to have high tissue signal for that TOO label, while samples with predicted values ​​below the tissue signal threshold are considered to not have high tissue signal (or low tissue signal) for that TOO label. In another approach, one or more top predictions in the TOO prediction are considered. For example, the TOO prediction of a sample has a first prediction of a colorectal TOO label, a second prediction of a breast TOO label, and a third prediction of a head / neck TOO label. If the top prediction is considered, the sample is deemed to have a high tissue signal of the TOO label in the first prediction, which in this example is the colorectal TOO label. If the top two predictions are considered, there is a high tissue signal in both the colorectal TOO label and the breast TOO label. Other approaches to determining tissue signals may include other models trained to determine tissue signals for one or more TOO labels. Such models may include classifiers trained to determine tissue signals for a subset of TOO labels. For example, a hematology-specific classifier may be trained and used to determine tissue signals for one or more hematology subtypes. Other models include deconvolution models that can deconvolute tissue signals from methylation sequencing data (and / or other types of sequencing data).

[0234] Referring now to Figure 32, which shows a process for stratifying hematology signals into two tiers according to one or more embodiments, the following description describes stratification using hematology signals, although the principles can be easily applied to other TOO signals.

[0235] The analysis system stratifies 1300A a held-up set of cancer and non-cancer samples into a low signal tier 1310 and a high signal tier 1320 according to the hematology signal. Each sample in the held-up set has a cancer score determined by a binary cancer classifier and a TOO prediction determined by a multi-class cancer classifier. In one embodiment, the hematology signal of the sample is determined according to the TOO prediction output by the multi-class cancer classifier. In one embodiment, when considering one or more top predictions (e.g., the top one, the top two, etc.), a high hematology signal is determined if at least one of the top predictions being considered is one of the hematology subtypes (e.g., lymphoid tumor subtype and myeloid tumor subtype). Other hematology subtypes may be included. Thus, if the sample has a TOO prediction where at least one of the top predictions is considered as lymphoid tumor subtype or myeloid tumor subtype, the sample is determined to have a high hematology signal. Otherwise, the sample is determined not to have a high hematology signal.

[0236] The analysis system determines a binary threshold cutoff for each tier for predicting the presence or absence of cancer in the samples. The samples in the low signal tier 1310 are used by the analysis system to determine 1305 a binary threshold cutoff for predicting the presence or absence of cancer in the samples in the low signal tier 1310. The binary threshold cutoff is determined 1305 according to a false positive budget set for the low signal tier 1310. Using the cancer scores of the samples in the low signal tier 1310, the analysis system sweeps over a range of candidate binary threshold cutoffs and evaluates the true positive rate (also called sensitivity) and false positive rate at each candidate binary threshold cutoff. The candidate binary threshold cutoff with the closest false positive rate within the false positive budget is determined to be the candidate binary threshold cutoff. The analysis system performs similar operations to determine 1315 a binary threshold cutoff for the high signal tier 1320. The false positive budget for the low signal tier 1310 and the false positive budget for the high signal tier 1320 may be set according to the ratio of the statistical true positive rates of the tiers. This ratio is intended to reduce the false positive rate in the high signal tier 1320.

[0237] For a test sample, the analytical system places the test sample in either the low signal tier 1310 or the high signal tier 1320 according to the hematology signal. If the test sample is placed in the low signal tier 1310, the analytical system applies 1315 the binary threshold cutoff of the low signal tier 1310 to the cancer score of the test sample. If the cancer score is equal to or greater than the binary threshold cutoff of the low signal tier 1310, the analytical system returns a prediction of the presence of cancer in the test sample, otherwise a prediction of no cancer. If the test sample is placed in the high signal tier 1320, the binary threshold cutoff of the low signal tier 1320 is applied 1325 to the cancer score of the test sample. If the cancer score is equal to or greater than the binary threshold cutoff of the high signal tier 1320, the analytical system returns a prediction of the presence of cancer in the test sample, otherwise a prediction of no cancer.

[0238] VI. Circulating Cell-Free Genome Atlas Study In various embodiments, each predictive cancer model is trained using a set of training data derived from a training subset of patients from the Circulating Cell-Free Genome Atlas (CCGA) study (see Non-Patent Document 1), and then subsequently tested using a set of testing or validation data derived from a testing or validation subset of patients from the CCGA study.

[0239] The predictive cancer model described herein was trained using multiple known cancer types from the Circulating Cell-Free Genome Atlas (CCGA) study. The CCGA sample set included the following cancer types: breast, lung, prostate, colorectal, kidney, uterine, pancreatic, esophageal, lymphoma, head and neck, ovarian, hepatobiliary, melanoma, cervical, multiple myeloma, leukemia, thyroid, bladder, stomach, and anorectal. Thus, the model can be a multi-cancer model (or multi-cancer classifier) ​​for detecting one or more, two or more, three or more, four or more, five or more, ten or more, or twenty or more different types of cancer. The predictive cancer model can be trained using an improved set of training data derived from a first subset of patients in the CCGA study, and then subsequently tested using an improved set of testing data derived from a second subset of patients from the CCGA study.

[0240] VII. Cancer Assay Panels In various embodiments, the predictive cancer models described herein use samples enriched using a cancer assay panel that includes multiple probes or multiple probe pairs. Several targeted cancer assay panels are known in the art, for example, as described in U.S. Patent Application Publication No. 2019 / 0133634, filed on April 2, 2019, U.S. Patent Application Publication No. 2019 / 0133634, filed on September 27, 2019, and U.S. Patent Application Publication No. 2020 / 0133634, filed on January 24, 2020 (hereby incorporated by reference). For example, in some embodiments, a cancer assay panel can be designed to include multiple probes (or probe pairs) that can capture fragments that together can provide information relevant to a cancer diagnosis. In some embodiments, the panel includes at least 50, 100, 500, 1,000, 2,000, 2,500, 5,000, 6,000, 7,500, 10,000, 15,000, 20,000, 25,000, or 50,000 pairs of probes. In other embodiments, the panel comprises at least 500, 1,000, 2,000, 5,000, 10,000, 12,000, 15,000, 20,000, 30,000, 40,000, 50,000, or 100,000 probes. The plurality of probes together can comprise at least 100,000, 200,000, 400,000, 600,000, 800,000, 1,000,000, 2,000,000, 3,000,000, 4,000,000, 5,000,000, 6,000,000, 7,000,000, 8,000,000, 9,000,000, or 10,000,000 nucleotides. The probe (or probe pair) is specifically designed to target one or more genomic regions that are differentially methylated in cancer samples and non-cancer samples. The target genomic regions can be selected to maximize classification accuracy according to a size budget (determined by the sequencing budget and the desired depth of sequencing).

[0241] Samples enriched using cancer assay panels can be subjected to targeted sequencing. Samples enriched using cancer assay panels can be used to detect the presence or absence of cancer generally and / or provide a cancer classification, such as cancer type, stage of cancer, such as I, II, III, or IV, or provide the primary tissue from which the cancer is believed to originate. Depending on the purpose, the panel can include probes (or probe pairs) that target genomic regions that are differentially methylated between general cancerous (pan-cancer) samples and non-cancerous samples, or only in cancerous samples with a specific cancer type (e.g., lung cancer-specific targets). In particular, cancer assay panels are designed based on bisulfite sequencing data generated from cell-free DNA (cfDNA) or genomic DNA (gDNA) from cancer and / or non-cancer individuals.

[0242] In some embodiments, the cancer assay panel designed by the methods provided herein comprises at least 1,000 pairs of probes, each pair of which comprises two probes configured to overlap each other by an overlapping sequence comprising a 30 nucleotide fragment. The 30 nucleotide fragment comprises at least 5 CpG sites, and at least 80% of these at least 5 CpG sites are either CpG or UpG. The 30 nucleotide fragment is configured to bind to one or more genomic regions in the cancerous sample, and these one or more genomic regions have at least 5 methylation sites with aberrant methylation patterns. Another cancer assay panel comprises at least 2,000 probes, each of which is designed as a complementary hybridization probe for one or more genomic regions. Each of the genomic regions is selected based on the criteria that it comprises (i) at least 30 nucleotides, and (ii) at least 5 methylation sites, and the at least 5 methylation sites have aberrant methylation patterns and are either hypomethylated or hypermethylated.

[0243] Each of the probes (or probe pairs) is designed to target one or more target genomic regions. The target genomic regions are selected based on several criteria designed to increase selective enrichment of relevant cfDNA fragments while reducing noise and non-specific binding. For example, the panel can include probes that can selectively bind and enrich differentially methylated cfDNA fragments in cancerous samples. In this case, sequencing of the enriched fragments can provide information relevant to the diagnosis of cancer. Furthermore, the probes can be designed to target genomic regions determined to have aberrant methylation patterns and / or hypermethylated or hypomethylated patterns to provide additional selectivity and specificity of detection. For example, genomic regions can be selected when they have a methylation pattern with a low p-value according to a Markov model trained on a set of non-cancerous samples, which further covers at least 5 CpGs, 90% of which are either methylated or unmethylated. In other embodiments, genomic regions can be selected utilizing a mixture model as described herein.

[0244] Each of the probes (or probe pairs) can target a genomic region that includes at least 25bp, 30bp, 35bp, 40bp, 45bp, 50bp, 60bp, 70bp, 80bp, or 90bp. The genomic region can be selected by containing less than 20, 15, 10, 8, or 6 methylation sites. The genomic region can be selected when at least 80, 85, 90, 92, 95, or 98% of at least 5 methylation (e.g., CpG) sites are either methylated or unmethylated in the non-cancerous or cancerous sample.

[0245] Genomic regions can be further filtered to select only those that are potentially informative based on their methylation patterns, for example, CpG sites that are differentially methylated between cancerous and non-cancerous samples (e.g., aberrantly methylated or unmethylated in cancer vs. non-cancer). For selection, a calculation can be performed for each CpG site. In some embodiments, a first count is determined, which is the number of cancer-containing samples that contain fragments that overlap with that CpG (cancer count), and a second count is determined, which is the number of all samples that contain fragments that overlap with that CpG (total). Genomic regions can be selected based on a criterion that is positively correlated with the number of cancer-containing samples that contain fragments that overlap with that CpG (cancer count) and inversely correlated with the number of all samples that contain fragments that overlap with that CpG (total).

[0246] In one embodiment, the number of non-cancerous samples that have fragments overlapping the CpG site (n non-cancer ) and the number of cancer samples (n cancer ) is then counted. The probability that the sample is cancer is then calculated as, for example, (n cancer +1) / (n cancer +n non-cancer +2). CpG sites by this metric are ranked and greedily added to the panel until the panel size budget is exhausted.

[0247] Depending on whether the assay is intended to be a pan-cancer or single-cancer assay, or depending on what kind of flexibility is desired in choosing which CpG sites contribute to the panel, which samples are used for cancer counting can vary. A panel for diagnosing a specific cancer type (e.g., TOO) can be designed using a similar process. In this embodiment, for each cancer type and for each CpG site, an information gain is calculated to determine whether to include a probe targeting that CpG site. The information gain is calculated for a sample with a given cancer type compared to all other samples. For example, two random variables, "AF" and "CT". "AF" is a binary variable indicating whether there is an abnormal fragment overlapping a specific CpG site in a specific sample (yes or no). "CT" is a binary random variable indicating whether the cancer is of a specific type (e.g., lung cancer or non-lung cancer). Given "AF", mutual information can be calculated with respect to "CT". That is, how many information bits are gained about the cancer type (lung vs. non-lung in this example) if we know if there is an aberrant fragment that overlaps with a particular CpG site? This can be used to rank CpGs based on how specific they are for a particular cancer type (e.g., TOO). This procedure is repeated for multiple cancer types. For example, if a particular region is typically differentially methylated only in lung cancers (and not in other cancer types or non-cancers), then CpGs in that region should tend to have high information gain for lung cancer. For each cancer type, CpG sites will be ranked by this information gain metric and then greedily added to the panel until the size budget for that cancer type is exhausted.

[0248] Further filtering can be performed to select target genomic regions that have off-target genomic regions that are smaller than a threshold value. For example, genomic regions are selected only when there are less than 15, 10 or 8 off-target genomic regions. In other cases, filtering is performed to remove genomic regions when the sequence of the target genomic region appears more than 5, 10, 15, 20, 25 or 30 times in the genome. Further filtering can be performed to select target genomic regions when the sequence that is 90%, 95%, 98% or 99% homologous to the target genomic region appears less than 15, 10 or 8 times in the genome, or to remove target genomic regions when the sequence that is 90%, 95%, 98% or 99% homologous to the target genomic region appears more than 5, 10, 15, 20, 25 or 30 times in the genome. This is to exclude repeated probes that may pull down off-target fragments, which may undesirably affect assay efficiency.

[0249] In some embodiments, it has been shown that at least 45 bp of fragment probe overlap is required to achieve a non-negligible amount of pull-down (although this number can vary depending on the assay details). Furthermore, it has been suggested that a mismatch rate of more than 10% between the probe and fragment sequence in the overlap region is sufficient to significantly disrupt binding and therefore pull-down efficiency. Thus, sequences that can match the probe along at least 45 bp with at least 90% match rate are candidates for off-target pull-down. Thus, in one embodiment, the number of such regions is scored. The best probes have a score of 1, meaning that they match in only one place (the intended target region). Probes with low scores (e.g., less than 5 or 10) are accepted, but any probes above this score are discarded. Other cut-off values ​​can be used for specific samples.

[0250] In various embodiments, the selected target genomic regions can be located at various locations in the genome, including, but not limited to, exons, introns, intergenic regions, and other portions, In some embodiments, probes can be added that target non-human genomic regions, such as those that target viral genomic regions.

[0251] VIII. Cancer Application Examples In some embodiments, the disclosed methods, analysis systems and / or classifiers can be used to detect the presence (or absence) of cancer, monitor the progression or recurrence of cancer, monitor therapeutic response or effectiveness, determine the presence or minimal residual disease (MRD), or any combination thereof. In some embodiments, the analysis systems and / or classifiers can be used to identify the tissue of origin of the cancer. For example, the systems and / or classifiers can be used to identify cancer, such as any of the following cancer types: head and neck cancer, liver / bile duct cancer, upper gastrointestinal cancer, pancreatic / gallbladder cancer, colorectal cancer, ovarian cancer, lung cancer, multiple myeloma, lymphoma, melanoma, sarcoma, breast cancer, and uterine cancer. For example, as described herein, the classifiers can be used to generate a likelihood or probability score (e.g., from 0 to 100) that a sample feature vector is from a subject with cancer. In some embodiments, the probability score is compared to a threshold probability to determine whether the subject has cancer or not. In other embodiments, the likelihood or probability score can be assessed at different time points (e.g., before or after treatment) to monitor disease progression or to monitor treatment effectiveness (e.g., therapeutic efficacy). In still other embodiments, the likelihood or probability score can be used to make or influence clinical decisions (e.g., cancer diagnosis, treatment selection, assessment of treatment effectiveness, etc.). For example, in one embodiment, if the likelihood or probability score exceeds a threshold, a physician can prescribe an appropriate treatment. In some embodiments, a test report can be generated to provide the patient with those test results, including, for example, a probability score that the patient has a disease state (e.g., cancer), a type of disease (e.g., type of cancer), and / or a disease tissue of origin (e.g., cancer tissue of origin).

[0252] IX.A. Early Detection of Cancer In some embodiments, the methods and / or classifiers of the present disclosure are used to detect the presence or absence of cancer in a subject suspected of having cancer. For example, a classifier (as described herein) can be used to determine the likelihood or probability score that a sample feature vector is from a subject with cancer.

[0253] In one embodiment, a probability score of 60 or greater may indicate that the subject has cancer. In yet other embodiments, a probability score of 65 or greater, 70 or greater, 75 or greater, 80 or greater, 85 or greater, 90 or greater, or 95 or greater may indicate that the subject has cancer. In other embodiments, the probability score may indicate the severity of the disease. For example, a probability score of 80 may indicate a more severe form, or a later stage, of cancer compared to a score below 80 (e.g., a score of 70). Similarly, an increase in the probability score over time (e.g., at a second subsequent time point) may indicate disease progression, or a decrease in the probability score over time (e.g., at a second subsequent time point) may indicate successful treatment.

[0254] In another embodiment, the cancer log odds ratio can be calculated by taking the logarithm of the ratio of the probability of being cancerous to the probability of being non-cancerous (i.e., 1 minus the probability of being cancerous) for a test subject, as described herein. According to this embodiment, a cancer log odds ratio greater than 1 can indicate that the subject has cancer. In yet other embodiments, a cancer log odds ratio greater than 1.2, greater than 1.3, greater than 1.4, greater than 1.5, greater than 1.7, greater than 2, greater than 2.5, greater than 3, greater than 3.5, or greater than 4 indicated that the subject has cancer. In other embodiments, the cancer log odds ratio can indicate the severity of the disease. For example, a cancer log odds ratio greater than 2 can indicate a more severe form of cancer, or a later stage, compared to a score less than 2 (e.g., a score of 1). Similarly, an increase in the cancer log odds ratio over time (e.g., at a second later time point) can indicate disease progression, or a decrease in the cancer log odds ratio over time (e.g., at a second later time point) can indicate successful treatment.

[0255] According to aspects of the present disclosure, the methods and systems of the present disclosure can be trained to detect or classify multiple cancer indications. For example, the methods, systems and classifiers of the present disclosure can be used to detect the presence of one or more, two or more, three or more, five or more, or ten or more different types of cancer.

[0256] In some embodiments, the cancer is one or more of head and neck cancer, liver / bile duct cancer, upper gastrointestinal cancer, pancreatic / gallbladder cancer, colorectal cancer, ovarian cancer, lung cancer, multiple myeloma, lymphoma, melanoma, sarcoma, breast cancer, and uterine cancer.

[0257] IX.B. Cancer and Therapy Surveillance In some embodiments, the first time point is before the cancer treatment (e.g., before a resection surgery or therapeutic intervention) and the second time point is after the cancer treatment (e.g., after a resection surgery or therapeutic intervention), and the method is utilized to monitor the effectiveness of the treatment. For example, if the second likelihood or probability score is decreased compared to the first likelihood or probability score, the treatment is considered successful. However, if the second likelihood or probability score is increased compared to the first likelihood or probability score, then the treatment is considered unsuccessful. In other embodiments, both the first and second time points are before the cancer treatment (e.g., before a resection surgery or therapeutic intervention). In yet other embodiments, both the first and second time points are after the cancer treatment (e.g., before a resection surgery or therapeutic intervention), and the method is used to monitor the effectiveness of the treatment or loss of effectiveness of the treatment. In yet other embodiments, cfDNA samples may be obtained from a cancer patient at a first and second time point and analyzed, for example, to monitor cancer progression, to determine whether the cancer is in remission (e.g., following treatment), to monitor or detect residual disease or disease recurrence, or to monitor treatment (e.g., therapeutic) effectiveness.

[0258] Those skilled in the art will readily appreciate that test samples can be obtained from a cancer patient over any desired set of time points and analyzed according to the disclosed methods to monitor the patient's cancer status. In some embodiments, the first and second time points are about 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, or about 24 hours, about 1, 2, 3, 4, 5, 10, 15, 20, 25, or about 30 days, or about 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, or 12 months, or about 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5 ... , 7, 7.5, 8, 8.5, 9, 9.5, 10, 10.5, 11, 11.5, 12, 12.5, 13, 13.5, 14, 14.5, 15, 15.5, 16, 16.5, 17, 17.5, 18, 18.5, 19, 19.5, 20, 20.5, 21, 21.5, 22, 22.5, 23, 23.5, 24, 24.5, 25, 25.5, 26, 26.5, 27, 27.5, 28, 28.5, 29, 29.5, or about 30 years. In other embodiments, test samples may be obtained from a patient at least once every three months, at least once every six months, at least once every year, at least once every two years, at least once every three years, at least once every four years, or at least once every five years.

[0259] IX.C. Treatment In yet another embodiment, information obtained from any of the methods described herein (e.g., likelihood or probability scores) can be used to make or influence clinical decisions (e.g., cancer diagnosis, treatment selection, assessment of treatment efficacy, etc.). For example, in one embodiment, if the likelihood or probability score exceeds a threshold, a physician can prescribe an appropriate treatment (e.g., resective surgery, radiation therapy, chemotherapy and / or immunotherapy). In some embodiments, information such as the likelihood or probability score can be provided as a lead to a physician or to a subject.

[0260] A classifier (described herein) can be used to determine a likelihood or probability score that the sample feature vector is from a subject with cancer. In one embodiment, when the likelihood or probability exceeds a threshold, an appropriate treatment (e.g., resective surgery or therapeutic) is prescribed. For example, in one embodiment, if the likelihood or probability score is 60 or greater, one or more appropriate treatments are prescribed. In another embodiment, if the likelihood or probability score is 65 or greater, 70 or greater, 75 or greater, 80 or greater, 85 or greater, 90 or greater, or 95 or greater, one or more appropriate treatments are prescribed. In other embodiments, a cancer log odds ratio can indicate the effectiveness of a cancer treatment. For example, an increase in the cancer log odds ratio over time (e.g., in a second period after treatment) can indicate that the treatment was ineffective. Similarly, a decrease in the cancer log odds ratio over time (e.g., in a second period after treatment) can indicate a successful treatment. In another embodiment, if the cancer log odds ratio is greater than 1, greater than 1.5, greater than 2, greater than 2.5, greater than 3, greater than 3.5, or greater than 4, one or more appropriate treatments are prescribed.

[0261] In some embodiments, the treatment is one or more cancer therapy agents selected from the group including chemotherapeutic agents, targeted cancer therapy agents, differentiation therapy agents, hormonal therapy agents, and immunotherapy agents. For example, the treatment can be one or more chemotherapeutic agents selected from the group including alkylating agents, antimetabolites, anthracyclines, antitumor antibiotics, cytoskeletal destaxans, topoisomerase inhibitors, mitotic inhibitors, corticosteroids, kinase inhibitors, nucleotide analogs, platinum-based drugs, and any combination thereof. In some embodiments, the treatment is one or more targeted cancer therapy agents selected from the group including signal transduction inhibitors (e.g., tyrosine kinase and growth factor receptor inhibitors), histone deacetylase (HDAC) inhibitors, retinoic acid receptor agonists, proteasome inhibitors, angiogenesis inhibitors, and monoclonal antibody conjugates. In some embodiments, the treatment is one or more differentiation therapy agents including retinoids, e.g., tretinoin, alitretinoin, and bexarotene. In some embodiments, the treatment is one or more hormonal therapy agents selected from the group including antiestrogens, aromatase inhibitors, progestins, estrogens, antiandrogens, and GnRH agonists or analogs. In one embodiment, the treatment is one or more immunotherapy agents selected from the group including monoclonal antibody therapy, e.g., rituximab (RITUXAN) and alemtuzumab (CAMPATH), non-specific immunotherapy and adjuvants, e.g., BCG, interleukin-2 (IL-2), and interferon-α, immunomodulatory agents, e.g., thalidomide and lenalidomide (REVLIMID). It is within the ability of a skilled physician or oncologist to select the appropriate cancer therapy agent based on characteristics such as tumor type, cancer stage, previous exposure to cancer treatments or therapeutic agents, and other characteristics of the cancer.

[0262] X. Example XA Example 1 - Whole Genome Bisulfite Sequencing (WBGS) First CCGA substudy: The data shown in Figures 7A-7C were obtained from the first CCGA substudy, where training data blood samples (N=1785) were collected from individuals diagnosed with untreated cancer (including 20 tumor types and all cancer stages), as well as healthy individuals diagnosed without cancer (controls), for plasma cfDNA extraction. Another set of blood samples (N=1,010) was collected to be used for validation. Unless otherwise specified, extracted cell-free DNA (cfDNA) and genomic DNA (gDNA) from the first CCGA substudy samples were subjected to whole-genome bisulfite sequencing assays.

[0263] In the classification process, the processing system 200 treats the fragment methylation states as drawn from a mixture of potential methylation patterns. The processing system 200 assigns a relative probability to an observed fragment that it originates from a particular cancer tissue of origin.

[0264] More specifically, as described herein, a probabilistic model was fitted to sequence reads derived from multiple regions (or windows) from each cancer type (and for non-cancer or healthy samples). In this case, a mixture model was used, where each mixture component was an independent site model (where methylation at each CpG is independent from methylation at other CpGs). The model was fitted using maximum likelihood estimation to identify a set of parameters that maximizes the total log-likelihood of all fragments derived from one cancer type (or non-cancer).

[0265] For each region, for each cancer type pair (including non-cancer as the negative type), the best performing stratum was used to train a multinomial logistic regression classifier. For each sample (regardless of label), for each region, for each cancer type, and for each fragment, the log-likelihood ratio was calculated as described previously, and for each set of "stratum" values, R cancer typeThe number of fragments with > 1 in each stratum was quantified. Each quantified read in the stratum was binarized and used as a feature to train a classifier.

[0266] Finally, where specified, feature values ​​were determined (as described above) to generate predictions for unknown samples, and the generated features were used to generate cancer and / or tissue of origin predictions utilizing a trained multinomial logistic regression classifier.

[0267] Exemplary Confusion Matrix: Figures 7A, 7B, and 7C include confusion matrices that indicate the accuracy of a classifier, according to various embodiments. In some embodiments, the processing system 200 uses a confusion matrix to determine the accuracy of a classifier. The confusion matrix includes information that describes the success rate of a classifier in identifying each of the disease states.

[0268] As shown in FIG. 7A, matrix 710 includes an exemplary performance of a multinomial model-based classifier trained using a set of cfDNA samples (without tissue samples). Matrix 720 includes an exemplary run of a mixture model-based classifier trained by processing system 200 using the same set of cfDNA samples. Scores along the diagonal of the matrix indicate correct predictions, i.e., cases where the predicted tissue of origin for a fragment matches the true tissue of origin. Compared to the multinomial model-based classifier as a baseline, the mixture model-based classifier has a greater overall accuracy in predicting the presence of the cancer types shown in the matrix.

[0269] The samples in the training set can be filtered based on one or more criteria (e.g., a particular specificity level). For example, the training set includes samples that were determined to have cancer based on 98% specificity by m-score. The remaining (e.g., 2%) non-cancer samples that were (incorrectly) identified as having cancer were excluded from being displayed in the confusion matrix for clarity.

[0270] As shown in Figure 7B, matrix 730 includes an example run of a classifier based on a mixture model trained using a cross-validation training set of cfDNA samples (without tissue samples). Matrix 740 includes an example run of a classifier based on a mixture model trained using a cross-validation training set of cfDNA and tissue samples.

[0271] As shown in FIG. 7C, matrix 750 includes an exemplary implementation of a mixture model-based classifier trained using a set of cfDNA samples (without tissue samples) from a clinical trial entitled Circulating Cell-Free Genome Atlas Study ("CCGA"). Matrix 740 includes an exemplary implementation of a mixture model-based classifier trained using a set of cfDNA and tissue samples from CCGA. The CCGA study was described in Non-Patent Document 1.

[0272] XB Example 2 - Classification of cancers using targeted bisulfite sequencing from early breakouts of the second CCGA substudy Second CCGA substudy: The data shown in Figures 9A-9B, 10A-10B, 11, and 12 were obtained from an early breakout from the second CCGA substudy, where training data blood samples (N=3,132) were collected from individuals diagnosed with untreated cancer (including 20 tumor types and all cancer stages) and healthy individuals diagnosed without cancer (controls) for plasma cfDNA extraction. Another set of blood samples (N=1,354) was collected to be used for validation. In some embodiments, where specified, the training set also included training data from tissue samples (i.e., gDNA). To determine the analysis population, the training data blood samples were filtered based on several factors. For example, 105 samples were excluded as clinically unlocked, 11 samples were excluded based on eligibility criteria, 58 samples were excluded due to undetermined cancer or treatment status (inevaluable), 4 untreated samples and 72 inevaluable assays were excluded (unanalyzable), and 581 samples were reserved for future analysis. As a result, the analytical population of 2,301 samples included 1,422 cancer samples and 879 noncancer samples.

[0273] Participant demographics for the individuals in the substudies are shown below in Table 1.

[0274] [Table 1]

[0275] Table 1: Participant demographics and stage distribution. Cancer and non-cancer groups were comparable with respect to age, race, sex, and body mass index (not shown). *Includes anorectal, bladder, brain, breast, cervical, colorectal, esophageal, gastric, head and neck, hepatobiliary, lung, lymphoid neoplasms (chronic lymphocytic leukemia, lymphoma), multiple myeloma, myeloid neoplasms (acute myeloid leukemia, chronic myeloid leukemia), ovarian, pancreatic, prostate, kidney, sarcoma, and uterine cancer. †Excludes 38 participants with missing smoking status information. ‡Excludes 2 participants with missing BMI values. §Invasive cancer only. ¶Staging information unavailable.

[0276] To identify cancer- and tissue-defined methylation signals, extracted cfDNA was subjected to bisulfite sequencing assays targeting the most informative regions of the methylome as identified from GRAIL's proprietary whole-genome bisulfite sequencing assay and methylation database.

[0277] We used a methylation database that queries genome-wide fragment-level methylation patterns across 811 cancer cell methylomes representing 21 tumor types (97% SEER cancer incidence). To generate a methylation database of cancer-defined methylation signals, genomic DNA from formalin-fixed paraffin-embedded (FFPE) tumor tissues and isolated cells from tumors were subjected to whole-genome bisulfite sequencing assays. The methylation database was used for panel design and training to optimize classifier performance, as described herein. A large cancer and non-cancer methylation sequence database was generated to enable target selection for a single test capable of classifying multiple cancers with high specificity and discriminating between tissues of origin.

[0278] Target Selection and Panel Design: Target genomic regions were selected using the methylation sequence database from the CCGA study as described herein. In particular, the cfDNA sequences in the database were filtered based on p-value using non-cancer distribution, and only fragments with p<0.001 were retained. The selected cfDNA was further filtered to only retain those that were at least 90% methylated or 90% unmethylated. Then, for each CpG site in the selected fragments, the number of cancer or non-cancer samples containing fragments overlapping with that CpG site was counted. In particular, the P(cancer|overlapping fragments) of each CpG was calculated, and genomic sites with high P-values ​​were selected as common cancer targets. By design, the selected fragments had significantly lower noise (i.e., fewer overlapping non-cancer fragments).

[0279] A similar selection process was performed to discover cancer type-specific targets. CpG sites were ranked based on their information gain comparing one cancer type to all other samples (i.e., non-cancer + other cancer types). As described herein, a cancer assay panel was generated that includes probes targeting selected genomic regions. In particular, the panel was designed to detect the presence of cancer in general (i.e., vs. non-cancer) or to detect the presence of a specific cancer type (e.g., TOO). The panel includes a probe set that targets each of the selected genomic regions.

[0280] Probes were designed to overlap any of the CpG sites contained within the start / stop range of any of the target regions (eg, aberrant fragments).

[0281] Classification: In classification processing, the processing system 200 treats fragment methylation states as drawn from a mixture of potential methylation patterns. The processing system 200 assigns a relative probability to the observed fragments that they originate from cancer. In tissue of origin classification, the processing system 200 assigns a relative probability to the observed fragments that they originate from a particular tissue. The processing system 200 combines fragments characterizing cancer and tissue of origin across target regions to classify cancer vs. non-cancer and / or identify tissue of origin. In binary cancer classification, the processing system 200 estimates sensitivity at 99% specificity.

[0282] More specifically, as described in Example VI.a, a probabilistic model was fitted to sequence reads derived from multiple regions (or windows) from each cancer type (and for non-cancerous or healthy samples), the identified features, and a trained multinomial logistic regression classifier. To generate predictions for unknown samples, feature values ​​were determined (as described above), and the generated features were used to generate cancer and / or tissue of origin predictions utilizing a trained multinomial logistic regression classifier.

[0283] Figures 9A and 9B show the sensitivity of the primary tissue classifier generated by the method described in this disclosure. Sensitivity is reported at 99% specificity, with 95% confidence intervals shown. Figure 9A shows the model predictions for a pre-specified list of cancers. Figure 9B shows the model predictions for other cancers included in the CCGA study. Demographic information alone (baseline modeling) correctly classified <5% of participants. Overall sensitivity was 76.1% (95% CI: 73.1-78.9%) among the pre-specified list of cancers (anorectal, breast [HR negative], colorectal, esophageal, gastric, head and neck, hepatobiliary, lung, lymphoid neoplasms [chronic lymphocytic leukemia, lymphoma], multiple myeloma, ovarian, pancreatic). Sensitivity was 68.8% (95% CI: 64.8-72.6%) in early stage (I-III) cancers in this cohort. The overall sensitivity was 55.1% (95% CI: 52.5-57.7%) across all cancer types and stages. In early stage (I-III) cancers, the sensitivity was 43.8% (95% CI: 40.7-46.8%).

[0284] Figures 10A and 10B show the sensitivity of the primary tissue classifier at various cancer stages. As indicated in the legend, sensitivity by individual stage of the pre-specified cancer of interest in the aggregate is reported at 99% specificity. Numbers in boxes represent the total number of samples included at each stage. 95% confidence intervals are shown. "Lymphoma" includes lymphoma (stages I-IV) and chronic lymphocytic leukemia (unstaged, included as "NI").

[0285] Figure 11 shows a run grid depicting the accuracy of tissue of origin localization. Using the tissue of origin classifier with the methylation database of stage I-IV samples, for each sample there is a match between the true (x-axis) and predicted (y-axis) tissue of origin. The sloped legend corresponds to the percentage of predicted tissue of origin (y-axis) that was correct (x-axis). This analysis showed that the accuracy of tissue of origin localization (fraction of all TOO predictions that were correct) was higher using the methylation database (p=0.0066). This was consistent in stage I-III prediction, i.e. 89.9% (384 / 427), as further shown in Table 2.

[0286] [Table 2]

[0287] Table 2: Primary tissue performance improves when including a methylation database. * p values ​​were calculated using the Stuart-Maxwell test. † Indeterminate calls were defined as samples detected as cancer but without a reliable tissue of origin assignment. ‡ Samples not called by tissue of origin analysis were classified as non-cancerous.

[0288] A valid multi-cancer test should ideally simultaneously detect clinically significant cancers across stages with extremely high specificity (thus should have a single fixed low false positive rate) and accurately determine tissue of origin. To demonstrate the potential of this approach, simultaneous detection (sensitivity reported at 99% specificity) and tissue of origin determination for a pre-specified list of cancer types at individual stages, in aggregate, are displayed in Figure 12. Thus, Figure 12 shows the accuracy and sensitivity of the tissue of origin classifier at various cancer stages.

[0289] Figures 13A and 13B show the receiver operating characteristic (ROC) curves of the primary tissue classifier. The ROC curves show the classifier performing with a sensitivity of 55% for all cancers and 76% for multiple cancers at a specificity of 99%.

[0290] These data show that the classification method using targeted methylation features detected multiple cancer types simultaneously at early stages with a specificity (99%) suitable for population screening. Detection of multiple cancers was achieved with a single fixed low false positive rate. The method also accurately localized the tissue of origin, which should streamline downstream diagnostic workup. Furthermore, the performance of the classifier was improved by incorporating data from a large methylation database.

[0291] Together, this supports the potential clinical applicability of the method described in this disclosure as an early multi-cancer detection test for a number of clinically significant cancer types.

[0292] XC Example 3 - Classification of cancer using targeted bisulfite sequencing from the complete second CCGA substudy Mixture model classifier generation: To maximize performance, the predictive cancer model described in this example was trained using sequence data obtained from multiple samples of known cancer types and non-cancer from both CCGA substudies (CCGA1 and CCGA2), multiple tissue samples for known cancer obtained from CCGA1, and multiple non-cancer samples from the STRIVE study (see Non-Patent Document 2). The STRIVE study is a prospective multi-center observational cohort study to validate assays for early detection of breast cancer and other invasive cancers, from which additional non-cancer training samples were obtained to train the classifier described herein. Known cancer types included from the CCGA sample set included the following: breast, lung, prostate, colorectal, kidney, uterus, pancreas, esophagus, lymphoma, head and neck, ovarian, hepatobiliary, melanoma, cervix, multiple myeloma, leukemia, thyroid, bladder, stomach, and anorectal. Thus, the model can be a multi-cancer model (or multi-cancer classifier) ​​for detecting one or more, two or more, three or more, four or more, five or more, ten or more, or twenty or more different types of cancer. 4,841 participants (2,836 cancer, 2,005 non-cancer) from the CCGA study and 2,202 non-cancer participants from the STRIVE study were included in this pre-specified analysis. Of these, 3,133 samples from CCGA were allocated to training (1,742 cancer, 1,391 non-cancer) and 1,354 were allocated to validation (740 cancer, 614 non-cancer). 1,587 samples from STRIVE were allocated to training and 615 to validation. Participant disposition is shown. Overall, 3,052 samples in training (1,531 cancer, 1,521 non-cancer) and 1,264 samples in validation (654 cancer, 610 non-cancer) were analyzable and were in the pre-specified primary analysis population. Additional details regarding the CCGA2 substudy and the analyses detailed in this example are described in Non-Patent Document 3.

[0293] Classifier performance data shown below was reported for locked classifiers trained on cancer and non-cancer samples taken from CCGA2, a CCGA substudy, and on non-cancer samples from STRIVE. Individuals in the CCGA2 substudy were different from those in the CCGA1 substudy, in which cfDNA was used to select the target genome (as described in U.S. Patent Application Publication No. 2019 / 0133634, filed April 2, 2019, U.S. Patent Application Publication No. 2019 / 0133634, filed September 27, 2019, and U.S. Patent Application Publication No. 2020 / 0133634, filed January 24, 2020, both of which are incorporated herein by reference). From the CCGA2 study, blood samples were collected from individuals diagnosed with untreated cancer (including 20 tumor types and all cancer stages) and healthy individuals diagnosed without cancer (controls). In STRIVE, blood samples were collected from women within 28 days of their screening mammogram. Cell-free DNA (cfDNA) was extracted from each sample and treated with bisulfite to convert unmethylated cytosines to uracils. The bisulfite-treated cfDNA was enriched for informative cfDNA molecules using hybridization probes designed to enrich for bisulfite-converted nucleic acids derived from each of multiple target genomic regions in three cancer assay panels: (1) pan-cancer assay panel #4 described and disclosed in U.S. Patent No. 5,339,663 (herein labeled as Assay Panel A), (2) pan-cancer assay panel #5 described and disclosed in U.S. Patent No. 5,339,663 (herein labeled as Assay Panel B), and (3) a large proprietary pan-cancer assay panel (Assay Panel C, described below). The enriched bisulfite-converted nucleic acid molecules were sequenced using paired-end sequencing on an Illumina platform (San Diego, Calif.) to obtain a set of sequence reads for each of the training samples, and the resulting read pairs were aligned to a reference genome and assembled into fragments to identify methylated and unmethylated CpG sites.

[0294] Mixture model-based feature quantification For each cancer type (including non-cancer), a probability mixture model was trained and utilized to assign a probability to each fragment from each cancer and non-cancer sample based on how likely it was that the fragment would be observed in a given sample type.

[0295] Fragment-level analysis Briefly, for each sample type (cancer and non-cancer samples), for each region (each region was used as is if it was smaller than 1 kb, or otherwise subdivided into 1 kb regions with a length of 50% overlap between adjacent regions (e.g., 500 base pairs overlapping)), a probabilistic model was fitted to fragments derived from the training samples for each cancer and non-cancer type. The probabilistic models trained for each sample type were mixture models, and each of the three mixture components was an independent site model where methylation at each CpG was assumed to be independent from methylation at other CpGs. Fragments were excluded from the model if they had a p-value (from the non-cancer Markov model) greater than 0.01, were marked as duplicate fragments, the fragments had a bag size greater than 1 (for target methylation samples only), they did not cover at least one CpG site, or the fragment length was greater than 1000 bases. Retained training fragments were assigned to a region if they overlapped with at least one CpG from that region. If a fragment overlapped with CpGs in multiple regions, it was assigned to all of them.

[0296] Local Source Model Each probabilistic model was fitted using maximum likelihood estimation to identify the set of parameters that maximized the log-likelihood of all fragments derived from each sample type, subject to a regularization penalty. In particular, for each classification domain, a set of probabilistic models was trained, one for each training label (i.e., one for each cancer type and one for non-cancer). Each model took the form of a Bernoulli mixture model with three components. Mathematically,

[0297]

number

[0298] n is the number of mixture components, which is set to 3, and m i ∈{0,1} is the observed methylation of the fragment at position i, and f k is the fractional allocation to component k (where f k ≧0 and f k = 1), β ki is the methylation fraction in component k at CpG i. The product over i included only positions where the methylation status could be discerned from sequencing. The parameters of each model {f k ,β ki The maximum likelihood value of} is estimated by using the rprop algorithm (e.g., the rprop algorithm described in Non-Patent Document 4) and is expressed as a beta-distributed prior, β ki The total log-likelihood of a single training label fragment, subject to the regularization penalty above, is maximized. Mathematically, the quantity that is maximized is:

[0299]

number

[0300] r is the regularization strength, which was set to 1.

[0301] Feature quantity Once the probabilistic model was trained, a set of numerical features was computed for each sample. In particular, features were extracted for each fragment from each training sample, in each region, for each cancer type and non-cancer sample. The extracted features were records of outlier fragments (i.e., aberrantly methylated fragments), defined as those whose log-likelihood under a first cancer model exceeded the log-likelihood under a second cancer model or non-cancer model by at least a threshold stratum value. Outlier fragments were recorded separately for each genomic region, sample model (i.e., cancer type), and stratum (for stratum 1, 2, 3, 4, 5, 6, 7, 8, and 9), resulting in nine features per region for each sample type. In this way, each feature was defined by three properties: the genomic region, the "positive" cancer type label (excluding non-cancer), and a stratum value selected from the set {1, 2, 3, 4, 5, 6, 7, 8, 9}. The numerical value of each feature was defined as the number of fragments in that region, as follows:

[0302]

number

[0303] These probabilities were defined by equation (1) using maximum likelihood estimated parameter values ​​corresponding to a “positive” cancer type (in the numerator of the logarithm) or non-cancer (in the denominator).

[0304] Feature ranking For each set of pairwise features, the features were ranked using mutual information based on their ability to distinguish the first cancer type (which defined the log-likelihood model from which the features were derived) from the second cancer type or non-cancer. In particular, for each unique pair of class labels, two ranked lists of features were compiled: one in which the first label was assigned as “positive” and the second label as “negative”, and another in which the positive / negative assignments were swapped (except for the “non-cancer” label, which was only allowed as a negative label). For each of these ranked lists, only features whose positive cancer type label (as in Eq. (3)) matched the positive label under consideration were included in the ranking. For each such feature, the fraction of training samples with non-zero feature values ​​was calculated separately for the positive and negative labels. Features for which this fraction was larger in the positive label were ranked by their mutual information with respect to that pair of class labels.

[0305] The top ranked 256 features from each pairwise comparison were identified and added to the final feature set for each cancer type and non-cancer. To avoid redundancy, if more than one feature was selected from the same positive type and genomic region (i.e., for multiple negative types), only the one assigned the lowest (most informative) rank for that cancer type pair was kept, breaking the link by choosing a higher hierarchical value. Features in the final feature set for each sample (cancer type and non-cancer) were binarized (any feature value greater than 0 was set to 1, such that all features were either 0 or 1).

[0306] classifier training The training samples were then split into separate 5-fold cross-validation training sets and a two-stage classifier was trained for each fold, in each case trained on 4 / 5 of the training samples and the remaining 1 / 5 was used for validation.

[0307] In the first stage of training, a binary (two-class) logistic regression model for detecting the presence of cancer was trained to distinguish cancer samples from non-cancer samples (regardless of TOO). When training this binary classifier, sample weights were assigned to male non-cancer samples to offset the gender imbalance in the training set. For each sample, the binary classifier outputs a prediction score indicating the likelihood of the presence or absence of cancer.

[0308] In the second stage of training, a parallel multi-class logistic regression model for determining cancer tissue of origin was trained using TOO as the target label. Only cancer samples that received a score above the 95th percentile of non-cancer samples in the first stage classifier were included in the training of this multi-class classifier. For each cancer sample used in training the multi-class classifier, the multi-class classifier outputs a predicted value for the cancer type being classified, where each predicted value is the likelihood that a given sample has a particular cancer type. For example, the cancer classifier can return a cancer prediction for a test sample, including a predicted score for breast cancer, a predicted score for lung cancer, and / or a predicted score for no cancer.

[0309] Both binary and multiclass classifiers were trained by stochastic gradient descent with mini-batches, and in each case training was terminated early when performance on the validation fold (as assessed by cross-entropy loss) began to degrade. At each stage, the scores assigned by the five cross-validated classifiers were averaged to predict to samples outside the training set. Scores assigned to gender-inappropriate cancer types were set to 0, and the remaining values ​​were renormalized to sum to 1.

[0310] The scores assigned to the validation folds in the training set were retained for use in assigning cutoff values ​​(thresholds) to some performance metrics of the target. In particular, the probability scores assigned to the training set non-cancer samples were used to define a threshold corresponding to a particular specificity level. For example, for a desired specificity target of 99.4%, the threshold was set to the 99.4th percentile of the cross-validated cancer detection probability scores assigned to the non-cancer samples in the training set. Training samples with probability scores above the threshold were called positive for cancer.

[0311] A TOO or cancer type assessment was then made from the multi-class classifier for each training sample determined to be positive for cancer. First, the multi-class logistic regression classifier assigned a set of probability scores to each sample, one for each predicted cancer type. The confidence of these scores was then assessed as the difference between the highest and second highest scores assigned by the multi-class classifier for each sample. The cross-validated training set scores were then used to identify the lowest threshold, and thus, of the cancer samples in the training set whose top two score differences exceeded the threshold, 90% were assigned the correct TOO label as their highest score. In this way, the scores assigned to the validation fold during training were further used to determine a second threshold for distinguishing between reliable and uncertain TOO calls.

[0312] At prediction time, samples that received a score from the binary (first stage) classifier below a predefined specificity threshold were assigned the "non-cancer" label. For the remaining samples, those whose top two TOO score deltas from the second stage classifier were below a second predefined threshold were assigned the "indeterminate cancer" label. The remaining samples were assigned the cancer label to which the TOO classifier assigned the highest score.

[0313] Running classifiers on targeted genomic region panels The differential value of the target genomic regions of assay panels A-C was evaluated by testing the ability of cancer classifiers to detect cancer and any of 20 different cancer types according to the methylation status of these target genomic regions. For assay panels A-B, performance was evaluated across a training set of 1,531 cancer samples and 1,521 non-cancer samples used to train the classifiers, as shown in Table 1. For assay panel C, performance was evaluated using 1,264 samples (654 cancer, 610 non-cancer) in validation on a classifier trained using the same set of 3,052 samples (1,531 cancer, 1,521 non-cancer) used in training assay panels A-B. For each sample, differentially methylated cfDNA was enriched using a bait set that included all of the target genomic regions included in assay panels A-C. The classifier was then forced to provide a cancer decision based solely on the methylation status of the target genomic regions in the list being evaluated. A two-stage classifier embodiment including a binary (two-class) logistic regression classifier model for detecting the presence of cancer trained to distinguish cancer samples from non-cancer (regardless of TOO) and a second stage trained multi-class logistic regression classifier model for determining cancer tissue of origin was trained with TOO as the target label as described previously in this example. Both classifier models were trained and validated using model-based featurization, also as described previously.

[0314] [Table 3]

[0315] Assay Panels A and B: Results from the classifier performance analysis for assay panels A and B are presented in Figure 26A and Figure 27A. In each figure, part A is the receiver operator curve (ROC) showing true positive and false positive results for cancer or no cancer determination. The asymmetric shape of these ROC curves indicates that the classifier is designed to minimize false positive results. The area under the curve for assay panels A and B was 0.83 for both assay panels.

[0316] Cancer type (i.e., TOO) determination was made using the classifier for all samples that tested positive for cancer. Figures 26B and 27B contain confusion matrices showing the accuracy of TOO accuracy for assay panels A and B, respectively. The confusion matrices contain information describing the success rate of the classifier in identifying each of the cancer types and ruling out indeterminate cancer calls.

[0317] As shown in Figure 26B and Figure 27B, the TOO confusion matrix shows the performance of the multi-class logistic regression classifier as described above. The agreement between the actual (x-axis) and predicted (y-axis) tissue of origin for each sample using the targeted methylation classifier is shown. Scores along the diagonal of the matrix indicate correct predictions, i.e., cases where the predicted tissue of origin for a fragment matches the true tissue of origin. As shown in Figure 26B, cancer assay panel A had a TOO accuracy of about 90.8% (711 / 783) when the uncertain cancer calls were excluded. And Figure 27B shows that assay panel B had a TOO accuracy of about 90.3% (705 / 781) when the uncertain cancer calls were excluded.

[0318] These classifier results are further summarized in Tables 2-3, which show the accuracy of cancer detection and cancer typing performed at a specificity of 0.990, indicating a 1% false positive rate. The results are delineated by cancer stage. They show improved cancer detection and cancer typing for samples from individuals with late stage cancer (e.g., stage III) compared to samples from individuals with early stage cancer (e.g., stage II). For all cancer stages (without segregation by stage), cancer typing was approximately 89% correct for both assay panels A and B (including indeterminate cancer calls).

[0319] Table 2. Classification accuracy using genomic regions of assay panel A. Data for cancer presence and cancer type at a specificity of 0.990 show percentage accuracy, 95% confidence interval in brackets, and number correctly assigned relative to the total in parentheses.

[0320] [Table 4]

[0321] Table 3. Classification accuracy using genomic regions of assay panel B. Data for cancer presence and cancer type at a specificity of 0.990 show percentage accuracy, 95% confidence interval in brackets, and number correctly assigned relative to the total in parentheses.

[0322] [Table 5]

[0323] Assay Panel C: As described above, a third, large proprietary pan-cancer assay panel was also tested. Assay Panel C was designed from WGBS data obtained from the first CCGA substudy CCGA1 using the feature selection methods disclosed in U.S. Patent Application Publication No. 2019 / 0233636, filed September 27, 2019, and U.S. Patent Application Publication No. 2020 / 0133994, filed January 24, 2020 (hereby incorporated by reference). The large proprietary targeted methylation panel covered 103,456 distinct regions (17.2 Mb), covering 1,116,720 CpGs. Assay panel C contained 363,033 CpGs in 68,059 regions (7.5 Mb) covered by probes targeting hypomethylated fragments, 585,181 CpGs in 28,521 regions (7.4 Mb) covered by probes targeting hypermethylated fragments, and 218,506 CpGs in 6,876 regions (2.3 Mb) targeting both types of fragments. Each aberrant target region contained between 1 and 590 CpGs, with a median CpG count of 3 for hypomethylated and 6 for hypermethylated target regions. CpGs were present in the following genomic regions: 193,818 (17%) in the region 1 to 5 kbp upstream of the transcription start site (TSS), 278,872 (24%) in promoters (<1 kbp upstream of the TSS), 500,996 (43%) in introns, 292,789 (25%) in exons, 247,752 (21%) at intron-exon boundaries, 134,144 (11%) in 5′-untranslated regions, 182,174 (16%) between genes, and the remaining 1,817 (<1%) were not annotated. Each CpG may have received multiple annotations due to overlapping genes and / or transcripts, so percentages are of the total number of CpGs and do not sum to 100%.

[0324] For this evaluation, samples were split into a training set (n=4,720) and an independent validation set (n=1,969). A total of 4,316 participants (training: 3,052 [1,531 cancers: stage I: 28%, stage II: 25%, stage III: 20%, stage IV: 24%, lost / not expected: 3%, 1,521 non-cancer]; validation: 1,264 [654 cancers: stage I: 28%, stage II: 25%, stage III: 21%, stage IV: 23%, lost / not expected: 3%, 610 non-cancer]) were analyzable and included in the primary analysis population.

[0325] Results from classifier performance analysis for the training and validation sets are shown in Figures 28-30. Panel A of Figure 28 shows specificity results for both the training and validation sets, and Panel B shows sensitivity for pre-specified cancers (a subset of 12 high signal cancers (anal, bladder, colon / rectum, esophagus, head and neck, liver / bile duct, lung, lymphoma, ovarian, pancreas, plasma cell neoplasms, stomach) based on results from the first substudy and mortality data), as well as for all cancer types (>20) in stages I to IV. Panel C of Figure 28 shows tissue of origin (TOO) accuracy results for both the training and validation sets, and Panel B shows sensitivity for pre-specified cancers and for all cancer types in stages I to IV. Figure 29 shows the TOO confusion matrix for both the training and validation sets, and Figure 30 shows sensitivity results for pre-specified cancer types for both the training and validation sets.

[0326] In Figure 28, sensitivity (y-axis) is reported by clinical stage (x-axis) among pre-specified cancer types (left panel) and all cancer types (right panel) for training (orange) and validation (teal blue). Tissue of origin accuracy (y-axis) is reported by clinical stage (x-axis) among pre-specified cancer types (left panel) and all cancer types (right panel) for training (orange) and validation (teal blue). Numbers indicate samples in training|validation set.

[0327] As shown in Figure 28, the classifier consistently achieved high specificity between the cross-validated training set and the independent validation set (99.8% [95% CI: 99.4-99.9%] vs. 99.3% [98.3-99.8%], respectively, P=0.095). This reflected a single consistent false positive rate (FPR) of less than 1% across all 20 cancer types. Specificity in the validation set was similar for CCGA and STRIVE non-cancer samples (99.3% [97.4-99.9%] vs. 99.4% [97.9-99.9%], respectively), supporting that performance was not biased by site or sample selected. Sensitivity was consistent in the training and validation sets. For all cancers, stage I-III sensitivity was 44.2% (95% CI: 41.3-47.2%) vs. 43.9% (39.4-48.5%), respectively (P = 1.000). For the prespecified set of 12 high-signal cancers, stage I-III sensitivity was 69.8% (65.6-73.7%) vs. 67.3% (60.7-73.3%), respectively (P = 0.988). Similarly, for stage I-IV sensitivity across all cancer types, it was 55.2% (52.7-57.7%) vs. 54.9% (51.0-58.8%), respectively (P = 0.897), and for prespecified cancers it was 77.9% (75.0-80.7%) vs. 76.4% (71.6-80.7%), respectively (P = 0.573).

[0328] Sensitivity also increased with increasing disease stage, as shown in Figure 28. In validation, sensitivity for prespecified cancer types was 39% (27-52%) for stage I (n=62), 69% (56-80%) for stage II (n=62), 83% (75-90%) for stage III (n=102), and 92% (86-96%) for stage IV (n=130). Across all cancer types, sensitivity was 18% (13-25%) for stage I (n=185), 43% (35-51%) for stage II (n=166), 81% (73-87%) for stage III (n=134), and 93% (87-96%) for stage IV (n=148).

[0329] The performance on each tumor type is shown in Figure 30. Sensitivity at 99.8% specificity (training, orange) or 99.3% specificity (validation, teal) with 95% confidence intervals is reported for each cancer type with at least 50 samples. Clinical stage is shown below the plot, as is the number of samples in training and validation.

[0330] As shown in Figure 28, a pre-specified analysis of TOO accuracy (fraction of all correct TOO predictions) found that TOO was predicted in 96% (344 / 359) of samples with cancer-like signals in the validation set, among which the accuracy was 93% (321 / 344). Accuracy was consistent between the training and validation sets and across stages. The classifier distinguished between >20 cancer types included in the study, and performance was consistent within individual cancer types.

[0331] FIG. 29 shows confusion matrices depicting the accuracy of tissue of origin localization in (A) the training set and (B) the validation set. The agreement between actual (x-axis) and predicted (y-axis) tissue of origin per sample using the targeted methylation classifier is shown. Colors correspond to the percentage of predicted tissue of origin calls. Included participants (training: n=844, validation: n=359) are people with cancer who were predicted to have cancer with 99.8% specificity (training) or 99.3% specificity (validation). A tissue of origin call was assigned in 95% of cases (806 / 844) in training and 96% of cases (344 / 359) in validation, and the call was correct in 92% of cases (744 / 806) in training and 93% of cases (321 / 344) in validation.

[0332] XD Example 4 - Adjusting the binary classification threshold According to a generalized embodiment of binary cancer classification, the analysis system determines a cancer score for the test sample based on the sequencing data of the test sample (e.g., methylation sequencing data, SNP sequencing data, other DNA sequencing data, RNA sequencing data, etc.). The analysis system compares the cancer score of the test sample to a binary threshold cutoff for predicting whether the test sample is likely to have cancer. The binary threshold cutoff can be adjusted using TOO thresholding based on one or more TOO subtype classes. The analysis system can further generate a feature vector for the test sample for use in a multi-class cancer classifier to determine a cancer prediction indicative of one or more likely cancer types.

[0333] FIG. 24A shows a confusion matrix illustrating the performance of a trained cancer classifier according to an exemplary implementation. The cancer classifier was trained according to the principles described above. The TOO labels include lymphoid tumors, lung, kidney, non-cancer, head and neck, prostate, breast, upper gastrointestinal tract, liver and bile duct, colorectal, cervix, pancreas and gallbladder, uterus, sarcoma, bladder and urothelium, ovary, anorectal, unknown, melanoma, multiple myeloma, bone marrow tumor, and thyroid. Notably, the classification accuracy is 89.1% across the 1,151 samples considered in this held-up set.

[0334] FIG. 24B shows a confusion matrix illustrating the performance of the trained cancer classifier with additional hematological cancer subtypes. The cancer classifier was trained according to the principles described above. In contrast to FIG. 24A, the TOO labels of the hematological subtypes are adjusted. In FIG. 24A, the hematological subtypes include lymphoid neoplasms, multiple myeloma, and myeloid neoplasms. In FIG. 24B, the hematological subtypes include Hodgkin's lymphoma (HL), NHL aggressive, NHL indolent, myeloid, circulating lymphoma (or lymphoid), and plasma cell. Remarkably, the classification accuracy is 87.5% across 1,076.

[0335] 25A and 25B show graphs illustrating cancer prediction accuracy for multiple cancer types across cancer stages. In this example, a cancer classifier is trained after removing non-cancerous samples according to process 1000 described above. The analysis system determined multiple TOO thresholds for hematological subtypes. The analysis system removed non-cancerous samples with at least one TOO probability equal to or greater than the corresponding TOO threshold for the hematological subtype. The illustrated graphs show classification sensitivity across various cancer stages for the following cancer types: anorectal, bladder and urothelial, breast, cervical, colorectal, head and neck, liver and bile duct, lung, melanoma, ovarian, pancreas and gallbladder, prostate, kidney, sarcoma, thyroid, upper gastrointestinal tract, and uterus. The graphs for each cancer type show the predictive sensitivity on each stage of the cancer type, with the first cancer classifier labeled as "locked_v1_orgi" and without TOO thresholding, and the second cancer classifier labeled as "v2_custom" and with TOO thresholding. Notably, for many cancer types, the second cancer classifier has higher predictive accuracy while maintaining tight confidence intervals, given more samples available for validation. Of particular note, there is a higher predictive accuracy for many cancer types at the stage I and II levels, indicating the improved predictive potential with TOO thresholding in early stage cancers.

[0336] XI. Additional Considerations The above description of embodiments of the present disclosure has been presented for purposes of illustration. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Those skilled in the art will recognize that numerous modifications and variations are possible in light of the above disclosure.

[0337] In some portions of this specification, embodiments of the disclosure are described in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to effectively convey the substance of their work to others skilled in the art. While these operations are described functionally, computationally, or logically, it will be understood that they are implemented by computer programs or equivalent electrical circuits, microcode, or the like. Moreover, it has proven convenient at times, without loss of generality, to refer to these configurations of operations as modules. The described operations and their associated modules can be implemented in software, firmware, hardware, or any combination thereof.

[0338] Any of the steps, operations, or processes described herein can be performed or implemented using one or more hardware or software modules, alone or in combination with other devices. In some embodiments, a software module is implemented using a computer program product that includes a computer-readable non-transitory medium containing computer program code that can be executed by a computer processor to perform any or all of the steps, operations, or processes described.

[0339] Embodiments may also relate to products produced by the computing processes described herein. Such products may include information resulting from the computing processes, which information may be stored in a non-transitory tangible computer-readable storage medium, and may include any of the embodiments of the computer program products or other data combinations described herein.

[0340] Finally, the language used herein has been selected primarily for ease of reading and instruction, and it should not be selected to define or limit the subject matter of the invention. Accordingly, the scope of the invention is not intended to be limited by this detailed description, but rather by any claims that issue applications based on this specification. Thus, the disclosure of the embodiments herein is intended to illustrate, rather than limit, the scope of the invention, which is set forth in the following claims.

Claims

1. 1. A method for identifying cancer signals in a test sample by analyzing sequence reads to generate features, comprising: obtaining a plurality of sequence reads from a test sample; applying a first probability model associated with a first disease state to each sequence read of the test sample to output a first probability that the sequence read is associated with the first disease state, wherein the first probability model is trained with a first plurality of reference sequence reads derived from one or more samples associated with the first disease state; applying a second probability model associated with a second disease state to each sequence read of the test sample to output a second probability that the sequence read is associated with the second disease state, wherein the second probability model is trained with a second plurality of reference sequence reads derived from one or more samples associated with the second disease state; determining a feature vector for the test sample based on the first probability and the second probability; applying a trained cancer classifier to the feature vector for the test sample to predict the likelihood of the test sample having a cancer signal; The method includes:

2. 2. The method of claim 1, wherein the first disease state is cancer and the second disease state is non-cancer.

3. 2. The method of claim 1, wherein the first disease state is a first type of cancer and the second disease state is a second type of cancer, and the first type of cancer and the second type of cancer are different.

4. The method comprises: obtaining a plurality of reference sequence reads from a third, fourth, fifth, sixth, seventh, eighth, ninth, and / or tenth reference sample, each of the third, fourth, fifth, sixth, seventh, eighth, ninth, and / or tenth reference samples having a different disease state, each of the different disease states being a different type of cancer; training a third, fourth, fifth, sixth, seventh, eighth, ninth, and / or tenth probabilistic model using the third, fourth, fifth, sixth, seventh, eighth, ninth, and / or tenth plurality of reference sequence reads, each of the third, fourth, fifth, sixth, seventh, eighth, ninth, and / or tenth probabilistic models each associated with a different type of cancer; The method of claim 1 further comprising:

5. 5. The method according to any one of claims 2 to 4, wherein the cancer or type of cancer is selected from the group comprising breast cancer, uterine cancer, cervical cancer, ovarian cancer, bladder cancer, urothelial carcinoma of the renal pelvis and ureter, non-urothelial renal cancer, prostate cancer, anorectal cancer, colorectal cancer, squamous cell carcinoma of the esophagus, non-squamous esophageal cancer, gastric cancer, hepatobiliary cancer arising from hepatocytes, hepatobiliary cancer arising from cells other than hepatocytes, pancreatic cancer, head and neck cancer associated with human papillomavirus, head and neck cancer not associated with human papillomavirus, lung adenocarcinoma, small cell lung cancer, squamous cell lung cancer and lung cancer other than adenocarcinoma or small cell lung cancer, neuroendocrine cancer, melanoma, thyroid cancer, sarcoma, multiple myeloma, lymphoma and leukemia.

6. 6. The method of claim 5, wherein the cancer type is further selected from the group including brain cancer, vulvar cancer, vaginal cancer, testicular cancer, pleural mesothelioma, peritoneal mesothelioma, and gallbladder cancer.

7. 2. The method of claim 1, wherein the first disease state comprises a first tissue of origin and the second disease state comprises a second tissue of origin.

8. 8. The method of claim 7, wherein the first primary tissue or the second primary tissue is selected from the group including breast tissue, thyroid tissue, lung tissue, bladder tissue, cervical tissue, small intestine tissue, colorectal tissue, esophageal tissue, stomach tissue, tonsil tissue, liver tissue, ovarian tissue, fallopian tube tissue, pancreatic tissue, prostate tissue, kidney tissue, and uterine tissue.

9. 9. The method of claim 8, wherein the first primary tissue or the second primary tissue is further selected from the group including brain tissue and cells, endocrine tissue and cells, vascular endothelial tissue and cells, head and neck tissue and cells, pancreatic exocrine tissue and cells, pancreatic endocrine tissue and cells, lymphatic tissue and cells, mesenchymal tissue and cells, bone marrow tissue and cells, pleural tissue and cells, muscle tissue and cells, bone marrow tissue and cells, adipose tissue and cells, gallbladder tissue and cells.

10. The method of claim 1 , wherein the first probability model or the second probability model is a constant model, a binomial model, an independent sites model, a neural net model, or a Markov model.

11. determining a methylation ratio for each of a plurality of CpG sites in the first plurality of reference sequence reads or the second plurality of reference sequence reads, wherein the first probability model or the second probability model is parameterized by a product of the methylation ratios. The method of claim 1 further comprising:

12. 12. The method of claim 11, wherein the first probabilistic model or the second probabilistic model is parameterized by a sum of multiple mixture components, each associated with a product of the proportions of methylation.

13. The method of claim 12 , wherein each mixture component of the plurality of mixture components is associated with a percentage allocation, the percentage allocations summing to one.

14. For each sequence read of the first plurality of reference sequence reads, the second plurality of sequence reads, or the plurality of sequence reads from the test sample, determining whether the sequence read is hypomethylated or hypermethylated by determining whether at least a threshold number of the CpG sites, having at least a threshold percentage of CpG sites, are unmethylated or methylated, respectively. The method of claim 1 further comprising:

15. For the first plurality of reference sequence reads, the second plurality of sequence reads, or each sequence read of the plurality of sequence reads, determining whether the sequence read is aberrantly methylated; filtering the first plurality of reference sequence reads using p-value filtering by removing sequence reads from the first plurality of reference sequence reads having a p-value below a threshold; The method of claim 1 further comprising:

16. The step of training the first probabilistic model comprises: determining, for the probabilistic model, a set of parameters that maximizes the sum of log-likelihoods of the first plurality of reference sequence reads derived from a subject associated with the first disease state associated with the probabilistic model.

2. The method of claim 1, comprising:

17. The step of training the first probabilistic model and the second probabilistic model includes: For each of the plurality of windows, selecting a first set of sequence reads from the first plurality of reference sequence reads within the window and utilizing the first set of sequence reads to train the first probabilistic model for the window; selecting a second set of sequence reads from the second plurality of reference sequence reads taken from the window, and using the second set of sequence reads to train the second probability model for the window; 2. The method of claim 1, comprising:

18. 20. The method of claim 17, wherein at least one window is separated by at least a threshold number of base pairs between the CpG sites.

19. 20. The method of claim 17, wherein at least one window comprises a number of base pairs selected from the group consisting of 200 base pairs (bp) and 10 kilobase pairs (kbp).

20. 2. The method of claim 1 , wherein determining the feature vector comprises determining a feature comprising a count of outlier sequence reads of the plurality of sequence reads for which the first probability value is greater than the second probability value.

21. 2. The method of claim 1, wherein determining the feature vector comprises determining one feature comprising a total count of anonymously methylated sequence reads.

22. The method of claim 1 , wherein determining the feature vector comprises determining a feature comprising a count of fragments containing one or more particular methylation patterns.

23. 2. The method of claim 1, wherein determining the feature vector comprises applying a discriminative classifier trained within a single genomic region to output one or more features based on the first probability and the second probability for the plurality of sequence reads of the test sample.

24. 24. The method of claim 23, wherein the discriminative classifier is a multi-layer perceptron or a convolutional neural net model.

25. The step of determining the feature vector comprises: comparing the first probability to the second probability to determine a ratio of the first probability to the second probability; determining one or more features based on a count of sequence reads having a ratio that exceeds a threshold ratio; 2. The method of claim 1, comprising:

26. The method of claim 1 , wherein the first probability or the second probability is a log-likelihood value.

27. For each feature, determining a criterion for the feature when distinguishing between the first disease state and the second disease state based on the first plurality of reference sequence reads and the second plurality of reference sequence reads; identifying a set of features for determining the feature vector based on the measure of the features in distinguishing between the first disease state and the second disease state; The method of claim 1 further comprising:

28. 10. The method of claim 1, further comprising training the cancer classifier with feature vectors from a plurality of training samples, a first cohort of training samples obtained from individuals diagnosed with cancer and a second cohort of training samples obtained from individuals diagnosed without cancer.

29. 29. The method of claim 28, wherein the cancer classifier is configured as a multi-layer perceptron, a logistic regression, a support vector machine, a multinomial logistic regression, a random forest, or a neural network.

30. 2. The method of claim 1, wherein obtaining the plurality of sequence reads for the test sample comprises performing targeted methylation sequencing to generate methylation sequence reads indicative of a methylation status at each of a plurality of methylation sites.

31. 1. A non-transitory computer readable storage medium storing instructions for identifying a cancer signal in a test sample, the instructions, when executed by a processor, causing the processor to: Obtaining a plurality of reference sequence reads from a test sample; applying a first probability model associated with a first disease state to each sequence read of the test sample to output a first probability that the sequence read is associated with the first disease state, wherein the first probability model is trained with a first plurality of reference sequence reads derived from one or more samples associated with the first disease state; applying a second probability model associated with a second disease state to each sequence read of the test sample to output a second probability that the sequence read is associated with the second disease state, wherein the second probability model is trained with a second plurality of reference sequence reads derived from one or more samples associated with the second disease state; determining a feature vector for the test sample based on the first probability and the second probability; applying a trained cancer classifier to the feature vector for the test sample to predict the likelihood of the test sample having a cancer signal; A non-transitory computer-readable medium for causing a computer to perform operations including:

32. 1. A system comprising: a sequencing device configured to perform sequencing to generate a plurality of sequence reads corresponding to a plurality of nucleic acid fragments in the test sample; receiving the plurality of sequence reads from the sequencing device; applying a first probability model associated with a first disease state to each sequence read of the test sample to output a first probability that the sequence read is associated with the first disease state, the first probability model being trained with a first plurality of reference sequence reads derived from one or more samples associated with the first disease state; applying a second probability model associated with a second disease state to each sequence read of the test sample to output a second probability that the sequence read is associated with the second disease state, the second probability model being trained with a second plurality of reference sequence reads derived from one or more samples associated with the second disease state; determining a feature vector for the test sample based on the first probability and the second probability; a computing device configured to apply a trained cancer classifier to the feature vector for the test sample to predict the likelihood of the test sample having a cancer signal; A system equipped with.

Images