A machine learning model for generating confidence classifications of genomic coordinates

Abstract

The present disclosure describes a method, a non-transitory computer-readable medium, and a system that can train a genome location classification model to classify or score a genome coordinate or a genome region according to the degree to which a nucleic acid base can be accurately identified in such genome coordinate or region. For example, the disclosed system can determine a sequencing metric for a sample nucleic acid sequence or a context nucleic acid subsequence surrounding a particular nucleic acid base call. By utilizing the ground truth classification for a genome coordinate, the disclosed system can train a genome location classification model to associate data from one or both of the sequencing metric and the context nucleic acid subsequence with a confidence classification for such genome coordinate or region. After training, the disclosed system can also apply the genome location classification model to the sequencing metric or the context nucleic acid subsequence to determine an individual confidence classification for each genome coordinate or region, and then generate at least one digital file including such confidence classification for display on a computing device.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of and priority to U.S. Provisional Application No. 63 / 216,382, entitled "MACHINE-LEARNING MODEL FOR GENERATING CONFIDENCE CLASSIFICATIONS FOR GENOMIC COORDINATES," filed June 29, 2021, the entire contents of which are incorporated herein by reference. [Background technology]

[0002] In recent years, biotechnology companies and research institutions have improved hardware and software for sequencing nucleotides and identifying variant calls for samples that contain nucleic acid bases that differ from a standard or reference genome. For example, some existing nucleic acid sequencing platforms determine individual nucleic acid bases of a nucleic acid sequence by using traditional Sanger sequencing or by using sequencing by synthesis (SBS). By using SBS, existing systems can monitor thousands, tens of thousands, or even more nucleic acid polymers that are synthesized in parallel to detect more accurate nucleic acid base calls from a larger base call data set. For example, a camera in an SBS platform can capture images of illuminated fluorescent tags from nucleic acid bases incorporated into such oligonucleotides. After capturing such images, existing SBS platforms transmit the base call data (or image data) to a computing device having sequencing data analysis software to determine the nucleic acid base sequence of the nucleic acid polymer (e.g., an exon region of the nucleic acid polymer) and use a variant caller to identify any single nucleotide variants (SNVs), insertions or deletions (indels), or other variants in the nucleic acid sequence of the sample.

[0003] Despite these recent advances in sequencing and variant calling, existing sequencing data analysis software often includes variant callers that identify nucleotide variants regardless of (or without) the location of the nucleotide variant within the sequence or genome. The location of the nucleotide variant may affect the probability of identifying a variant as a true positive or false positive, as the context of the location of the variant call may affect the confidence of the call, with certain genomic regions more likely to show predictable sequences and other genomic regions more likely to show variation. Furthermore, the probability of correctly identifying a variant for a given genomic region may vary depending on the particular sequencing method or device. In the absence of built-in mechanisms for analyzing the accuracy of genomic regions and correlating variant calls with such regions, especially for certain sequencing pipelines, clinicians often use other sequencing methods (e.g., Sanger to supplement SBS sequencing) or supplemental validation tests to orthogonally validate variant calls.

[0004] A variant call for a particular variant can range between insignificant and significant depending on the genomic region of the variant call. However, existing variant callers often cannot correlate the variant call with the accuracy probability for the genomic region or location, so clinicians have limited confidence in the accuracy of the variant call. For example, a variant call that identifies a specific single nucleotide polymorphism (SNP) in the hemoglobin beta (HBB) gene can have significant implications. If a variant caller identifies a SNP at rs344 on chromosome 11, the variant caller will either correctly identify the genetic cause of sickle cell anemia or potentially miss the cause of the disease. As a further example, a variant call that correctly or inaccurately identifies a deletion of one or more copies of the hemoglobin subunit alpha 1 (HbA1) or hemoglobin subunit alpha 2 (HbA2) genes can result in either correctly identifying the genetic cause of an inherited blood disease or missing the gene deletion entirely. Thus, variant calling for such SNPs or other variants on genes can be important, but often lacks an empirical indication of the probability of accuracy for the regions in which traditional variant callers discriminate variants.

[0005] Despite the potential importance of variation and variant calling in genomic regions for nucleic acid base calling, existing nucleic acid sequencing platforms and sequencing data analysis software (collectively, hereinafter, existing sequencing systems) lack an empirically proven method for identifying reportable ranges for regions of higher or lower accuracy in the genome. Such existing sequencing systems also lack an empirically proven method for distinguishing different variant types within such reportable ranges. Existing sequencing systems further lack such empirically proven methods for identifying reportable ranges or distinguishing variant types within those ranges for a particular sequencing pipeline.

[0006] Traditionally, clinicians and biotechnology organizations may rely on features of a reference genome that are not tethered to a specific sequencing pipeline. Researchers have identified reportable region ranges in reference genomes with higher or lower accuracy, including high confidence regions of the reference genome identified by the Genome in a Bottle Consortium (GIAB) and the Global Alliance for Genomic Health (GA4GH). However, these existing reportable ranges are from the GIAB and GA4GH restricted reportable ranges to benchmark genome regions that exclude difficult genomic regions, and approximately 79-84% of the human genome is within the benchmark genome regions. They cannot distinguish between different types of accuracy tiers for regions, and do not distinguish reportable ranges by variant type (e.g., SNV vs. indel). Because only approximately 79-84% of the reference genome maps to the benchmark regions and there is no distinction of reportable ranges by variant call type, traditional reportable ranges leave a significant portion of the reference genome without an indication of detection accuracy and without an indication of whether a particular variant call type affects detection accuracy.

[0007] Even with these conventional reportable ranges, clinicians require specialized knowledge of how the features of a reference genome translate to a particular sequencing pipeline, for example to account for changes in nucleotide sample preparation (e.g., PCR or longer reads), different sequencing instruments, or different sequencing data analysis software. Indeed, despite the reportable ranges of the reference genome, existing sequencing systems cannot identify reportable ranges that are specific to a sequencing pipeline or derived from empirical data.

[0008] In addition to traditional reportable coverage from GIAB and GA4GH, Illumina, Inc. collaborated with research institutions to develop a catalog of high-confidence variant calls in a set of benchmark genomes. By generating whole-genome sequence data for people with three-generation pedigrees and calling variants in each genome, the team developed the Platinum Genome, which has a catalog of 4.7 million SNVs and 700,000 small indels (1-50 base pairs) that are consistent with inheritance patterns among these people. Although the variant calling truthset in the Platinum Genome can be used to validate and measure the performance of variant calling in curated samples, the Platinum Genome and other truthsets from GIAB exclude problematic genomic regions that contain both random and systematic errors. Additionally, the Platinum Genome or other truthsets cannot account for sample-specific errors in variant calling. Catalogs of high-confidence variant calls prove to be an impractical approach to determining the accuracy and confidence of variant calls at each genomic coordinate, because problematic regions are excluded regardless of the underlying cause of the problem, and such time-intensive cataloging is difficult (if not impossible) to scale. Summary of the Invention [Means for solving the problem]

[0009] The present disclosure describes embodiments of methods, non-transitory computer readable media, and systems that can train a genomic location classification model to classify or score a genomic coordinate or genomic region according to the degree to which a nucleic acid base can be accurately identified in such genomic coordinate or genomic region. For example, the disclosed system can determine one or both of the sequencing metrics for the various sample nucleic acid sequences and the context nucleic acid subsequences surrounding a particular nucleic acid base call. By utilizing the ground truth classification for the genomic coordinate, in some cases, the disclosed system trains a genomic location classification model to associate data from one or both of the sequencing metrics and the context nucleic acid subsequences with a confidence classification for such genomic coordinate or region. After training such a model, the disclosed system can similarly apply the genomic location classification model to the data from the sequencing metrics or the context nucleic acid subsequences to determine individual confidence classifications for individual genomic coordinates or regions. Such coordinate-specific or region-specific confidence classifications can be further packaged into new augmented files or new file types, i.e., digital files with confidence classifications for the genomic coordinates or regions (e.g., to supplement the variant calls).

[0010] Beyond training new types of machine learning models, the disclosed system can also apply models to supplement or contextualize variant calls with empirically trained confidence classifications. For example, after detecting a call variant at a genomic coordinate (or region) in a sample sequence, the disclosed system can identify a coordinate-specific or region-specific confidence classification from a digital file for the genomic coordinate or region corresponding to the variant call. Based on the identified coordinate-specific or region-specific confidence classification, the disclosed system can generate an indicator of the confidence classification for the genomic coordinate or region corresponding to the variant call for display on a graphical user interface. Thus, the disclosed system can facilitate a graphical or textual indicator on a computing device that specifies a confidence classification for a variant call at a genomic coordinate or region.

[0011] By training a genome location classification model as described herein, the disclosed system creates the first machine learning model of its kind to generate a reportable range of confidence classifications for genome coordinates or regions. Unlike existing solutions that rely on confidence ranges that are tied to a reference genome and are not bound to empirical data from a sequencing pipeline, the disclosed genome location classification model can be empirically trained and tailored to generate confidence classifications for a specific sequencing pipeline. Because the genome location classification model generates confidence classifications from an empirically trained process, the coordinate or region-specific confidence classifications from the genome location classification model provide context and newly discovered accuracy for variant calls or other nucleobase calls. [Brief description of the drawings]

[0012] The detailed description refers to the drawings, which are briefly described below. [Figure 1] FIG. 1 is a block diagram of a sequencing system including a genome classification system according to one or more embodiments. [Diagram 2]1 illustrates an overview of a genomic classification system that trains a machine learning model to determine confidence classifications of genomic coordinates, according to one or more embodiments. [Diagram 3] FIG. 1 illustrates an overview of a genome classification system for determining sequencing metrics relative to a reference genome, according to one or more embodiments. [Figure 4] FIG. 1 illustrates an overview of the process by which a genome classification system adjusts or prepares sequencing metrics for input into a genome location classification model, according to one or more embodiments. [Diagram 5] FIG. 1 shows a context nucleic acid subsequence surrounding a nucleobase call according to one or more embodiments. [Figure 6A] FIG. 1 illustrates a genome classification system that trains a machine learning model to determine a confidence classification for a genomic coordinate based on one or both of sequencing metrics and context nucleic acid subsequences, according to one or more embodiments. [Figure 6B] FIG. 1 illustrates a genome classification system that applies a trained version of a genome location classification model to determine a confidence classification for a genome coordinate based on one or both of sequencing metrics and context nucleic acid subsequences, according to one or more embodiments. [Figure 6C] FIG. 1 illustrates a sequencing or genomic classification system that identifies and displays confidence classifications from a genomic location classification model that corresponds to the genomic coordinates of a variant call, according to one or more embodiments. [Figure 6D] FIG. 1 shows a genomic classification system that determines a ground truth classification based on sequencing metrics for a sample nucleic acid sequence from a genomic sample and one or both of a recall or precision rate for calling a particular type of variant reflective of cancer or mosaicism based on a mixture of genomic samples, according to one or more embodiments. [Figure 6E]FIG. 1 shows a genomic classification system that determines a ground truth classification based on sequencing metrics for a sample nucleic acid sequence from a genomic sample and one or both of a recall or precision rate for calling a particular type of variant reflective of cancer or mosaicism based on a mixture of genomic samples, according to one or more embodiments. [Figure 6F] FIG. 1 shows a genomic classification system that determines a ground truth classification based on sequencing metrics for a sample nucleic acid sequence from a genomic sample and one or both of a recall or precision rate for calling a particular type of variant reflective of cancer or mosaicism based on a mixture of genomic samples, according to one or more embodiments. [Figure 6G] FIG. 1 shows a genomic classification system that determines a ground truth classification based on sequencing metrics for a sample nucleic acid sequence from a genomic sample and one or both of a recall or precision rate for calling a particular type of variant reflective of cancer or mosaicism based on a mixture of genomic samples, according to one or more embodiments. [Figure 6H] FIG. 1 shows a genomic classification system that determines a ground truth classification based on sequencing metrics for a sample nucleic acid sequence from a genomic sample and one or both of a recall or precision rate for calling a particular type of variant reflective of cancer or mosaicism based on a mixture of genomic samples, according to one or more embodiments. [Figure 7A] FIG. 1 depicts a graph showing useful sequencing metrics and sequencing metric derivation data for a genome location classification model, in accordance with one or more embodiments. [Figure 7B] FIG. 1 depicts a graph showing useful sequencing metrics and sequencing metric derivation data for a genome location classification model, in accordance with one or more embodiments. [Figure 7C] FIG. 1 depicts a graph showing useful sequencing metrics and sequencing metric derivation data for a genome location classification model, in accordance with one or more embodiments. [Figure 7D]FIG. 1 depicts a graph showing useful sequencing metrics and sequencing metric derivation data for a genome location classification model, in accordance with one or more embodiments. [Figure 7E] FIG. 1 depicts a graph showing useful sequencing metrics and sequencing metric derivation data for a genome location classification model, in accordance with one or more embodiments. [Figure 7F] FIG. 1 depicts a graph showing useful sequencing metrics and sequencing metric derivation data for a genome location classification model, in accordance with one or more embodiments. [Figure 7G] FIG. 1 depicts a graph showing useful sequencing metrics and sequencing metric derivation data for a genome location classification model, in accordance with one or more embodiments. [Figure 8] FIG. 1 depicts a graph illustrating the accuracy with which a genomic location classification model accurately determines a confidence classification for a genomic coordinate based on sequencing metrics, in accordance with one or more embodiments. [Figure 9] FIG. 1 depicts a graph illustrating the accuracy with which a genomic location classification model accurately determines confidence classifications for genomic coordinates corresponding to different nucleotide variants based on context nucleic acid subsequences, according to one or more embodiments. [Figure 10A] FIG. 1 depicts a graph illustrating the accuracy with which a genomic location classification model accurately determines confidence classifications for genomic coordinates corresponding to different nucleotide variants based on both sequencing metrics and context nucleic acid subsequences, in accordance with one or more embodiments. [Figure 10B] FIG. 1 depicts a graph illustrating the accuracy with which a genomic location classification model accurately determines confidence classifications for genomic coordinates corresponding to different nucleotide variants based on both sequencing metrics and context nucleic acid subsequences, in accordance with one or more embodiments. [Figure 11A] FIG. 1 illustrates a flowchart of a series of actions for training a machine learning model to determine confidence classifications for genomic coordinates, in accordance with one or more embodiments. [Figure 11B]FIG. 1 illustrates a flowchart of a series of actions for training a machine learning model to determine confidence classifications for genomic coordinates, in accordance with one or more embodiments. [Figure 12] FIG. 1 illustrates a flowchart of a series of actions for generating an indicator of a confidence classification for a genomic coordinate of a variant-nucleobase call from a digital file in accordance with one or more embodiments. [Figure 13] FIG. 1 depicts a block diagram of an exemplary computing device in accordance with one or more embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The present disclosure describes an embodiment of a genome classification system that trains a genome location classification model to determine a label or score for a genome coordinate (or genome region) that indicates the degree or range to which a nucleobase can be accurately identified in the genome coordinate or region. To prepare input for the genome location classification model, the genome classification system determines one or both of sequencing metrics for the sample nucleic acid sequence and the context nucleic acid subsequence surrounding the particular nucleobase call. In some cases, the genome classification system uses a particular sequencing and bioinformatics pipeline to determine such metrics and the context nucleic acid subsequence. Thus, based on data derived or created from one or both of the sequencing metrics and the context nucleic acid subsequence, and by leveraging ground truth classifications for the genome coordinate, the genome classification system trains a genome location classification model to determine a confidence classification for the genome coordinate.

[0014] In certain implementations, the genome classification system further determines a confidence classification for the genome coordinate (or region) by passing data from sequencing metrics or context nucleic acid subsequences corresponding to the sample through a genome location classification model. The genome classification system further encodes such coordinate-specific or region-specific confidence classifications into at least one digital file that includes the confidence classification for the particular genome coordinate or region. For example, the digital file may include an indicator of annotations or other data for the genome coordinate and / or region.

[0015] In addition to or independently of training the genome location classification model, the genome classification system can further determine a confidence classification for the nucleobase call (e.g., an invariant call or a variant call) based on a particular genomic coordinate or region of the call. For example, using data from a sequencing device, the genome classification system determines a variant-nucleobase call or a nucleobase call invariant at a particular genomic coordinate (or particular region) in the sample nucleic acid sequence. Such a nucleobase call can be determined using the same sequencing and bioinformatics pipeline used for the training data to train the genome location classification model. The genome classification system can then identify a confidence classification for the genomic coordinate or region corresponding to the nucleobase call (e.g., by accessing the confidence classification data in the digital file generated by the trained genome location classification model). By identifying the confidence classification, the genome classification system generates an indicator of the confidence classification for the genomic coordinate or region of the variant-nucleobase call or the nucleobase call invariant for display in a graphical user interface.

[0016] As mentioned in the previous paragraph, in some cases, the genome classification system uses a single sequencing pipeline to determine the nucleobase calls, context nucleic acid subsequences, or variant-nucleobase calls that underlie the sequencing metrics. For example, the genome classification system may use a single sequencing pipeline with the same nucleic acid sequence extraction method (e.g., extraction kit), the same sequencing device, and the same sequence analysis software. Such sequence analysis software may include alignment software that aligns sequence reads with a reference genome, and variant caller software that identifies variant-nucleobase calls, such that the single sequencing pipeline uses the same alignment software and / or variant caller. By using a single sequencing pipeline, in certain implementations, the genome classification system can both determine confidence classifications specific to the sequencing pipeline, and train and apply genome location classification models that increase the accuracy of those classifications for variant calls or other nucleobase calls by the pipeline.

[0017] To generate data to be input for training or applying a genome location classification model, in some embodiments, the genome classification system determines sequencing metrics including one or more of: (i) an alignment metric to quantify the alignment of a sample nucleic acid sequence with the genomic coordinates of an exemplary nucleic acid sequence (e.g., a nucleic acid sequence from a reference genome or an ancestral haplotype); (ii) a depth metric to quantify the depth of a nucleic acid base call for a sample nucleic acid sequence at the genomic coordinates of the exemplary nucleic acid sequence; or (iii) a call data quality metric to quantify the quality of a nucleic acid base call for a sample nucleic acid sequence at the genomic coordinates of the exemplary nucleic acid sequence. For example, the genome classification system determines a mapping quality metric, a soft clipping metric, or other alignment metric that measures the alignment of a sample sequence with a reference genome. As another example, the genome location classification system determines a forward-reverse depth metric (or other such depth metric) or a callability metric for variant-nucleobase calls (or other such call data quality metric).

[0018] In addition to or instead of using such sequencing metrics as data input for a genome location classification model, in some cases, the genome classification system determines a context nucleic acid subsequence surrounding a nucleobase call at a particular genome coordinate. For example, in some embodiments, the genome classification system identifies nucleobases from the reference genome (or from an ancestral haplotype sequence) located both upstream and downstream of any nucleobase call invariant or variant-nucleobase call, such as SNV, indel, structural variation, or copy number variation (CNV), as the context nucleic acid subsequence. To illustrate, the genome classification system may identify 50 nucleobases upstream and 50 nucleobases downstream from an SNV located at a particular genome coordinate in the reference genome or ancestral haplotype sequence as the context nucleic acid subsequence.

[0019] Regardless of whether the genome classification system uses data derived from sequencing metrics or contextual nucleic acid subsequences or both, the genome classification system creates the data as input for training a genome location classification model. In some cases, the genome classification system trains the genome location classification model by determining predicted confidence classifications for genome coordinates and comparing the predicted classifications to ground truth classifications that reflect Mendelian inheritance patterns or replicate matches of the nucleic acid base calls at the genome coordinates. By using a loss function to compare the projected confidence classifications to the ground truth classifications for a particular genome coordinate, the genome classification system can iteratively adjust parameters of the genome location classification model to more accurately determine the confidence classifications.

[0020] As alluded to above, the genome location classification model can output confidence classifications in a variety of forms, including labels or scores. The genome classification system may determine a tier of confidence levels, including, for example, a high confidence classification, a medium confidence classification, or a low confidence classification, that indicates the degree to which a nucleobase call may be relied upon at a given genome coordinate. Additionally or alternatively, the genome classification system may determine a confidence score from a range of scores that indicates the degree to which a nucleobase call may be relied upon at a given genome coordinate.

[0021] After training and determining the confidence classifications, the genome classification system can generate or annotate one or more digital files to include the confidence classifications specific to the genome coordinates. By way of example only, in some cases, the genome classification system generates a modified version of a Browser Extensible Data (BED) file that includes annotations for each nucleobase call at the genome coordinate that identifies the corresponding confidence classification for the genome coordinate. In some cases, the genome classification system generates a BED file that includes annotations for the genome coordinates according to the confidence classification type, for example, a BED file with annotations for genome coordinates with high confidence classifications, a BED file with annotations for genome coordinates with medium confidence classifications, and a BED file with annotations for genome coordinates with low confidence classifications. The genome classification system may similarly generate digital files with confidence classifications in Wiggle (WIG) format, Binary version of Sequence Alignment / Map (BAM) format, Variant Call File (VCF) format, Microarray format, or other digital file format. Upon identifying the associated confidence classifications of the nucleotide call variants from the digital file, the genome classification system may similarly provide an indicator of the classification for display on a graphical user interface. Such an indicator may be, for example, a graphical indicator of high confidence, medium confidence, or low confidence classification (eg, a color-coded graphical indicator).

[0022] As alluded to above, the genome classification system provides several technical benefits and improvements compared to conventional nucleic acid sequencing systems and corresponding sequencing data analysis software. For example, the genome classification system introduces a first-of-its-kind machine learning model that is uniquely trained to perform a new application that generates confidence classifications for specific genome coordinates where nucleotide variant calls or other nucleic acid bases are determined. Unlike conventional variant callers that rely primarily on reference genome characteristics or conventional reportable ranges, the genome classification system uses empirical data to train genome location classification models and generate coordinate-specific or region-specific confidence classifications, resulting in empirical reportable ranges of confidence classifications for nucleic acid base calls. Unlike the monolithic conventional classifications for reference genomes, the reportable ranges can include a variety of easy-to-understand labels, such as high confidence classifications, medium confidence classifications, or low confidence classifications. In further contrast to the one-size-fits-all approach of existing sequencing systems that rely on confidence regions developed for a reference genome, in some embodiments, the genome classification system adapts the confidence classification of a genome location classification model to a single sequencing pipeline, thereby increasing the accuracy of the confidence classification of nucleic acid base calls from a particular sequencing device (and corresponding pipeline component) at the individual genome coordinate level.

[0023] In addition to introducing a first-of-its-kind machine learning model, compared to existing sequencing systems, the genome classification system improves the accuracy and breadth of determining confidence levels for nucleobase calls at specific genomic coordinates across the genome. For example, the genome classification system increases the precision, recall, and concordance with which the sequencing system accurately identifies variants at genomic coordinates. In some implementations, the sequencing system accurately identifies SNVs at genomic coordinates labeled with high confidence classifications by the disclosed genome location classification model for about 90.3% of the reference genome with approximately 99.9% precision, 99.9% recall, and 99.9% concordance. The present disclosure reports the following additional statistics for precision, recall, and concordance. In contrast to the precision and breadth of the disclosed genome classification system, the conventional reportable range (with a single classification) of GIAB or GA4GH for a reference genome is limited to about 79-84% of the reference genome. Furthermore, the Platinum Genome excludes problematic genomic regions that genome classification can currently classify with exceptional precision, recall, and concordance.

[0024] In addition to improved accuracy, in certain embodiments, the genome classification system improves flexibility over conventional methods by reliably determining confidence classifications for different variant types at a particular genome coordinate. As described above, conventional reportable ranges developed by GIAB and GA4GH do not distinguish between variant types. In contrast, in some implementations, the genome classification system determines confidence classifications for genome coordinates that are specific to a variant type (e.g., variant-nucleobase calls that reflect SNV, indel, cancer, or mosaicism). For example, the genome location classification model may generate different confidence classifications for genome coordinates where a single nucleotide variant, a nucleobase insertion, a nucleobase deletion, a portion of a structural change, or a portion of a CNV is detected. Thus, the confidence classification from the genome location classification model may indicate a certain confidence that a single nucleotide variant can be accurately determined at a particular genome coordinate, in contrast to confidence classifications that may differ for a nucleobase insertion, a nucleobase deletion, a portion of a structural change, or a portion of a CNV.

[0025] Regardless of improved accuracy or flexibility, in some cases, the genome classification system generates new or newly extended file types that differ from traditional genome files and introduce specific confidence classifications for specific genome coordinates or regions. By way of background, traditional BED files often contain fields for the name of the chromosome (e.g., chrom=chr3, chrY), the start position of the chromosomal nucleobase or feature (e.g., chromStart=0 for the first base number), and the end position of the feature (e.g., chromEnd=100). In some cases, the BED file also contains fields for identifying specific genes and identifying the variants detected. Like WIG, BAM, VSF, or microarray files, traditional BED files do not have fields or annotations for confidence classifications for specific genome coordinates. In contrast, the genome classification system generates new digital files with annotations or other indicators of confidence classifications for specific genome coordinates or regions in BED, BAM, WIG, VCF, microarray, or other digital file formats. As discussed above, in some cases, the genome classification system generates different digital files each containing annotations for a genomic coordinate according to a different confidence classification type (e.g., a different digital file for each of high, medium, and low confidence classifications). By introducing new confidence classification indicators, the genome classification system can provide specific confidence classifications in the form of labels or scores for a variety of different variant-nucleobase calls at a particular genomic coordinate or region.

[0026] As illustrated by the above description, the present disclosure describes various features and advantages of the genome classification system. For example, the term "sample nucleic acid sequence" or "sample sequence" as used in the present disclosure refers to a sequence of nucleotides isolated or extracted from a sample organism (or a copy of such an isolated or extracted sequence). In particular, the sample nucleic acid sequence is isolated or extracted from a sample organism and includes a segment of a nucleic acid polymer composed of nitrogenous heterocyclic bases. For example, the sample nucleic acid sequence can include a segment of deoxyribonucleic acid (DNA), ribonucleic acid (RNA), or other polymeric forms of nucleic acid, or chimeric or hybrid forms of nucleic acids as described below. More specifically, in some cases, the sample nucleic acid sequence is prepared or isolated by the kit and found in a sample received by a sequencing device.

[0027] As further used herein, the term "nucleobase call" refers to the assignment or determination of a particular nucleobase added to an oligonucleotide for a sequencing cycle. In particular, nucleobase call refers to the assignment or determination of the type of nucleotide incorporated into an oligonucleotide on a nucleotide-sample slide. In some cases, nucleobase call includes the assignment or determination of a nucleobase to an intensity value resulting from a fluorescently tagged nucleotide added to an oligonucleotide on a nucleotide-sample slide (e.g., in a well of a flow cell). Alternatively, nucleobase call includes the assignment or determination of a nucleobase to a chromatogram peak or current change resulting from a nucleotide passing through a nanopore on a nucleotide-sample slide. By using nucleobase calls, the sequencing system determines the sequence of a nucleic acid polymer. For example, a single nucleobase call can include an adenine call, a cytosine call, a guanine call, or a thymine call (abbreviated as A, C, G, T) for DNA, or a uracil call (for thymine call) (abbreviated as U) for RNA.

[0028] As described above, in some embodiments, the genome classification system determines a sequencing metric for comparing the sample nucleic acid sequence to the exemplary nucleic acid sequence (e.g., a nucleic acid sequence from a reference genome or an ancestral haplotype). As used herein, the term "sequencing metric" refers to a quantitative measurement or score that indicates the degree to which an individual nucleic acid base call (or a sequence of nucleic acid base calls) aligns, compares, or quantifies to the genomic coordinates or genomic regions of the exemplary nucleic acid sequence. In particular, the sequencing metric may include an alignment metric that quantifies the degree to which the sample nucleic acid sequence aligns to the genomic coordinates of the exemplary nucleic acid sequence, such as a deletion size metric or a mapping quality metric. Furthermore, the sequencing metric may include a depth metric that quantifies the depth of the nucleic acid base call to the sample nucleic acid sequence at the genomic coordinates of the exemplary nucleic acid sequence, such as a forward-reverse depth metric or a normalized depth metric. The sequencing metric may also include a call data quality metric that quantifies the quality or accuracy of the nucleic acid base call, such as a nucleic acid base call quality metric, a callability metric, or a somatic cell quality metric. In some embodiments, data derived or prepared from the sequencing metrics can be input into a genome location classification model. The present disclosure further describes the sequencing metrics and provides additional examples below with reference to FIG.

[0029] As described above, in some embodiments, the genome classification system can determine a context nucleic acid subsequence surrounding a nucleobase call at a genomic coordinate. As used herein, the term "context nucleic acid subsequence" refers to a series of nucleobases from an exemplary nucleic acid sequence that surrounds (e.g., flanks or flanks) a genomic coordinate for a particular nucleobase call in a sample nucleic acid sequence. In some examples, the context nucleic acid subsequence refers to a series of nucleobases from a reference sequence (or a genome or ancestral haplotype sequence) that surrounds a nucleotide variant or invariant call in a sample nucleic acid sequence. In particular, the context nucleic acid subsequence includes nucleobases from an exemplary nucleic acid sequence that (i) is located both upstream and downstream from the genomic coordinate(s) for a particular nucleobase call(s) of the sample nucleic acid sequence, and (ii) is within a threshold number of genomic coordinates from the genomic coordinate(s) for the particular nucleobase call(s). Thus, the context nucleic acid subsequence can include nucleobases of the upstream 50 nucleobases and downstream 50 nucleobases from an SNV located at a particular genomic coordinate in an exemplary nucleic acid sequence (e.g., a reference genome).

[0030] As described above, the genome classification system can determine a context nucleic acid subsequence from an exemplary nucleic acid sequence. As used herein, the term "exemplary nucleic acid sequence" refers to a sequence of nucleotides from a reference or related genome, such as a reference genome or an ancestral haplotype sequence. In particular, the exemplary nucleic acid sequence includes a segment of a nucleic acid sequence inherited from a sample's ancestor (e.g., an ancestral haplotype) or a segment of a digital nucleic acid sequence (e.g., a reference genome). In some cases, the ancestral haplotype sequence is derived from a parent or grandparent of the sample.

[0031] As further used herein, the term "genomic coordinate" refers to a specific location or position of a nucleobase within a genome (e.g., the genome of an organism or a reference genome). In some cases, the genomic coordinate includes an identifier for a particular chromosome of the genome and an identifier for the position of the nucleobase within the particular chromosome. For example, the genomic coordinate(s) may include a number, name, or other identifier for the chromosome (e.g., chr1 or chrX), and a specific position(s), such as a numbered position following the identifier for the chromosome (e.g., chr1:1234570 or chr1:1234570-1234870). Furthermore, in certain implementations, the genomic coordinate refers to the source of the reference genome (e.g., mt for mitochondrial DNA reference genome, or SARS-CoV-2 for the reference genome of the SARS-CoV-2 virus), and the position of the nucleobase within the source for the reference genome (e.g., mt:16568 or SARS-CoV-2:29001). In contrast, in certain cases, genomic coordinates refer to the location of a nucleic acid base within a reference genome, without reference to a chromosome or source (e.g., 29727).

[0032] As mentioned above, a "genomic region" refers to a range of genomic coordinates. Similar to genomic coordinates, in certain embodiments, a genomic region can be identified by a chromosomal identifier and a specific location(s), such as a numbered location following the chromosomal identifier (e.g., chr1:1234570-1234870).

[0033] As described above, the genome coordinates include a location in a reference genome. Such a location may be within a particular reference genome. As used herein, the term "reference genome" refers to a digital nucleic acid sequence assembled as a representative of an organism's genes. Regardless of sequence length, in some cases, the reference genome represents an exemplary set of genes or a set of nucleic acid sequences in a digital nucleic acid sequence determined by scientists as representative of a particular species of organism. For example, the linear human reference genome may be GRCh38 or other versions of the reference genome from the Genome Reference Consortium. As a further example, the reference genome may include a reference graph genome, such as Illumina DRAGEN Graph Reference Genome hg19, that includes both a linear reference genome and paths that represent nucleic acid sequences from ancestral haplotypes.

[0034] As used herein, the term "genomic location classification model" refers to a machine learning model trained to generate a confidence classification for a genomic coordinate or genomic region. Thus, a genomic location classification model can include a statistical machine learning model or a neural network trained to generate such a confidence classification. In some cases, for example, a genomic location classification model takes the form of a logistic regression model, a random forest classifier, or a convolutional neural network (CNN). However, other machine learning models may be trained or used.

[0035] As described above, the genome location classification model may be a genome location classification neural network. A neural network includes a model of interconnected artificial neurons (e.g., organized into layers) that communicate and learn to approximate a complex function and generate an output (e.g., a generated digital image) based on multiple inputs provided to the neural network. In some cases, a neural network refers to an algorithm (or set of algorithms) that implements deep learning techniques to model high-level abstractions in data.

[0036] Regardless of form, the genome location classification model generates a confidence classification. As used herein, the term "confidence classification" refers to a label, score, or metric that indicates the confidence or degree of certainty that a nucleic acid base can be determined or detected at a genome coordinate or genome region. In particular, the confidence classification includes a label, score, or metric that classifies the degree to which a nucleic acid base can be correctly called for a particular genome coordinate or within a particular genome region. For example, in certain implementations, the confidence classification includes a label that identifies a high confidence classification, a medium confidence classification, or a low confidence classification for the genome coordinate. Additionally or alternatively, the confidence classification includes a score that indicates the probability or likelihood that a nucleic acid base can be correctly determined at a genome coordinate.

[0037] The following paragraphs describe the genome classification system with reference to exemplary diagrams depicting exemplary embodiments and implementations. For example, FIG. 1 illustrates a schematic diagram of a system environment (or "environment") 100 in which a genome classification system 106 operates according to one or more embodiments. As shown, the environment 100 includes one or more server devices 102 connected to user client devices 108 and sequencing devices 114 via a network 112. While FIG. 1 illustrates an embodiment of the genome classification system 106, the present disclosure describes the following alternative embodiments and configurations.

[0038] 1, server device 102, user client device 108, and sequencing device 114 are connected via network 112. Thus, each of the components of environment 100 can communicate via network 112. Network 112 includes any suitable network with which computing devices can communicate. An exemplary network is described in further detail below in conjunction with FIG. 13.

[0039] As illustrated by FIG. 1, the sequencing device 114 includes a device for sequencing nucleic acid polymers. In some embodiments, the sequencing device 114 analyzes nucleic acid segments or oligonucleotides extracted from a sample to generate data using computer-implemented methods and systems (described herein) either directly or indirectly on the sequencing device 114. More specifically, the sequencing device 114 receives and analyzes nucleic acid sequences extracted from a sample in a nucleotide-sample slide (e.g., a flow cell). In one or more embodiments, the sequencing device 114 utilizes SBS to sequence the nucleic acid polymers. In some embodiments, the sequencing device 114 communicates directly with the user client device 108, in addition to or as an alternative to communicating via the network 112, bypassing the network 112.

[0040] As further illustrated by FIG. 1, the server device(s) 102 can generate, receive, analyze, store, and transmit electronic data, such as data for determining a nucleobase call or for sequencing a nucleic acid polymer. As illustrated in FIG. 1, the sequencing device 114 can transmit (and the server device(s) 102 can receive) call data 116 from the sequencing device 114. The server device 102 can also communicate with the user client device 108. In particular, the server device(s) 102 can transmit to the user client device 108 a digital file 118 that includes a confidence classification for the genomic coordinates. As illustrated by FIG. 1, in some embodiments, the server device(s) 102 transmits separate digital files, each of which includes a different confidence classification (e.g., a different digital file for each of a high, medium, and low confidence classification). In some cases, the digital file 118 (and / or other digital files) also include the nucleobase call, error data, and other information.

[0041] In some implementations, the server device(s) 102 comprise a collection of distributed servers, where the server device(s) 102 include multiple server devices distributed across the network 112 and located at the same or different physical locations. Additionally, the server device 102 may include a content server, an application server, a communication server, a web hosting server, or another type of server.

[0042] As further shown in FIG. 1 , the server device 102 can include a sequencing system 104. Generally, the sequencing system 104 analyzes call data 116 received from the sequencing device 114 to determine the nucleobase sequence of the nucleic acid polymer. For example, the sequencing system 104 can receive raw data from the sequencing device 114 and determine the nucleobase sequence for the nucleic acid segment. In some embodiments, the sequencing system 104 determines the sequence of the nucleobases in the DNA and / or RNA segments or oligonucleotides. In addition to processing and determining the sequence of the nucleic acid polymer, the sequencing system 104 can also generate a digital file 118 that includes a confidence classification and transmit the digital file 118 to the user client device 108.

[0043] As just described and as shown in FIG. 1, the genome classification system 106 analyzes the call data 116 from the sequencing device 114 to determine a nucleobase call for a sample nucleic acid sequence. In some embodiments, the genome classification system 106 determines one or both of the sequencing metrics for such sample nucleic acid sequence and the context nucleic acid subsequences surrounding a particular nucleobase call. Based on the data derived or created from one or both of the sequencing metrics and the context nucleic acid subsequences, and the ground truth classification for the genomic coordinates, the genome classification system 106 trains a genome location classification model to determine a confidence classification for the genomic coordinates. The genome classification system 106 further determines a set of confidence classifications for a set of genomic coordinates (or regions) by providing as input to the genome location classification model: (i) a set of sequencing metrics corresponding to the sample, or (ii) data prepared from the context nucleic acid subsequences corresponding to the sample. Based on these inputs, for example, the genome classification system 106 uses the genome location classification model to determine a confidence classification for each genomic coordinate of the reference genome. As described above, the genome classification system 106 further generates a digital file that includes the confidence classifications for the set of genomic coordinates or regions.

[0044] As further illustrated and shown in FIG. 1 , the user client device 108 can generate, store, receive, and transmit digital data. In particular, the user client device 108 can receive call data 116 from the sequencing device 114. Additionally, the user client device 108 can be in communication with the server device(s) 102 to receive digital files 118 including the nucleic acid base calls and / or confidence classifications. Thus, the user client device 108 can present the confidence classifications of the genomic coordinates along with the nucleotide variant or invariant calls in a graphical user interface to a user associated with the user client device 108.

[0045] The user client devices 108 illustrated in FIG. 1 can include various types of client devices. For example, in some embodiments, the user client devices 108 include non-mobile devices, such as desktop computers or servers, or other types of client devices. In yet other embodiments, the user client devices 108 include mobile devices, such as laptops, tablets, cell phones, or smartphones. Further details regarding the user client devices 108 are described below with respect to FIG. 13.

[0046] As further illustrated in FIG. 1 , the user client device 108 includes a sequencing application 110. The sequencing application 110 may be a web application or a native application (e.g., a mobile application, a desktop application) stored and executed on the user client device 108. The sequencing application 110 can receive data from the genome classification system 106 and present data from the digital files 118 for display on the user client device 108 (e.g., by presenting a particular confidence classification by genomic coordinate). Additionally, the sequencing application 110 can instruct the user client device 108 to display an indicator of the confidence classification for the variant-nucleobase call or the nucleobase call-invariant genomic coordinate.

[0047] 1, the genome classification system 106 may be located on the user client device 108 as part of the sequencing application 110 or on the sequencing device 114. Thus, in some embodiments, the genome classification system 106 is implemented by (e.g., located in whole or in part on) the user client device 108. In yet other embodiments, the genome classification system 106 is performed by one or more other components of the environment 100, such as the sequencing device 114. In particular, the genome classification system 106 may be implemented in a variety of different ways across the server device(s) 102, the network 112, the user client device 108, and the sequencing device 114.

[0048] 1 illustrates components of environment 100 communicating over network 112, in certain implementations, components of environment 100 may also communicate directly with one another, bypassing the network. For example, as discussed above, in some implementations, user client device 108 communicates directly with sequencing device 114. Additionally, in some embodiments, user client device 108 communicates directly with genome classification system 106. Furthermore, genome classification system 106 may access one or more databases housed on or accessed by server device 102 or elsewhere in environment 100.

[0049] As described above, the genome classification system 106 trains a genome location classification model to determine confidence classifications for genome coordinates or genomic regions. FIG. 2 shows an overview of the genome classification system 106 that uses one or both of the sequencing metrics and the context nucleic acid subsequences to train the genome location classification model 208. As described further below, the genome classification system 106 determines one or both of the sequencing metrics 202 and the context nucleic acid subsequences 204 for the sample nucleic acid sequences. Based on data derived or prepared from one or more of the sequencing metrics 202 or the context nucleic acid subsequences 204, the genome classification system 106 trains the genome location classification model 208 to generate confidence classifications for the genome coordinates. After training and testing the genome location classification model 208, the genome classification system 106 generates a digital file 214 that includes confidence classifications for particular genome coordinates and can cause a computing device 220 to display such confidence classifications from the digital file 214.

[0050] As shown in FIG. 2, for example, the genome classification system 106 optionally determines a sequencing metric 202 for comparing a sample nucleic acid sequence to genomic coordinates of an exemplary nucleic acid sequence (e.g., a nucleic acid sequence from a reference genome or an ancestral haplotype). In preparation for determining the sequencing metric 202, in some cases, the sequencing system 104 or genome classification system 106 receives call data and determines nucleic acid base calls for nucleic acid sequences extracted from a diverse cohort of samples. In some cases, for example, the genome classification system 106 uses nucleic acid base calls and nucleic acid sequences determined from 30-150 samples across different populations. To extract and determine nucleic acid base calls for each sample nucleic acid sequence, in certain implementations, the genome classification system 106 uses a common or single sequencing pipeline that includes the same nucleic acid sequence extraction method, sequencing device, and sequence analysis software for each sample.

[0051] Based on the nucleobase calls in the sample nucleic acid sequence, the genome classification system 106 determines a sequencing metric 202. As described above, the sequencing metric 202 can include one or more of: (i) an alignment metric that quantifies the degree to which the sample nucleic acid sequence aligns with an exemplary nucleic acid sequence (e.g., a nucleic acid sequence of a reference genome or an ancestral haplotype); (ii) a depth metric that quantifies the depth of the nucleobase calls to the sample nucleic acid sequence at the genomic coordinates of the exemplary nucleic acid sequence; or (iii) a call data quality metric that quantifies the quality or accuracy of the nucleobase calls of the exemplary nucleic acid sequence. When determining an alignment metric, for example, the genome classification system 106 determines one or more of a deletion entropy metric, a deletion size metric, a mapping quality metric, a positive insertion size metric, a negative insertion size metric, a soft clipping metric, a read position metric, or a read reference mismatch metric for the sample nucleic acid sequence. When determining a depth metric, in contrast, the genome classification system 106 determines one or more of a forward-reverse depth metric, a normalized depth metric, a depth under metric, a depth over metric, or a peak count metric. For example, when determining a call data quality metric, the genome classification system 106 determines one or more of a nucleobase call quality metric, a callability metric, or a somatic cell quality metric for the sample nucleic acid sequence. Sequencing metrics 202 are further described below with respect to FIG. 3.

[0052] In addition to determining the sequencing metrics 202, as shown in FIG. 2, the genome classification system 106 further prepares data 206 from the sequencing metrics 202 for input to the genome location classification model 208. When preparing the data for input, the genome classification system 106 may extract data from the sequencing metrics 202 by summarizing or averaging the sequencing metrics 202 in various ways. In addition to extracting, in certain cases, the genome classification system 106 also modifies the sequencing metrics 202 or the data extracted from the sequencing metrics 202 to format the data for input to the genome location classification model 208. After extracting and modifying the sequencing metrics 202, or in addition thereto, in some embodiments, the genome classification system 106 further standardizes different types of sequencing metrics 202 to the same scale (e.g., mean 0 and standard deviation 1).

[0053] As further shown in FIG. 2, in addition to or instead of determining the sequencing metric 202, the genome classification system 106 determines a context nucleic acid subsequence 204 from an exemplary nucleic acid sequence (e.g., a reference genome or ancestral haplotype sequence) that surrounds the nucleobase call at a particular genomic coordinate. For each such context nucleic acid subsequence, in some cases, the genome classification system 106 determines both the upstream and downstream nucleobases in the reference genome that are within a threshold coordinate distance from the genomic coordinate for the particular nucleobase call or from the genomic coordinate for the particular nucleobase call. For example, the genome classification system 106 can determine the upstream and downstream nucleobases within 20, 50, 100, or a different number of nucleobases from the genomic coordinate of the SNV, indel, structural change, CNV, or other variant.

[0054] As described further below, the context nucleic acid subsequence 204 can include or exclude nucleic acid base call(s) for genomic coordinate(s) corresponding to a particular SNV, indel, structural variation, CNV, or other variant type of interest. Additionally, in certain implementations, the genome classification system 106 derives or prepares data from the context nucleic acid subsequence 204, for example, by applying a vector algorithm to package or compress the context nucleic acid subsequence 204 into a format for input to the genome location classification model 208.

[0055] After determining one or both of the sequencing metrics 202 and the data generated from the context nucleic acid subsequences 204, the genome classification system 106 trains a genome location classification model 208 based on such data. For example, the genome classification system 106 iteratively inputs one or both of the sequencing metrics 202 and the data generated from the context nucleic acid subsequences 204, along with indicators of the corresponding genomic coordinates or regions, to the genome location classification model 208. Based on the iterative inputs, the genome location classification model 208 generates a predicted confidence classification for each corresponding genomic coordinate or genomic region.

[0056] Upon generating the predicted confidence classification, the genome classification system 106 evaluates the performance 210 of the genome location classification model 208 using the predicted confidence classification in training iterations. For example, the genome classification system 106 compares the projected confidence classification to a ground truth classification from the ground truth classification 212 for the corresponding genome coordinate or genomic region. In each training iteration, for example, the genome classification system 106 executes a loss function to determine a loss between the predicted confidence classification for the genome coordinate and the ground truth classification for the genome coordinate. Based on the determined loss, the genome classification system 106 adjusts one or more parameters of the genome location classification model 208 to improve the accuracy with which the genome location classification model 208 generates the predicted confidence classification. By iteratively performing such training iterations, the genome classification system 106 trains the genome location classification model 208 to determine confidence classifications.

[0057] After training the genome location classification model 208, in some embodiments, the genome classification system 106 uses the trained version of the genome location classification model 208 to determine a set of confidence classifications for a set of genome coordinates (or regions) based on a set of sequencing metrics and / or a set of context nucleic acid subsequences. In some embodiments, the genome classification system 106 determines a set of sequencing metrics and / or a set of context nucleic acid subsequences from different samples. By determining a confidence classification for each genome coordinate or region, or for at least a subset of genome coordinates or regions that correspond to a reference genome, the genome classification system 106 generates coordinate-specific or region-specific classifications that indicate whether a nucleic acid base can be accurately detected at such genome coordinate or region. Because the nucleic acid base calls for which the sequencing metrics 202 or context nucleic acid subsequences 204 are determined use a single or defined sequencing pipeline, the genome classification system 106 can similarly determine confidence classifications for genome coordinates or regions based on sample nucleic acid sequences analyzed using the same defined sequencing pipeline.

[0058] As further shown in FIG. 2, the genome classification system 106 generates a digital file 214 that includes confidence classifications for genomic coordinates or regions. In some cases, the digital file 214 includes the confidence classifications as a reference file that a computing device can access to identify the confidence classification for a particular genomic coordinate or region. The digital file 214 (or set of digital files) can include a high, medium, or low confidence confidence classification or confidence score for each genomic coordinate. Additionally, in some cases, the genome classification system 106 makes the nucleobase calls in the digital file 214 for orthogonal validation using different sequencing methods since the nucleobase calls are located at genomic coordinates that correspond to lower confidence classifications (e.g., low confidence classifications or below a confidence score threshold).

[0059] As described further below, in some cases, the digital file 214 includes a nucleotide variant call for a particular genomic coordinate and a confidence classification for the particular genomic coordinate. In such cases, the digital file 214 provides a context for the confidence with which a clinician or patient may rely on the nucleobase call, including the nucleotide variant call. As further illustrated by FIG. 2, in some embodiments, the genome classification system 106 generates separate digital files, each of which includes a different confidence classification (e.g., a different digital file for each of a high, medium, and low confidence classification).

[0060] In addition to generating the digital file 214, as further shown in FIG. 2, in some embodiments, the genome classification system 106 further provides to the computing device 220 a confidence indicator 216 of a particular confidence classification for the genomic coordinate of the nucleobase call, such as a variant-nucleobase call or a nucleobase call invariant. As shown by FIG. 2, the genome classification system 106 can integrate the confidence classification not only with the digital file 214, but also with the data for reporting the variant call or invariant call on a graphical user interface 218 of the computing device 220. For example, as shown in FIG. 2, the sequencing system 104 or genome classification system 106 provides the confidence indicator 216 for display in the graphical user interface 218 along with the genomic coordinate of the variant call and the identifier of the particular gene. The sequencing system 104 or genome classification system 106 can similarly provide a confidence indicator for the invariant call for display on the graphical user interface along with the same or similar genomic coordinate and / or gene information.

[0061] As described above, the genome classification system 106 determines sequencing metrics for comparing the sample nucleic acid sequence to the genomic coordinates of the reference genome. According to one or more embodiments, FIG. 3 shows that the genome classification system 106 determines nucleobase calls for the sample nucleic acid sequence 302, aligns the sequence nucleobase calls with an exemplary nucleic acid sequence 304, and determines sequencing metrics for the sample nucleic acid sequence 306. As described below, the genome classification system 106 determines nucleobase calls, aligns the sample nucleic acid sequence, and determines sequencing metrics for specific genomic coordinates within the reference genome.

[0062] As shown in FIG. 3, for example, the genome classification system 106 determines nucleobase calls for a sample nucleic acid sequence 302. In preparation for such nucleobase calls, in some embodiments, nucleic acid sequences are extracted or isolated from samples of diverse ethnicities using extraction kits or specific nucleic acid sequence extraction methods. After extraction, the sequencing device 114 synthesizes copies and reverse strands of the sample nucleic acid sequence using SBS sequencing or Sanger sequencing to generate call data indicative of the individual nucleobases incorporated into the growing nucleic acid sequence. Based on the call data, the sequencing system 104 determines nucleobase calls within the nucleic acid sequence.

[0063] In some embodiments, a single or defined pipeline processes and determines the nucleobases of such nucleic acid sequences for each sample. For example, the sequencing system 104 may use a single sequencing pipeline that includes the same nucleic acid sequence extraction method (e.g., extraction kit), the same sequencer, and the same sequence analysis software. In particular, the single pipeline may include, for example, extracting DNA segments using Illumina Inc.'s TruSeq PCR-Free sample preparation kit for nucleic acid sequence extraction method, sequencing using NovaSeq 6000 Xp, NextSeq 550, NextSeq 1000, or NextSeq 2000 for sequencer, and determining nucleobase calls using Dragen Germline Pipeline for sequence analysis software.

[0064] After determining the nucleobase calls for the sample nucleic acid sequences, the genome classification system 106 aligns the sequence nucleobase calls with an exemplary nucleic acid sequence 304, as further shown in FIG. 3. For example, the sequencing system 104 or genome classification system 106 approximately matches the nucleobases of a particular nucleic acid sequence (across various reads) with the nucleobases of a reference genome (e.g., a linear reference genome or a graph reference genome). As shown by FIG. 3, the genome classification system 106 repeats the alignment process for the nucleic acid sequences from each sample. In addition to or instead of aligning the nucleobase calls with the reference genome as described above, in some cases, the nucleobase calls (e.g., from the nucleotide reads) are aligned with one or more nucleic acid sequences from an ancestral haplotype. Once approximately aligned, the genome classification system 106 can identify the nucleobase calls at specific genomic coordinates of the reference genome for each sample.

[0065] As suggested by Figure 3, in some implementations, the sequencing system 104 or genome classification system 106 aligns sequence nucleobase calls to the exemplary nucleic acid sequences 304 and aggregates read data and sample data for such nucleobase calls as part of generating one or both of a BAM file and a VCF file. To do so, the sequencing system 104 or genome classification system 106 generates, for each sample, a BAM file that contains data for the aligned sample nucleic acid sequences and a VCF file that contains data for the nucleic acid variant calls at genomic coordinates of the reference genome.

[0066] As further shown in FIG. 3, after determining the nucleic acid base calls and aligning the sample nucleic acid sequences, the genome classification system 106 determines sequencing metrics for the sample nucleic acid sequences 306. In some embodiments, the genome classification system 106 determines sequencing metrics for the sample nucleic acid sequences at each genomic coordinate (or each genomic region). As indicated above, the genome classification system 106 optionally determines sequencing metrics from the BAM and VCF files for various samples. As described below, the genome classification system 106 determines one or more sequencing metrics that quantify depth, alignment, or call data quality at the genomic coordinates. The following paragraphs describe exemplary sequencing metrics, loosely grouped according to alignment, depth, and call data quality.

[0067] As indicated immediately above, the genome classification system 106 can determine an alignment metric that quantifies the alignment of the nucleobase calls to the sample nucleic acid sequence with the genomic coordinates of an exemplary nucleic acid sequence (e.g., the nucleic acid sequence of the reference genome or the ancestral haplotype). To illustrate, in some cases, the genome classification system 106 determines the mapping quality metric for the sample nucleic acid sequence, for example, by determining the mean or median mapping quality of the reads at the genomic coordinate. In some such embodiments, the genome classification system 106 identifies or generates a mapping quality (MAPQ) score for the nucleobase calls at the genomic coordinate, where the MAPQ score represents -10 log10 Pr{mapped position incorrect} rounded to the nearest integer. As an alternative to the mean or median mapping quality, in some embodiments, the genome classification system 106 determines the mapping quality metric for the sample nucleic acid sequence by determining the entire distribution of mapping quality for all reads that align with the genomic coordinate or the ancestral haplotype. In addition to or instead of the mapping quality metric, the genome classification system 106 can determine a soft clip metric for the sample nucleic acid sequence, for example, by determining the total number of soft clipped nucleic acid bases that span a genomic coordinate corresponding to a reference genome or an ancestral haplotype. Thus, in some cases, the genome classification system 106 determines the number of nucleic acid bases at a particular genomic coordinate on either side of the read (e.g., the 5 prime end or 3 prime end of the read) that do not match an exemplary nucleic acid sequence (e.g., a reference genome or an ancestral haplotype) and are ignored for alignment purposes.

[0068] As a further example of an alignment metric, in some embodiments, the genome classification system 106 determines a read-reference-mismatch metric for a sample nucleic acid sequence, for example, by determining the total number of nucleobases that do not match nucleobases of an exemplary nucleic acid sequence (e.g., a reference genome or ancestral haplotype) at a particular genomic coordinate, across multiple reads (e.g., all reads overlapping a particular genomic coordinate) or across multiple cycles (e.g., all cycles). In contrast, in certain cases, the genome classification system 106 determines a read position metric for a sample nucleic acid sequence, for example, by determining the average or median position within the sequencing reads of the nucleobases that cover the genomic coordinate.

[0069] In addition to the alignment metrics described above, the genome classification system 106 can determine alignment by determining an indel metric that quantifies indels in genomic coordinates of the sample nucleic acid sequence, such as a deletion metric. In some cases, the genome classification system 106 determines a deletion size metric for the sample nucleic acid sequence, for example, by determining the average or median size of deletions across the genomic coordinates of the reference genome. Furthermore, in certain implementations, the genome classification system 106 determines a deletion entropy metric for the sample nucleic acid sequence, for example, by determining a distribution or variance of deletion sizes for genomic coordinates or genomic regions of the reference genome. A genomic coordinate or region with consistent or repeated deletions in the sample nucleic acid sequence of a single nucleobase (e.g., 20% of the samples contain a single nucleobase deletion) has less deletion entropy than a different genomic coordinate or region with a variety of deletion sizes in the sample nucleic acid sequence (e.g., 20% of the samples contain either a single nucleobase deletion, a 5 nucleobase deletion, or a 10 nucleobase deletion).

[0070] In addition to the deletion metric as an example of the alignment metric above, the genome classification system 106 can determine an insertion size metric that quantifies an insertion in a genomic coordinate of a sample nucleic acid sequence. For example, in certain implementations, the genome classification system 106 determines a positive insertion size metric for the sample nucleic acid sequence by determining the average or median positive insertion size of the reads that cover the genomic coordinate. Such a positive insertion may include a region of DNA or RNA fragment that is not covered by either of the two sequencing reads. In contrast to the positive insertion size metric, in some cases, the genome classification system 106 determines a negative insertion size metric for the sample nucleic acid sequence. For example, the genome classification system 106 determines the average or median negative insertion size of the sequencing reads that cover the genomic coordinate as a negative insertion size metric. Such a negative insertion may include an overlap between two sequencing reads.

[0071] In addition to or in lieu of the alignment metric, the genome classification system 106 can determine a depth metric that quantifies the depth of the nucleobase calls in the genomic coordinates of the sample nucleic acid sequence. The depth metric can, for example, quantify the number of nucleobase calls determined and aligned in the genomic coordinates. In certain implementations, the genome classification system 106 determines a forward-reverse depth metric for the sample nucleic acid sequence by determining the depth of both the forward and reverse strands in the genomic coordinates. Additionally or alternatively, the genome classification system 106 determines a normalized depth metric for the sample nucleic acid sequence by, for example, determining the depth on a normalized scale in the genomic coordinates. In some such cases, the genome classification system 106 uses a scale where a normalized depth of 1 refers to diploid and a normalized depth of 0.5 refers to haploid.

[0072] In addition to the forward-reverse depth metric or normalized depth metric, in some cases, the genome classification system 106 determines a depth under metric or depth over metric for the sample nucleic acid sequence. For example, the genome classification system 106 can determine a depth under metric by quantifying the number of nucleobase calls that are below the expected or threshold depth coverage in a genome coordinate or genomic region. In some cases, the genome classification system 106 multiplies the average depth coverage in a genome coordinate by -1, adds 1, and sets a minimum value of 0. For example, if a genome coordinate has an average depth coverage of 0.75, the genome classification system 106 determines a depth under metric of 0.25 for the genome coordinate. In contrast, the genome classification system 106 can determine a depth over metric by quantifying the number of nucleobase calls that are above the expected or threshold depth coverage in a genome coordinate or genomic region.

[0073] As described above, in some implementations, the genome classification system 106 determines the peak count metric, for example, by determining a distribution of depths for genomic coordinates or regions across genomic samples (e.g., a diverse cohort of genomic samples) and identifying local maxima for depth coverage from the distribution. In particular implementations, the genome classification system 106 smoothes the depth metric for a genomic region into a distribution of depth ranges using a Gaussian kernel and applies the find-peaks function from the signal processing subpackage at SciPy.org to the distribution to identify local maxima in the depth ranges.

[0074] Independently of the depth metric, the genome classification system 106 can determine a call data quality metric that quantifies the nucleobase call quality for the sample nucleic acid sequence at the genomic coordinate. In certain embodiments, for example, the genome classification system 106 determines the nucleobase call quality metric by determining a percentage or subset of nucleobase calls that meet a threshold quality score (e.g., Q20) at the genomic coordinate of an exemplary nucleic acid sequence (e.g., a nucleic acid sequence of a reference genome or an ancestral haplotype). To illustrate, the quality score (or Q score) can indicate that the probability of an incorrect nucleobase call at the genomic coordinate is equal to 1 in 100 for a Q20 score, 1 in 1,000 for a Q30 score, 1 in 10,000 for a Q40 score, etc.

[0075] In addition to or instead of the nucleobase call quality metric, in some embodiments, the genome classification system 106 determines a callability metric for the sample nucleic acid sequence, for example, by determining a score indicative of a correct nucleotide variant call or nucleobase call at a genomic coordinate. In some cases, the callability metric represents the proportion or percentage of non-N reference positions that have a passing genotype call, as implemented by Illumina, Inc. Additionally, in some implementations, the genome classification system 106 determines the callability metric using a version of the Genome Analysis Toolkit (GATK).

[0076] Beyond the nucleobase call quality metric or callability metric, in some embodiments, the genome classification system 106 determines a somatic quality metric for the sample nucleic acid sequence, for example, by determining a score that estimates the probability of determining the number of aberrant reads in the tumor sample. For example, the somatic quality metric can represent an estimate of the probability of determining a given (or more extreme) number of aberrant reads in the tumor sample using the Fisher Exact Test, given the counts of aberrant and normal reads in the tumor and normal BAM files. In some cases, the genome classification system 106 determines the somatic quality metric using a Phred algorithm and expresses the somatic quality metric as a Phred-scaled score, such as a quality score (or Q-score) that ranges from 0 to 60. Such a quality score may be equal to -10 log10 (probability variant is somatic).

[0077] As alluded to above, after determining the sequencing metrics, the genome classification system 106 can generate data from the sequencing metrics for input to the genome location classification model. According to one or more embodiments, FIG. 4 shows the genome classification system 106 generating data 404 from the sequencing metrics by (i) extracting data from the sequencing metrics 406, (ii) transforming the sequencing metrics or metric extraction 408, and (iii) redesigning or reorganizing the sequencing metrics or metric extraction 410. As shown by the Uniform Manifold Approximation and Projection (UMAP) graphs 402a and 402b and further described below, the data generation effectively curates data for the genome location classification model as measured by the Platinum Base and Nonplatinum Base from the regions cataloged by the Platinum Genome. As used herein, the term "Platinum Base" or "truthset base" refers to a nucleic acid base from a defined confidence region of the Platinum Genome developed by Illumina, Inc. In particular, a Platinum Base (or truthset base) refers to a nucleic acid base from a genomic coordinate that has a defined Mendelian inheritance pattern and / or consistent homozygous inheritance.

[0078] As shown in FIG. 4, for example, the genome classification system 106 extracts data from the sequencing metrics 406 to prepare data for input to the genome location classification model. By extracting data or features from the sequencing metrics, the genome classification system 106 can summarize information from the sequencing metrics that the genome location classification model may not otherwise be able to identify or learn. For example, in some embodiments, the genome classification system 106 extracts data from the sequencing metrics by determining one or more of: (i) a rolling average of a particular sequencing metric to provide a local summary of the sequencing metrics with respect to the genome coordinates; (ii) a masked rolling average of a particular sequencing metric to provide a local summary of the sequencing metrics without the genome coordinates; or (iii) a statistical measurement from a statistical test that evaluates a particular hypothesis regarding a given sequencing metric.

[0079] As just mentioned, the genome classification system 106 can perform various statistical tests to extract data from specific sequencing metrics for input into the genome location classification model. In some cases, for example, the genome classification system 106 performs a Kolmogorov-Smirnov (KS) test on the depth metric (e.g., forward-reverse depth metric, normalized depth metric) to determine whether the depth is normally distributed across a population of samples. In some cases, the KS test quantifies the distance between the depths of sample nucleic acid sequences from each sample according to an empirical distribution function. As a further example of a statistical test, in certain embodiments, the genome classification system 106 performs a binomial test on the depth metric (e.g., forward-reverse depth metric) to determine whether the depths are equally distributed on the forward and reverse strands. In certain situations, the binomial test determines the statistical significance of deviations from the expected distribution of depths into categories for the forward and reverse strands.

[0080] In addition to (or as an alternative to) the KS test or binomial test as a statistical test, the genome classification system 106 performs a binomial proportion test on the call data quality metric (e.g., nucleobase call quality metric) and / or other sequencing metrics to determine whether the reads on the forward and reverse strands have the same percentage of quality scores that meet a quality score threshold (e.g., Q20 score). In some cases, the binomial test determines a binomial distribution of the probability that the reads on the forward and reverse strands have at least the same percentage of Q20 scores. In contrast, in certain implementations, the genome classification system 106 performs a Bates distribution test to determine whether the average start position for the genome coordinates from the reference genome is in the middle of the reads for the sample nucleic acid sequence. For example, the Bates distribution test can determine the probability distribution of the average number of average start positions is in the middle of the reads.

[0081] In addition to extracting data from the sequencing metrics, as further shown in FIG. 4, the genome classification system 106 transforms the sequencing metrics or metric extraction 408 to prepare the data for input to the genome location classification model. By converting the sequencing metrics (or data extracted from the sequencing metrics) to a new format or scale, the genome classification system 106 can rescale certain sequencing metrics to avoid overtraining or unnecessarily training the genome location classification model. For example, in some embodiments, the genome classification system 106 transforms the sequencing metrics (or data extracted from the sequencing metrics) by one or more of: (i) normalizing the sequencing metrics, including counts or totals, by dividing such counts or totals by the coverage; (ii) standardizing all or a portion of the sequencing metrics and / or data extracted from the sequencing metrics to be on the same scale; (iii) determining an average or local average for the sequencing metrics; or (iv) determining, for the sequencing metrics, a portion or fraction of reads on the forward strand versus the reverse strand of the original oligonucleotide from the genome sample. In contrast, the genome classification system 106 optionally does not convert certain sequencing metrics, such as by not converting mapping quality metrics, read position metrics, deletion size metrics, depth metrics, depth under metrics, depth over metrics, positive insertion size metrics, negative insertion size metrics, and nucleic acid base call quality metrics.

[0082] To illustrate a particular transformation, in some embodiments, the genome classification system 106 coverage normalizes the soft clip metric by converting the total number of soft clipped nucleobases that span the genomic coordinate into a percentage based on the total number of reads from the sample. As a further transformation example, in some cases, the genome classification system 106 standardizes the depth metric to be a value within a standard deviation, such as a mean of 0 and a standard deviation of 1. Additionally, the genome classification system 106 may determine a local average of the read-reference-mismatch metric by determining the average number of nucleobases that do not match the nucleobases of the reference genome in the genomic coordinate or genomic region. As another transformation example, in some implementations, the genome classification system 106 determines a portion or percentage of reads on the forward strand vs. the reverse strand of the original oligonucleotide from the genomic sample for the nucleobase call quality metric or depth metric. By determining the percentage of forward strand vs. reverse strand for the sequencing metric, the genome classification system 106 can generate a forward percentage metric, such as a forward percentage-nucleobase call quality metric or a forward percentage-depth metric.

[0083] After extracting data from and converting the sequencing metrics, in some embodiments, the genome classification system 106 redesigns or reorganizes the sequencing metrics or metric extractions 410 to prepare the data for input into the genome location classification model. By redesigning or reorganizing a particular sequencing metric or metric extraction, the genome classification system 106 can package the particular sequencing metric or metric extraction into a format that the genome location classification model can process. For example, the genome classification system 106 may redesign or reorganize a sequencing metric or metric extraction by (i) applying a linear scaling function to scale a particular sequencing metric or metric extraction, (ii) clipping probability values ​​(p-values) from a particular sequencing metric, (iii) determining the absolute value of a particular sequencing metric or metric extraction, (iv) discretizing a sequencing metric to change such metric from continuous values ​​to categories of values, (v) replacing a particular sequencing metric or metric extraction with another value (e.g., to avoid zero values), or (vi) smooth clipping a particular sequencing metric to minimize the impact of outliers by log-transforming values ​​outside a defined range. In contrast, the genome classification system 106 optionally does not redesign or reorganize a particular sequencing metric, such as a mapping quality metric, a soft clipping metric, a nucleobase call quality metric, a deletion entropy metric, a depth metric, a read reference mismatch metric, and a peak count metric.

[0084] To illustrate a particular redesign or rearrangement of the sequencing metrics, in some embodiments, the genome classification system 106 performs a step of, for example, redesigning or rearranging the sequencing metrics by: *A linear scaling function is applied to scale a particular sequencing metric or metric extraction by scaling the value using a linear function of "x" + b, where "x" represents the original value of the sequencing metric or metric extraction, "y" represents the scaled value of the sequencing metric or metric extraction, and "a" and "b" represent different variables for scaling. In certain cases, the genome classification system 106 applies a linear scaling function to the values ​​of the read position metric, the under-depth metric, the over-depth metric, and the forward fraction metric. As a further example of redesigning or reorganizing the sequencing metrics, in some cases, the genome classification system 106 replaces 0.0 values ​​with 0.5 values ​​for the read position metric and the forward fraction metric, and / or replaces 0.0 values ​​with 1.0e-100 for the binomial proportional test for the nucleobase call quality metric. Additionally, the genome classification system 106 may determine absolute values ​​for the read position metric and the forward fraction metric.

[0085] In some embodiments, in addition to (or instead of) replacing values ​​or determining absolute values ​​to redesign or realign certain sequencing metrics, the genome classification system 106 logarithmically smooths the deletion size metric, depth metric, and depth over metric of the logarithmic smooth clip to effectively create a deletion size clip metric, a depth clip metric, and a depth over clip metric. For example, the genome classification system 106 logarithmically smooths and clips the deletion size metric, the normalized depth metric, and the depth over metric that exceed a value of 5, while not modifying other values ​​of these sequencing metrics. For example, for a value of 1.5, the genome classification system 106 does not modify the value and maintains the original value of the corresponding sequencing metric that is input into the genome location classification model. However, for a value of 9, the genome classification system 106 converts the value of 9 using the logarithmic formula of 5+log(9-5+1) to output and use a value of ∼5.7.

[0086] Beyond or instead of smooth clipping, in some cases, the genome classification system 106 clips p-values ​​from a KS test on a depth metric, a binomial test on a depth metric, a binomial proportional test on a call data quality metric, or a Bates distribution test on a read location metric. For each value in such statistical tests, for example, the genome classification system 106 log-smooths Phred-scaled p-values ​​above 5.0 to avoid overtraining the genome location classification model. For example, the genome classification system 106 log-smooths a Phred-scaled p-value of 40 to approximately 6.5.

[0087] To further illustrate certain redesigns or rearrangements of sequencing metrics, in some embodiments, the genome classification system 106 discretizes continuous values ​​from the positive and negative insertion size metrics into value categories. For example, the genome classification system 106 discretizes various sizes of positive or negative insertions into three categories: insertions less than 200 nucleobases in a first category, insertions between 200 and 800 nucleobases in a second category, and insertions greater than 800 nucleobases in a third category.

[0088] As described further below, in some embodiments, the genome classification system 106 inputs the extracted, transformed, and rescaled data from the sequencing metrics into a genome location classification model for training or application. For example, the genome classification system 106 aggregates the rescaled data from the sequencing metrics for each genome coordinate and iteratively inputs the rescaled sequencing metric data along with the genome coordinate identifier into the genome location classification model.

[0089] By preparing data from sequencing metrics as described above, the genome classification system 106 effectively transforms the sequencing metrics (or derivations from the sequencing metrics) to indicate a relatively higher or lower confidence of the genome coordinates in the genome location classification model. To orthogonally test the effectiveness of such data preparation, the researchers performed the UMAP algorithm to visualize the nucleic acid bases at specific genome coordinates according to the sequencing metrics before data preparation in UMAP graph 402a, and (ii) visualize the nucleic acid bases at specific genome coordinates according to the sequencing metrics after data preparation in UMAP graph 402b, as shown in FIG. 4. As UMAP graphs 402a and 402b show, the data preparation effectively separates the nucleic acid base calls from genome regions that have variant calls verified according to the Platinum Genome (here, in the Platinum Base) from the nucleic acid base calls from genome regions that do not have variant calls verified according to the Platinum Genome (here, in the Nonplatinum Base). It should be noted that UMAP graphs 402a and 402b do not represent components of the genome location classification model or components of the data generation, but merely visualize an orthogonal validation of the data generation.

[0090] In addition to or instead of determining the sequencing metric, in some embodiments, the genome classification system 106 determines a context nucleic acid subsequence from an example nucleic acid sequence (e.g., a reference genome, an ancestral haplotype) surrounding the nucleic acid base call as an input for the genome location classification model. According to one or more embodiments, FIG. 5 shows an example of a genome classification system 106 determining a context nucleic acid subsequence 504 corresponding to a nucleic acid base call 502 as such an input.

[0091] As shown in FIG. 5, the genome classification system 106 identifies a nucleobase call 502 for a particular genomic coordinate. In some cases, the genome classification system 106 identifies a nucleotide call variant or invariant from the VCF file at the genomic coordinate. Based on the genomic coordinate, the genome classification system 106 further identifies a series of nucleobases from the reference genome that are located both upstream and downstream from the genomic coordinate of the nucleobase call 502 and within a threshold number of genomic coordinates from the genomic coordinate of the nucleobase call 502. As shown in FIG. 5, the genome classification system 106 identifies this series of upstream and downstream nucleobases from the exemplary nucleic acid sequence as a context nucleic acid subsequence 504 for the nucleobase call 502. After identification, in some embodiments, the genome classification system 106 further prepares the context nucleic acid subsequence 504 by applying a vector algorithm (e.g., Nucl2Vec, one-hot vector) to encode the context nucleic acid subsequence 504 into a vector for input into a genome location classification model.

[0092] When identifying a context nucleic acid subsequence from an exemplary nucleic acid sequence, the genome classification system 106 can use various threshold numbers of genomic coordinates. For example, the context nucleic acid subsequence can include nucleic acid bases of the reference genome within 10, 50, 100, 400, or any other number of genomic coordinates from the genomic coordinate of a particular nucleic acid base call. As described further below, in some cases, the genome classification system 106 increases the accuracy with which the genome location classification model determines a confidence classification for the genomic coordinate as the threshold number of genomic coordinates for a nucleic acid base increases for the context nucleic acid subsequence.

[0093] In addition to varying the threshold number of genomic coordinates, in some embodiments, the genome classification system 106 uses a variety of different variant call types as the nucleobase call for which the threshold number of genomic coordinates are determined. As shown by Figure 5, for example, the genome classification system 106 identifies SNVs for the nucleobase call 502. However, in some embodiments, the genome classification system 106 identifies the genomic coordinate(s) of an indel, structural variation, or CNV as a reference point for determining the nucleobases within the threshold number of genomic coordinates that constitute the context nucleic acid subsequence.

[0094] In some cases, the genome classification system 106 uses variant calls from a VCF file to identify nucleotide variant calls as a basis for determining a context nucleic acid subsequence. By way of example only, the genome classification system 106 can identify variant calls from the match data of a VCF file for NA12878 (or other samples) from the HapMap project. In one such case, the genome classification system 106 determines variant calls from 96 replicates of NA12878 as a basis for determining a context nucleic acid subsequence for input into the genome location classification model and training.

[0095] After determining the sequencing metrics and the context nucleic acid subsequences and preparing the data for input, the genome classification system 106 trains and applies a genome location classification model. According to one or more embodiments, Figures 6A-6C show the genome classification system 106 training and applying a genome location classification model 608 to determine confidence classifications for genomic coordinates (or regions) and then providing confidence indicators for the confidence classifications corresponding to the nucleic acid base calls for display on a computing device. As shown in Figure 6A, the genome classification system 106 performs multiple training iterations in which the genome classification system 106 (i) determines predicted confidence classifications based on one or both of the sequencing metrics and the context nucleic acid subsequences and (ii) compares such predicted confidence classifications to ground truth classifications. After training, as shown in Figure 6B, the genome classification system 106 applies a trained version of the genome location classification model 608 to determine a set of confidence classifications for a set of genomic coordinates (or regions) and generates a digital file including the set of confidence classifications. Based on the generated digital files, the genome classification system 106 provides confidence classifications for the genomic coordinates (or regions) of the nucleobase calls for display on a graphical user interface, as shown in FIG. 6C.

[0096] Briefly, the present disclosure describes an initial training iteration followed by an overview of subsequent training iterations shown in Figure 6 A. In the initial training iteration shown by Figure 6A, for example, the genome classification system 106 inputs data derived or prepared from one or both of the sequencing metrics 602 and the context nucleic acid subsequences 606 corresponding to the genome coordinate identifiers 604 for particular genome coordinates into a genome location classification model 608.

[0097] As suggested and depicted in FIG. 6A , in some embodiments, the genome classification system 106 inputs data generated from sequencing metrics 602 specific to a genome coordinate into the genome coordinate identifier 604 without a corresponding context nucleic acid subsequence for the genome coordinate. In some such embodiments, the input includes data from one or more of a KS test, a binomial test, a binomial proportion test, or a Bates distribution test. In contrast, in certain implementations, the genome classification system 106 inputs a context nucleic acid subsequence 606 specific to a genome coordinate for the genome coordinate identifier 604 without a corresponding sequencing metric. Alternatively, the genome classification system 106 inputs data derived or prepared from both the sequencing metrics 602 and the context nucleic acid subsequence 606.

[0098] As alluded to above, the genome classification system 106 inputs such data into the genome location classification model 608 in various formats. For example, in some embodiments, the genome classification system 106 aggregates the rescaled data from the sequencing metrics 602 for the genome coordinates into a vector or matrix that includes each rescaled sequencing metric for the genome coordinate identifier 604. In some cases, the genome classification system 106 aggregates the rescaled data from the sequencing metrics 602 for the genome coordinates corresponding to the genome coordinate identifier 604 into an input vector or matrix along with the context nucleic acid subsequence 606. In contrast, in certain implementations, the genome classification system 106 aggregates the rescaled data from the sequencing metrics 602 for the genome coordinates corresponding to the genome coordinate identifier 604 and the rescaled sequencing metrics for each genome coordinate for the nucleic acid bases in the context nucleic acid subsequence 606 into an input vector or matrix along with the context nucleic acid subsequence 606.

[0099] To illustrate, in some embodiments, the genome classification system 106 inputs the data derived or generated from the sequencing metrics 602 into the genome location classification model 608 as a set of numeric arrays. For example, the genome classification system 106 stores the data derived or generated from the sequencing metrics 602 in a Hierarchical Data Format 5 (HDF5) file and inputs the data into the genome location classification model 608 as a set of numeric arrays (e.g., one-dimensional Python NumPy arrays).

[0100] To further explain, in certain implementations, the genome classification system 106 inputs (to the genome location classification model 608) data derived or prepared from both the sequencing metrics 602 and the context nucleic acid subsequence 606 as a matrix having a first dimension for the size or length of the context nucleic acid subsequence 606 and a second dimension for the number of individual sequencing metrics and / or derivatives from the individual sequencing metrics. For example, the first dimension of the size or length of the context nucleic acid subsequence 606 may include the number of nucleobases in the context nucleic acid subsequence 606 + 1 (e.g., 51 dimensions for 25 bases on each side of a nucleobase call, 101 dimensions for 50 bases on each side of a nucleobase call). In contrast, the second dimension for some individual sequencing metrics may include several dimensions representing each of the individual sequencing metrics, the derivations from the sequencing metrics, and a vectorized representation of the context nucleic acid subsequence (e.g., a one-hot-coded context nucleic acid subsequence taking up five positions).

[0101] Furthermore, when inputting multiple examples of context nucleic acid subsequences corresponding to multiple nucleobase calls into the genome location classification model 608, in some cases the genome classification system 106 inputs a three-dimensional tensor. Such a tensor can include a first dimension representing the number of examples, a second dimension representing the size or length of the context nucleic acid subsequence, and a third dimension for the number of individual sequencing metrics and / or derivatives from the individual sequencing metrics.

[0102] When inputting data derived or created from the context nucleic acid subsequences 606 into the genome location classification model 608, in some cases the genome classification system 106 inputs data derived from a single strand of DNA or RNA. For example, the genome classification system 106 inputs a vectorized form of the context nucleic acid subsequences from the positive or negative sense strands of the exemplary nucleic acid sequences (e.g., ancestral haplotypes). In some embodiments, the genome classification system 106 inputs separately the vectorized form of the context nucleic acid subsequences from both the positive and negative sense strands of the context nucleic acid subsequences determined from the exemplary nucleic acid sequences (e.g., ancestral haplotypes) and determines the confidence classifications corresponding to each of the positive and negative sense strands.

[0103] After inputting the data derived or prepared from one or both of the sequencing metrics 602 and the context nucleic acid subsequences 606, the genome classification system 106 executes a genome location classification model 608. As described above, the genome location classification model 608 can take a variety of forms. The genome location classification model 608 may be, for example, a statistical machine learning model or a neural network. In some cases, the genome location classification model takes the form of a logistic regression model, a random forest classifier, a CNN, or a long short-term memory (LSTM) network, to name a few.

[0104] For example, in some embodiments, the genome location classification model 608 takes the form of a CNN that includes two convolutional layers and one fully connected layer. In contrast, in certain cases, the genome location classification model 608 takes the form of a CNN that includes 8, 12, 20 convolutional layers and one fully connected layer. Alternatively, the genome location classification model 608 takes the form of a modified Inception Network that includes multiple convolutional layers linked together in each layer (e.g., conv3, conv5, conv7, conv9), where each convolutional layer is derived from the same previous layer.

[0105] As further shown in FIG. 6A, upon receiving the input data of the initial training iteration, the genome location classification model 608 determines a predicted confidence classification 610 of the genome coordinate corresponding to the genome coordinate identifier 604. In some embodiments, for example, the predicted confidence classification 610 includes a label indicating a high confidence classification, a medium confidence classification, or a low confidence classification that a nucleic acid base can be accurately determined at the genome coordinate corresponding to the genome coordinate identifier 604. In contrast, in certain implementations, the predicted confidence classification 610 includes a score indicating a probability or likelihood that a nucleic acid base can be determined with high confidence at the genome coordinate corresponding to the genome coordinate identifier 604. Based on such a probability or likelihood score, in some cases the genome classification system 106 determines a high confidence classification, a medium confidence classification, or a low confidence classification.

[0106] As described above, in certain implementations, the genome classification system 106 determines confidence classifications for variant type-specific genomic coordinates. Thus, when determining the predicted confidence classification 610, the genome classification system 106 can determine predicted variant confidence classifications for genomic coordinates specific to SNPS, insertions of various sizes (e.g., short, medium, or long insertions), deletions of various sizes (e.g., short, medium, or long deletions), structural variations of various sizes, or CNVs of various sizes. Additionally or alternatively, the genome classification system 106 can determine predicted variant confidence classifications for genomic coordinates specific to somatic nucleobase variants or germline nucleobase variants, such as somatic nucleobase variants reflecting cancer or somatic mosaicism or germline nucleobase variants reflecting germline mosaicism. To train the genome location classification model 608 to generate variant type-specific variant confidence classifications, as described below, the genome classification system 106 uses ground truth classifications specific to the corresponding variant type.

[0107] As further shown in FIG. 6A, after determining the predicted confidence classification 610, the genome classification system 106 compares the predicted confidence classification 610 to a ground truth classification 614 for the genome coordinate corresponding to the genome coordinate identifier 604. For example, in some implementations, the genome classification system 106 uses a loss function 612 to compare the predicted confidence classification 610 to the ground truth classification 614 (and determine any differences). As described below, in some cases, the ground truth classification 614 reflects a Mendelian inheritance pattern or a replicate match of the nucleobase calls at the genome coordinate corresponding to the genome coordinate identifier 604. As further shown in FIG. 6A, the genome classification system 106 utilizes the loss function 612 to determine a loss 616 from the predicted confidence classification 610 and the ground truth classification 614.

[0108] Depending on the type of genome location classification model 608, the genome classification system 106 can use various loss functions for the loss function 612. In certain implementations, for example, the genome classification system 106 uses logistic loss (e.g., for logistic regression models), Gini impurity or information gain (e.g., for random forest classifiers), or cross-entropy loss function or least squared error function (e.g., for CNN, LSTM).

[0109] As discussed above, the genome classification system 106 can use a variety of bases or rationales to identify a ground truth classification. In some embodiments, for example, the genome classification system 106 labels a genomic coordinate with a high confidence ground truth classification if the genomic coordinate corresponds to a nucleotide variant call that has one (or any combination) of the following characteristics: a Mendelian inheritance pattern, consistent homozygous inheritance (e.g., a genomic coordinate where the same allele comes from both parents), or a threshold number (or threshold portion) of replicates that indicate the nucleotide variant call at the genomic coordinate. For example, the genome classification system 106 can label a genomic coordinate with a high confidence ground truth classification if the threshold number (or threshold portion) of replicates equals or exceeds 56% (e.g., 54 out of 96 samples) of the sample nucleic acid sequences that indicate the nucleotide variant call. In one additional exemplary embodiment, the genome classification system 106 labels a genome coordinate with a high confidence ground truth classification if the genome coordinate corresponds to a Platinum Base or truthset base from the Platinum Genome and labels the genome coordinate with a low confidence ground truth classification if the genome coordinate does not correspond to a Platinum Base or truthset base from the Platinum Genome.

[0110] In contrast, in some cases, the genome classification system 106 labels a genomic coordinate with a low confidence ground truth classification when the genomic coordinate corresponds to a nucleotide variant call having one (or any combination) of the following characteristics: a non-Mendelian inheritance pattern, a failure or discrepancy in homozygous inheritance, or a threshold number (or threshold portion) of replicates indicative of the nucleotide variant call in the genomic coordinate. For example, the genome classification system 106 may label a genomic coordinate with a low confidence ground truth classification when the threshold number (or threshold portion) of replicates is 15% or less of the sample nucleic acid sequences indicative of the nucleotide variant call (e.g., 14 out of 96 samples).

[0111] In some embodiments, the genome classification system 106 optionally uses medium confidence labels. For example, the genome classification system 106 labels a genome coordinate with a medium confidence ground truth classification if the genome coordinate corresponds to a nucleotide variant call that has at most two of the following: a Mendelian inheritance pattern, consistent homozygous inheritance (e.g., a portion of the genome coordinate of a gene where the same allele comes from both parents), and reproducibility across technical replicates. However, the genome classification system 106 can also use labels for high confidence and low confidence classifications as ground truth classifications without the medium confidence classification.

[0112] As described above, in some cases, the genome classification system 106 labels the genomic coordinates with a ground truth classification for a particular type of nucleotide variant call. For example, the genome classification system 106 labels the genomic coordinates with a ground truth classification for one or more of SNPs, insertions of various sizes, deletions of various sizes, structural variations of various sizes, CNVs of various sizes, somatic nucleobase variants reflecting cancer or somatic mosaicism, or germline nucleobase variants reflecting germline mosaicism. Such somatic mosaicism may include either or both mosaicism in cancer cells or healthy cells with mosaic mutations. In certain implementations, the genome classification system 106 labels the genomic coordinates with a ground truth classification specific to the type of nucleotide variant call based on a threshold number (or a threshold portion) of replicates that indicate the nucleotide variant call in the genomic coordinate.

[0113] As shown in Table 1 below, the researchers identified threshold replicate counts for identifying a particular type of nucleotide variant call (e.g., SNP, deletion, insertion) at a genomic coordinate as the basis for labeling the genomic coordinate with a high or low confidence ground truth classification. In particular, the researchers determined a positive predictive value (PPV) for the rate of detecting a probabilistic false positive of a particular type of nucleotide variant call based on the technical replicate count of the particular type of nucleotide variant call from all 96 samples at a given genomic coordinate. By comparing the replicate counts to the PPV, the researchers determined the minimum replicate number reported in Table 1 at which the probabilistic false positive rate for a nucleotide variant call met a target threshold, such as a target threshold for a rate of probabilistic false positive nucleotide variant calls at a genomic coordinate for high confidence ground truth classification of less than 0.05%.

[0114] [Table 1]

[0115] As reported in Table 1, short deletions can span 1-5 nucleobases, medium deletions can span 5-15 nucleobases, long deletions can span more than 15 nucleobases and include deletions of 50 nucleobases (or less), short insertions can span 1-5 nucleobases, medium insertions can span 5-15 nucleobases, and long insertions can span more than 15 nucleobases and include insertions of 50 nucleobases (or less). The researchers determined minimum copy numbers of 54, 64, 63, 70, 63, 80, and 47 out of a total of 96 samples as thresholds for labeling genomic coordinates with high confidence ground truth classifications for SNPs, short deletions, medium deletions, long deletions, short insertions, medium insertions, and long insertions, respectively. As shown in Table 1, the minimum replication counts for labeling genomic coordinates with high confidence ground truth classifications above the corresponding minimum replication counts listed immediately above correspond to average confidences of 95.07%, 95.22%, 93.83%, 94.14%, 95.25%, 97.39%, and 81.92% for variant call reproducibility for SNPs, short deletions, medium deletions, long deletions, short insertions, medium insertions, and long insertions, respectively. In other words, the average high confidence reproducibility in Table 1 indicates the minimum number of variant replications for setting the high confidence threshold. Table 1 further reports a number of sites (e.g., genomic coordinates or genomic regions) that the genomic classification system 106 labels with high or low confidence ground truth classifications for SNPs, deletions, and insertions, according to one or more embodiments.

[0116] Instead of a label, in some embodiments, the genomic classification system 106 assigns a ground truth classification to a genomic coordinate that reflects a weighted confidence score for whether the genomic coordinate corresponds to a nucleotide variant call that has one or more of a Mendelian inheritance pattern, consistent homozygous inheritance, or reproducibility across technical replicates. For example, in some embodiments, such a confidence score for a genomic coordinate represents the sum or product of one value point for a Mendelian inheritance pattern multiplied by a first weight, one value point for consistent homozygous inheritance multiplied by a second weight, and one value point for reproducibility across technical replicates multiplied by a third weight.

[0117] Based on the determined loss 616 from the loss function 612, the genome classification system 106 then adjusts parameters of the genome location classification model 608. By adjusting the parameters, the genome classification system 106 increases the accuracy with which the genome location classification model 608 accurately determines predicted confidence classifications across training iterations. After the initial training iterations and parameter adjustment, as shown by FIG. 6A, the genome classification system 106 further determines predicted confidence classifications for different genome coordinates based on data derived or prepared from one or both of the sequencing metrics and the context nucleic acid subsequences for the different genome coordinates. In some cases, the genome classification system 106 performs training iterations until the parameters (e.g., values ​​or weights) of the genome location classification model 608 do not change significantly across training iterations or otherwise meet a convergence criterion.

[0118] While FIG. 6A illustrates training iterations that generate predicted confidence classifications for genomic coordinates, in some embodiments, the genome classification system 106 similarly inputs data and determines confidence classifications for genomic regions. In the training iterations of such embodiments, the genome classification system 106 inputs genomic region identifiers for the genomic regions and data derived or created from one or both of the sequencing metrics and context nucleic acid subsequences for each genomic coordinate within the genomic region. The genome classification system 106 further uses a genome location classification model 608 to determine a predicted confidence classification for the genomic region based on such genomic region-specific input. The genome classification system 106 similarly uses a loss function to compare the predicted confidence classification for the genomic region to a ground truth classification for the genomic region and adjusts parameters of the genome location classification model 608 based on the determined loss from the loss function.

[0119] After training the genomic location classification model 608, as shown in FIG. 6B, the genomic classification system 106 applies the trained version of the genomic location classification model 608 to determine a set of confidence classifications for the set of genomic coordinates and generate a digital file including the set of confidence classifications. Similar to the training process described above, as shown in FIG. 6B, the genomic classification system 106 determines confidence classifications for subsequent genomic coordinates based on data derived or prepared from one or both of the sequencing metrics and the context nucleic acid subsequences corresponding to the particular genomic coordinate. For simplicity, this disclosure describes an initial application iteration or initial process for determining a single confidence classification, followed by a summary of subsequent application iterations shown in FIG. 6B.

[0120] For example, in the first application iteration shown in FIG. 6B, the genome classification system 106 inputs data derived or prepared from one or both of the sequencing metrics 618 and the context nucleic acid subsequences 622 corresponding to the genome coordinate identifier 620 for a particular genome coordinate into a trained version of the genome location classification model 608. As in training, the genome classification system 106 can input any combination of data created from the sequencing metrics 618 specific to the genome coordinate corresponding to the genome coordinate identifier 620 and / or the context nucleic acid subsequences 622 specific to the genome coordinate. The genome classification system 106 can similarly input data created from the sequencing metrics 618 and / or the context nucleic acid subsequences 622 by using an input vector or input matrix in the same format as described above. The context nucleic acid subsequences 622 input to the trained version of the genome location classification model 608 can similarly be single stranded (e.g., positive or negative) of DNA or RNA. However, in some embodiments, the genome classification system 106 uses a different set of sequencing metrics and / or a different set of context nucleic acid subsequences (and corresponding nucleobase calls) to apply a trained version of the genome location classification model 608 than the sequencing metrics and context nucleic acid subsequences used for training.

[0121] As further shown in FIG. 6B, in a first application iteration, the trained version of the genome location classification model 608 determines a confidence classification 624 for the genome coordinate corresponding to the genome coordinate identifier 620. Consistent with the training above, the confidence classification 624 may include (i) a label for a high, medium, or low confidence classification that the nucleobase can be accurately determined at the genomic coordinate corresponding to the genome coordinate identifier 620, or (ii) a score indicating the probability or likelihood that the nucleobase can be determined with high confidence at the genomic coordinate corresponding to the genome coordinate identifier 620. Based on the type of ground truth classification used to train the genome location classification model 608, the confidence classification 624 may similarly be specific to the type of nucleotide variant call, such as being specific to one or more of SNPs, insertions of various sizes, deletions of various sizes, structural variations of various sizes, CNVs of various sizes, somatic nucleobase variants reflective of cancer or somatic mosaicism, or germline nucleobase variants reflective of germline mosaicism.

[0122] After the first application iteration, the genome classification system 106 further determines confidence classifications for the different genome coordinates based on data derived or prepared from one or both of the sequencing metrics and the context nucleic acid subsequences for the different genome coordinates. Upon completing such application iterations, as shown in FIG. 6B, the genome classification system 106 determines a set of confidence classifications for the set of genome coordinates based on data derived or prepared from the set of sequencing metrics and the context nucleic acid subsequences. In some cases, the set of confidence classifications includes a confidence classification for each genome coordinate in the reference genome. In contrast, in certain implementations, the set of confidence classifications includes confidence classifications for some (but not all) genome coordinates in the reference genome.

[0123] As further shown in Figure 6B, the genome classification system 106 further generates a digital file 626 that includes confidence classifications 628. As shown in Figure 6B, the confidence classifications 628 include a set of confidence classifications for the set of genomic coordinates generated by the genome location classification model 608 of Figure 6B. Similar to the confidence classifications 624, depending on the type of ground truth classifications used to train the genome location classification model 608, the confidence classifications 628 may similarly be specific to the type of nucleotide variant call, such as being specific to one or more of a SNP, various sized insertions, various sized deletions, structural variations, CNVs, somatic nucleobase variants reflective of cancer or somatic mosaicism, or germline nucleobase variants reflective of germline mosaicism.

[0124] To generate or modify the digital file 626, in certain implementations, the genome classification system 106 generates or modifies a BED file to include annotations for each genome coordinate including a corresponding confidence classification. In contrast, in some embodiments, the genome classification system 106 generates or modifies a WIG file, a BAM file, a VCF file, a microarray file, or other suitable digital file type to include the confidence classifications 628. As further illustrated by FIG. 6B, in some embodiments, the genome classification system 106 can generate separate digital files each including a different confidence classification type from the predicted confidence classifications (e.g., a different digital file for each of a high, medium, and low confidence classification).

[0125] While Figure 6B illustrates an application iteration that generates confidence classifications for genomic coordinates, in some embodiments, the genome classification system 106 similarly inputs data and determines confidence classifications for genomic regions. In an application iteration of such an embodiment, the genome classification system 106 inputs genomic region identifiers for the genomic regions and data derived or created from one or both of the sequencing metrics and context nucleic acid subsequences for each genomic coordinate within the genomic region. The genome classification system 106 further uses a genome location classification model 608 to determine a confidence classification for the genomic region based on such genomic region-specific input.

[0126] After generating the digital file 626 (e.g., a portion of a separate digital file), in some cases, the genome classification system 106 uses the digital file 626 to provide a particular confidence classification for the genomic coordinate (or region) of the nucleobase call for display on a graphical user interface. According to one or more embodiments, Figure 6C shows the sequencing system 104 or genome classification system 106 identifying and displaying a particular confidence classification from the genome location classification model 608 that corresponds to a particular genomic coordinate of the nucleotide variant call.

[0127] As shown by FIG. 6C, for example, the sequencing device 630 incorporates nucleobases into the sample nucleic acid sequence during sequencing and captures a corresponding image (or other data) indicative of the incorporated nucleobases. Based on the image or other data, the sequencing system 104 or genome classification system 106 detects variant-nucleobase calls 632a, 632b, and 632n in the sample nucleic acid sequence at genomic coordinates. In some embodiments, the variant-nucleobase calls 632a-632n represent SNVs, nucleobase insertions, nucleobase deletions, structural variations, CNVs. Additionally or alternatively, in certain implementations, the variant-nucleobase calls 632a-632n represent somatic-nucleobase variants reflecting cancer or somatic mosaicism, or germline-nucleobase variants reflecting germline mosaicism. The variant-nucleobase calls 632a-632n may be caused by genetic or epigenetic modifications as well.

[0128] As further shown in Figure 6C, genome classification system 106 integrates variant-nucleobase calls 632a-n with one or more of the confidence classifications 628 from digital file 626 (or from one of the digital files). For example, in some cases, genome classification system 106 encodes variant-nucleobase calls 632a-n into digital file 626, compares variant-nucleobase calls 632a-n with confidence classifications 628 from digital file 626 (or from one of the digital files), or retrieves confidence classifications 628 from digital file 626 and integrates them into a separate digital file (e.g., a VCF file) for variant-nucleobase calls 632a-n. Additionally or alternatively, in certain implementations, digital file 626 includes a lookup table for genomic coordinates that correspond to confidence classifications, such as a different lookup table for different variant types that includes confidence classifications to which genomic coordinates correspond. Regardless of how such integration occurs, the genome classification system 106 identifies a particular confidence classification from the confidence classifications 628 for a particular genomic coordinate of a variant-nucleobase call 632a-632n.

[0129] In addition to including variant-nucleobase calls 632a-632n, in some cases, genome classification system 106 identifies variant-nucleobase calls or non-variant-nucleobase calls in digital file 214 proposed for orthogonal validation using different sequencing methods. For example, if a variant-nucleobase call is located at a genomic coordinate that corresponds to a lower confidence classification for a particular type of variant (e.g., a low confidence classification or below a confidence score threshold), genome classification system 106 includes an identifier for such variant-nucleobase call in digital file 214 to suggest orthogonal validation. By using a particular confidence classification as the confidence threshold, genome classification system 106 can flag a particular variant-nucleobase call or non-variant-nucleobase call that a single sequencing pipeline cannot determine with sufficient confidence.

[0130] After identifying such confidence classifications from the digital file 626, as further shown in Figure 6C, the genome classification system 106 provides a confidence indicator of the particular confidence classification for the genomic coordinates of the variant-nucleobase calls 632a-632n to the computing device 636. For example, as shown in Figure 6C, the sequencing system 104 or genome classification system 106 provides confidence indicators 638a and 638b of the confidence classification along with the genomic coordinates of the variant-nucleobase calls 632a and 632b and the identifiers of the corresponding genes for display within the graphical user interface 634 of the computing device 636. By providing the confidence indicators 638a and 638b, the genome classification system 106 provides important information to a clinician, test subject, or other person indicating the confidence of the variant-nucleobase calls 632a and 632b for a particular gene.

[0131] As alluded to above, in some embodiments, the genome classification system 106 trains or applies a genome location classification model to determine confidence classifications specific to somatic nucleobase variants reflecting cancer or somatic mosaicism, or specific to germline nucleobase variants. To train such a genome location classification model, in some embodiments, the genome classification system 106 determines a subset of nucleic acid sequences from different genomic samples that simulate nucleobase variants from a type of cancer or mosaicism. The genome classification system 106 further determines specific sequencing metrics for the sample nucleic acid sequences with respect to genomic coordinates of the reference genome. Based on these sequencing metrics, the genome classification system 106 generates ground truth classifications specific to both specific genomic coordinates and specific variant-nucleobase calls, such as somatic nucleobase variants reflecting mosaicism or germline nucleobase variants. Using the ground truth classifications, as described above, the genome classification system 106 can further train a genome location classification model to determine confidence classifications specific to both genomic coordinates and types of variant-nucleobase calls.

[0132] According to one or more embodiments, Figures 6D-6H show a genomic classification system 106 that determines a ground truth classification based on one or both of (i) specific sequencing metrics for sample nucleic acid sequences from genomic samples (e.g., a diverse cohort of genomic samples as described above), and (ii) variant-calling data for a mixture of genomic samples reflecting cancer or mosaicism (e.g., recall or precision rates for calling a specific type of variant for a mixture of genomic samples reflecting cancer or mosaicism). As shown in Figure 6D, the genomic classification system 106 determines a subset (e.g., percentage) of sample nucleic acid sequences from a combination of male and female genomic samples that together simulate the variant-allele frequencies of genomic samples with cancer or mosaicism. As shown in Figure 6E, the genomic classification system 106 determines genomic coordinates that exhibit normal behavior in one or more of depth metrics, mapping quality metrics, or nucleobase call quality metrics for the sample nucleic acid sequences as a basis for determining a ground truth classification for the high confidence genomic coordinates. As further shown in Figures 6F-6H, the genomic classification system 106 determines a ground truth classification further based on the somatic quality metrics for the nucleobase calls from the sample nucleic acid sequences and one or both of the recall or precision rates for determining a particular type of variant-nucleobase call based on a mixture of genomic samples.

[0133] As shown in FIG. 6D, for example, the genome classification system 106 determines a subset of sample nucleic acid sequences from different genome samples that form a mixed genome. When the corresponding sample nucleic acid sequence subsets are mixed together, the mixed genome simulates a genome sample with cancer or mosaicism. For example, to simulate such a genome sample with cancer or mosaicism, the genome classification system 106 determines a proportion of sample nucleic acid sequences 640a from the first genome sample 639a and a proportion of sample nucleic acid sequences 640b from the second genome sample 639b that, when mixed together, simulate the variant-allele frequency of the genome sample that exhibits the characteristics of cancer or mosaicism. As part of determining the subsets of sample nucleic acid sequences 640a and 640b, the genome classification system 106 estimates the variant-allele frequency of the different subset mixtures (or percentage mixtures) from the truthset bases of the Platinum Genome for the first genome sample 639a and the second genome sample 639b.

[0134] In some embodiments, the genome classification system 106 uses sample nucleic acid sequences from mixed genomes rather than a single naturally occurring genome, because sequencing systems often cannot consistently or accurately detect nucleobase variants reflecting cancer or mosaicism in sequences from naturally occurring genomes. For example, a metastatic tumor may mutate nucleobases in the DNA of some somatic cell types but not others. In fact, some tumors may affect all cells of a particular cell type, such as leukemia that spreads in the blood, making a sample of the tumor only available exclusively and making it impractical or impossible to obtain a control sample. In different biopsy tissue samples or at different biopsy times, DNA extracted from naturally occurring genomes with cancer may have significantly different nucleobase allele frequencies, making the sample of naturally occurring genomes an unpredictable sample for estimating variant-allele frequencies caused by some cancers. To avoid the unpredictable variability of nucleobase variants in DNA of cancer or healthy cells, in some implementations, the genome classification system 106 determines a mixed genome that simulates variants reflecting cancer.

[0135] In contrast to variants resulting from cancer, naturally occurring mosaicism in the DNA of a sample may represent rare variants that are difficult to detect during sequencing, whether the mosaicism is caused by a tumor, genetic inheritance, replication errors, or some other factor. Although a single person may have a small percentage of DNA that represents mosaicism, many existing sequencing systems are unable to detect common nucleobase variants that reflect mosaicism unless the sequencing system sequences oligonucleotides from a much larger group of samples that have that type of mosaicism. To create training genome samples without finding rare groups of samples that represent mosaicism, in certain embodiments, the genome classification system 106 determines a mixed genome to simulate variants that reflect somatic or germline mosaicism.

[0136] FIG. 6D shows an example of the genome classification system 106 determining a subset of sample nucleic acid sequences for one such mixed genome and determining the corresponding variant-allele frequency. As shown in FIG. 6D, the genome classification system 106 determines the variant-allele frequency for both heterozygous and homozygous alleles of SNPs for the mixed genome. According to the percentage reflected by the subset of sample nucleic acid sequences 640a (here 60%) and the subset of sample nucleic acid sequences 640b (here 40%), the genome classification system 106 determines or predicts the associated variant-allele frequency by referring to the truthset bases of the first genome sample 639a (e.g., NA12877) and the second genome sample 639b (e.g., NA12878) from Platinum Genome. Although FIG. 6D shows the variant-allele frequencies for SNPs from the mixed genome, the genome classification system 106 can determine the mixed genome and variant-allele frequencies for other specific variant types, such as insertions, deletions, structural changes, or CNVs.

[0137] For example, as shown in the allele frequency table 642 presented in Figure 6D, the genome classification system 106 determines that the unique homozygous alleles and the unique heterozygous alleles from the second genomic sample 639b occur in the mixed genome at variant-allele frequencies of 0.4 and 0.2, respectively. As further shown, the genome classification system 106 determines that the unique homozygous alleles and the unique heterozygous alleles from the first genomic sample 639a occur in the mixed genome at variant-allele frequencies of 0.6 and 0.3, respectively. In contrast, the genomic classification system 106 determines that common alleles present in the 60% and 40% mixed genomes as homozygous-homozygous combinations, heterozygous-homozygous combinations, homozygous-heterozygous combinations, and heterozygous-heterozygous combinations occur at variant-allele frequencies of 1.0, 0.8, 0.7, and 0.5, respectively, according to the corresponding allele zygosity in the second genomic sample 639b and the first genomic sample 639a.

[0138] To select an appropriate mixed genome representative of a genomic sample with cancer or mosaicism, the genome classification system 106 can determine variant-allele frequencies from the truthset bases of various combinations (and percentages) of genomic samples in a given mixed genome. In addition to the variant-allele frequencies present in the 60% and 40% mixed genomes shown in Figure 6D, in some embodiments, the genome classification system 106 determines variant-allele frequencies for other possible mixed genomes to simulate a genomic sample with cancer or mosaicism. For example, the genome classification system 106 determines that 30% of the sample nucleic acid sequences from the first genome sample 639a and 70% of the sample nucleic acid sequences from the second genome sample 639b generate unique homozygous alleles from the first genome sample 639a and the second genome sample 639b at variant-allele frequencies of 0.7 and 0.3, respectively, and unique heterozygous alleles from the first genome sample 639a and the second genome sample 639b at variant-allele frequencies of 0.35 and 0.15, respectively. In contrast, the genome classification system 106 determines or predicts that common alleles present in such 30% and 70% mixed genomes as homozygous-homozygous combinations, heterozygous-homozygous combinations, homozygous-heterozygous combinations, and heterozygous-heterozygous combinations according to the same 30% and 70% mixtures will generate variant-allele frequencies of 1.0, 0.85, 0.65, and 0.5, respectively.

[0139] In addition to determining various mixed genomes from the first genomic sample 639a and the second genomic sample 639b, in certain implementations, the genome classification system 106 determines variant-allele frequencies from combinations of the different sample genomes to identify an appropriate mixed genome that simulates a genomic sample having cancer or mosaicism. By determining variant-allele frequencies for various mixed genomes, the genome classification system 106 can select a mixed genome that more closely (or most closely) simulates the variant-allele frequencies of the target type or cancer or mosaicism.

[0140] As described above, the genome classification system 106 can generate ground truth classifications specific to somatic nucleobase variants reflecting cancer or mosaicism, or specific to germline nucleobase variants based in part on specific sequencing metrics. As shown in FIG. 6E, in some embodiments, the genome classification system 106 sorts or labels genomic coordinates with a high confidence classification (or other confidence classification) by (i) determining a sequencing metric distribution 644 for sample nucleic acid sequences from a genomic sample (e.g., a diverse cohort of genomic samples as described above) across the genomic coordinates, and (ii) identifying genomic coordinates having specific sequencing metrics that fall within a target portion of the normal distribution. In the illustrated example, the genome classification system 106 identifies genomic coordinates within a high confidence region 652 if they exhibit a depth metric, a mapping quality metric, and a nucleobase call quality metric within a standard deviation of the normal distribution of the three sequencing metrics. As discussed below, genomic coordinates that exhibit normal depth metrics, mapping quality metrics, and nucleobase call quality metrics and thus are part of high confidence regions 652 also exhibit better accuracy for determining variant-nucleobase calls based on mixtures of genomic samples.

[0141] As shown in FIG. 6E, the genome classification system 106 determines a sequencing metric distribution 644 of sample nucleic acid sequences from genome samples (e.g., a diverse cohort of genome samples) at the genome coordinates of the reference genome. To determine such a distribution, the genome classification system 106 system determines sequencing metrics for sequenced genome samples from the diverse cohorts and determines the distribution of sequencing metrics according to different genome coordinates. For example, in certain cases, the genome classification system 106 determines nucleic acid base calls for genome samples (e.g., by using tumor-only analysis in the DRAGEN Somatic Pipeline) and determines sequencing metrics for the sequences determined for the genome samples. In some embodiments, the genome classification system 106 determines depth metrics, mapping quality metrics, and nucleic acid base call quality metrics for the sample nucleic acid sequences for each genome coordinate. In contrast, in certain implementations, the genome classification system 106 determines one or more of any of the above sequencing metrics, including but not limited to any of one or more of the above alignment metrics, depth metrics, or call data quality metrics.

[0142] As further shown in FIG. 6E, the genome classification system 106 identifies normal genomic coordinates 646 and outlier genomic coordinates 648 based on one or more of the sequencing metric distributions 644. For example, the genome classification system 106 fits a Bayesian Gaussian mixture model to the genome-wide distribution for each of the depth metrics, mapping quality metrics, nucleobase call quality metrics, and / or other sequencing metrics described above across genome coordinates. The genome classification system 106 then uses an algorithm to prune or remove components (e.g., a subset of sequencing metrics) that do not contribute or contribute little to a proper fit of the genome-wide distribution to the Bayesian Gaussian mixture model for each sequencing metric. Based on the fit distribution for each sequencing metric, the genome classification system 106 sets a p-value threshold to define or identify normal genomic coordinates 646 that fall within the fit distribution and outlier genomic coordinates 648 that fall outside the fit distribution according to each particular sequencing metric. Thus, a genomic coordinate may be one of the normal genomic coordinates 646 for one sequencing metric, but one of the outlier genomic coordinates 648 for another sequencing metric.

[0143] After identifying normal genomic coordinates 646 and outlier genomic coordinates 648, the genome classification system 106 further identifies genomic coordinates that exhibit normal depth metrics, mapping quality metrics, and nucleobase call quality metrics as part of a high confidence region 652. As shown by overlap visualization 650, the genome classification system 106 determines genomic coordinates that fall within a distribution (e.g., a fit distribution) for each of the depth metrics, mapping quality metrics, and nucleobase call quality metrics. The identified genomic coordinates form the high confidence region 652, which includes 89.9% of the reference genome, excluding gaps in other regions. Genomic coordinates that fall outside the distribution for any one of the depth metrics, mapping quality metrics, and nucleobase call quality metrics form a low confidence region 654. As shown in FIG. 6E, in certain embodiments, the genome classification system 106 labels genomic coordinates within the high confidence region 652 with a high confidence ground truth classification for a somatic-nucleobase variant that reflects cancer.

[0144] As alluded to above, genomic coordinates that exhibit normal depth metrics, mapping quality metrics, and nucleobase call quality metrics also indicate better accuracy or precision for determining variant-nucleobase calls. To test confidence and further distinguish ground truth classification, in some embodiments, the genome classification system 106 determines nucleobase calls for the mixed genome and compares the nucleobase calls to the truthset bases unique to the genomic samples that form the mixed genome from the Platinum Genome. By comparing the variant calls for the mixed genome to the corresponding truthset bases, the genome classification system 106 can identify true positive variants in the corresponding genomic coordinates.

[0145] Since there are very few variants in the mixed genome that simulate cancer or mosaicism, in some implementations, the genome classification system 106 identifies false positive variants determined in genomic coordinates using a normal-normal subtraction method. In particular, the genome classification system 106 determines nucleobase calls for two replicates of the same genome sample (e.g., NA12877) from the mixture by treating one replicate as a tumor sample and another replicate as a normal sample in tumor / normal data analysis from Illumina, Inc., and compares the nucleobase calls from the two replicates to identify false positive variants. When performing such an analysis, for example, the genomic classification system 106 can use the tumor / normal data analysis described by Illumina, Inc., "Evaluating Somatic Variant Calling in Tumor / Normal Studies" (2015) (available at https: / / www.illumina.com / content / dam / illumina-marketing / documents / products / whitepapers / whitepaper_wgs_tn_somatic_variant_calling.pdf), the contents of which are incorporated herein by reference. By measuring the density of false positive variants at a genomic coordinate or genomic region, the genomic classification system 106 can identify genomic coordinates or regions that are least likely to produce errors in determining a nucleobase variant call for a given genomic sample with cancer or mosaicism. According to one or more embodiments, FIG. 6F shows a false positive density graph 656 illustrating the density of false positives determined within the high confidence region 652 and the low confidence region 654 from FIG. 6E at different read depths.

[0146] In addition to determining the density of false positive variants, in some embodiments, the genome classification system 106 determines a somatic quality metric of the nucleobase calls from the sample nucleic acid sequences of the mixed genome and determines the density of false positive variants within the portion of the low confidence region 654 from FIG. 6E divided by a somatic quality metric threshold. As described further below, in some cases, the genome classification system 106 uses the somatic quality metric threshold to distinguish different tiers of ground truth classification for genomic coordinates in either the low confidence region 654 or the high confidence region 652. According to one or more embodiments, FIG. 6F further illustrates a false positive density graph 656 illustrating the density of false positives determined within different tiers of the low confidence region 654 from FIG. 6E at different somatic quality metric thresholds and different read depths.

[0147] As shown in the false positive density graph 656 of FIG. 6F, the genomic classification system 106 determines the density of false positive variants per million bases (Mb) in genomic coordinates in high confidence and low confidence regions at different read depths. The genomic classification system 106 further determines the density of false positive variants in low confidence regions according to different somatic quality metric thresholds, i.e., somatic quality metrics having values ​​of 17.5, 20, and 25. For a read depth of 100 in a genomic coordinate, the genomic classification system 106 determines a false positive density of just over 0.1 / Mb for genomic coordinates in the high confidence region, a false positive density of over 1.6 / Mb for genomic coordinates in the low confidence region with a somatic quality metric between 17.5 and 20, a false positive density of over 0.8 / Mb for genomic coordinates in the low confidence region with a somatic quality metric between 20 and 25, and a false positive density of over 0.2 / Mb for genomic coordinates in the low confidence region with a somatic quality metric above 25. For a read depth of 75 at a given genomic coordinate, the genomic classification system 106 determines a false positive density of just under 0.1 / Mb for genomic coordinates in high confidence regions, a false positive density of over 1.1 / Mb for genomic coordinates in low confidence regions with somatic quality metrics of 17.5 to 20, a false positive density of over 0.7 / Mb for genomic coordinates in low confidence regions with somatic quality metrics of 20 to 25, and a false positive density of approximately 0.3 / Mb for genomic coordinates in low confidence regions with somatic quality metrics above 25.

[0148] As the false positive density graph 656 shows, the density of false positive variants increases as the somatic quality metric for genomic coordinates in the low confidence region decreases. Conversely, as the somatic quality metric threshold increases, the density of false positive variants decreases and the density of false negative variants increases. Because the density of false positive variants is an inverse indicator of the accuracy of the somatic variant caller, the false positive density graph 656 shows that the accuracy with which the genomic classification system 106 determines somatic variant calls for false positive variants increases as the somatic quality metric for genomic coordinates in the low confidence region decreases.

[0149] By using the somatic cell quality metric threshold, in some implementations, the genome classification system 106 can accordingly distinguish ground truth classifications for genomic coordinates in low confidence regions. For example, in some cases, the genome classification system 106 can label genomic coordinates from low confidence regions with a low confidence classification if the corresponding somatic cell quality metric is less than 25, and with a medium confidence classification if the corresponding somatic cell quality metric is greater than 25. In contrast, the genome classification system 106 can score genomic coordinates from low confidence regions with a lower confidence score if the corresponding somatic cell quality metric is less than 25, and with a higher confidence score if the corresponding somatic cell quality metric is greater than 25. As just mentioned, the threshold of 25 for distinguishing ground truth classifications is merely an example. In additional embodiments, the genome classification system 106 uses one or more different thresholds (e.g., 15, 20, 30) for the somatic cell quality metric.

[0150] As further illustrated by the false positive density graph 656 in FIG. 6F, in some embodiments, the genome classification system 106 can use a different, more stringent somatic quality metric threshold for low confidence regions to identify more confident genomic regions among those that are often considered low quality by conventional systems. Conventional variant callers typically use a threshold for somatic variant call quality. For candidate nucleobase calls with a quality below the threshold, the conventional variant caller excludes the corresponding nucleobase call (e.g., marks it as non-PASS). If the threshold somatic quality metric increases, the variant caller filters more nucleobase calls, which results in a reduction in false positive variants but an increase in false negative variants. Typically, the threshold for the somatic quality metric used by the variant caller is selected to achieve an optimal balance of false positive and false negative variants. However, by filtering nucleobase calls using the above somatic quality metric threshold, the genome classification system 106 can significantly reduce false positive variants without unduly penalizing recall, as further illustrated below.

[0151] As described above, in certain implementations, the genome classification system 106 determines a recall for determining variant-nucleobase calls at particular genomic coordinates and generates a ground truth classification based in part on the recall. For example, in certain cases, the genome classification system 106 determines somatic variant calls for a mixture of genomic samples and compares the somatic variant calls to a truthset (e.g., from Platinum Genome) for corresponding genomic samples from the mixture to determine the recall. In some embodiments, the genome classification system 106 determines the recall by determining the number of correctly determined true positive nucleobase call variants divided by the number of all true positive nucleobase call variants. Thus, the genome classification system 106 can determine and use such recall to identify ground truth classifications specific to (i) somatic nucleobase variants reflecting cancer or mosaicism, or (ii) germline nucleobase variants reflecting mosaicism.

[0152] According to one or more embodiments, Figure 6G shows recall graphs 658a and 658b illustrating recall for a genomic classification system 106 that determines somatic nucleobase variants reflective of cancer at genomic coordinates in different genomic regions and at different variant-allele frequencies. In particular, recall graphs 658a and 658b show recall at 100 read depth and 75 read depth, respectively, for genomic coordinates in high confidence regions and low confidence regions partitioned according to somatic quality metric thresholds of 17.5, 20, and 25 across different variant-allele frequencies.

[0153] As shown by recall graphs 658a and 658b for read depths of 100 and 75, respectively, at a given genomic coordinate, the genomic classification system 106 determines recall for determining somatic variants reflective of cancer at various genomic coordinates and across various variant-allele frequencies. As shown in both recall graphs 658a and 658b, genomic coordinates within the high confidence region exhibit higher recall across variant-allele frequencies than either of the partitioned low confidence regions. Because nucleobase variants with variant-allele frequencies between 0.05 and 0.2 are present in relatively few reads at a given genomic coordinate, the sequencing system lacks sufficient reads (even at read depths of 100 and 75 for the genomic coordinate) to determine the corresponding nucleobase variant call in the high confidence region with a recall of near 1.0, as indicated by the higher variant-allele frequencies.

[0154] As further shown in both recall graphs 658a and 658b, genomic coordinates in the low confidence region with a somatic quality metric of 25, the low confidence region with a somatic quality metric threshold of 20, and the low confidence region with a somatic quality metric threshold of 17.5 each show increasingly better recall across variant-allele frequencies. In other words, as the somatic quality metric threshold for filtering increases for a genomic coordinate, the recall for determining somatic variants reflective of cancer decreases for the genomic coordinate. Note that this relationship between the somatic quality metric threshold and recall does not represent a somatic quality metric increase. As the somatic quality metric increases, the recall for determining somatic variants should increase as well, and somatic variant calls will be less prone to both false negative and false positive variants.

[0155] By using both the somatic cell quality metric threshold and the recall, in some implementations, the genome classification system 106 can distinguish ground truth classifications for genomic coordinates in low confidence regions accordingly. For example, in some cases, the genome classification system 106 labels genomic coordinates from low confidence regions with a low confidence classification when the corresponding somatic cell quality metric is below 25 (or some other somatic cell quality metric threshold). Conversely, the genome classification system 106 labels genomic coordinates from low confidence regions with a medium confidence classification when the corresponding somatic cell quality metric is above 25 (or some other somatic cell quality metric threshold). In contrast, the genome classification system 106 can score genomic coordinates from low confidence regions with a lower (or higher) confidence score when the corresponding somatic cell quality metric is above or below 25.

[0156] In contrast, in some embodiments, the genome classification system 106 can distinguish ground truth classifications for genomic coordinates in low confidence regions based on the F-scores of genomic coordinates with different somatic quality metric thresholds. For example, the genome classification system 106 can determine an F-score for determining variant-nucleobase calls in genomic coordinates in low confidence regions based on both recall and precision rates. In some embodiments, the genome classification system 106 determines the precision rate by determining the number of correctly determined true positive nucleobase call variants divided by the number of all determined nucleobase call variants. In some cases, the genome classification system 106 determines the F1 score by determining the harmonic mean of the precision rate and recall rate. Thus, the genome classification system 106 can label genomic coordinates in low confidence regions with different somatic quality metric thresholds with different ground truth classifications according to the corresponding F-scores of the genomic coordinates with different somatic quality metric thresholds.

[0157] As further indicated above, in certain implementations, the genome classification system 106 determines one or both of a precision rate and a recall rate to determine a variant-nucleobase call at a particular genomic coordinate, and generates a ground truth classification based on one or both of the precision rate and the recall rate. For example, in some cases, the genome classification system 106 determines somatic variant calls for a mixture of genomic samples (e.g., by using a tumor / normal DRAGEN somatic pipeline when determining a somatic variant call that simulates cancer, or by using tumor-only analysis in the DRAGEN somatic pipeline when determining a somatic variant call that simulates mosaicism). The genome classification system 106 then compares the somatic variant calls to a truthset (e.g., from Platinum Genome) for corresponding genomic samples from the mixture to determine precision and recall. Thus, the genome classification system 106 can determine and use such precision or recall to identify ground truth classifications specific to (i) somatic nucleobase variants reflective of cancer or mosaicism, or (ii) germline nucleobase variants reflective of mosaicism.

[0158] According to one or more embodiments, Figure 6H shows precision graphs 660a and 660b illustrating the precision with which the genome classification system 106 determines variant-nucleobase calls that reflect mosaicism at genomic coordinates and different variant-allele frequencies within different genomic regions. Figure 6H further shows recall graphs 662a and 662b illustrating the recall for the genome classification system 106 determining nucleobase variants that reflect mosaicism at genomic coordinates and different variant-allele frequencies within different genomic regions.

[0159] As shown by accuracy graphs 660a and 660b for read depths of 100 and 75, respectively, at a given genomic coordinate, the genome classification system 106 determines a percentage of accuracy for determining nucleobase variants that reflect mosaicism at various genomic coordinates and across various variant-allele frequencies. As shown in both accuracy graphs 660a and 660b, genomic coordinates in the high confidence regions generally exhibit higher accuracy across variant-allele frequencies than genomic coordinates in the low confidence regions. Starting at a variant-allele frequency of 0.15 in both accuracy graphs 660a and 660b, genomic coordinates in the low confidence regions exhibit approximately the same accuracy rate as genomic coordinates in the high confidence regions, approximately 1.000.

[0160] As shown by recall graphs 662a and 662b, respectively, for read depths of 100 and 75 at a given genomic coordinate, the genome classification system 106 determines recall for determining nucleobase variants that reflect mosaicism at various genomic coordinates and across various variant-allelic frequencies. As shown in both recall graphs 662a and 662b, genomic coordinates within high confidence regions consistently show higher recall across variant-allelic frequencies than genomic coordinates within low confidence regions.

[0161] As alluded to above, nucleobase variants with variant-allele frequencies between 0.05 and 0.15 are present in relatively few nucleotide reads at a given genomic coordinate, and therefore the sequencing system lacks sufficient reads (even at read depths of 100 and 75 for a genomic coordinate) to determine the corresponding nucleobase variant calls with near 1.0 precision or near 1.0 recall rates indicated by the higher variant-allele frequencies.

[0162] In addition to determining precision and recall, in certain implementations, the genome classification system 106 further determines an F-score for determining variant-nucleobase calls in genome coordinates based on precision and recall. As described above, in some cases, the genome classification system 106 determines the F1 score by determining the harmonic mean of the precision rate and recall. Thus, the genome classification system 106 can label genome coordinates or regions, such as high confidence regions and low confidence regions, with different ground truth classifications according to the relative F1 scores.

[0163] Based on one or both of the recall and precision rates, in some implementations, the genome classification system 106 distinguishes between ground truth classifications for genomic coordinates in high confidence regions and low confidence regions. For example, in some cases, the genome classification system 106 labels genomic coordinates in high confidence regions with a high confidence classification, in part because genomic coordinates in high confidence regions exhibit better recall and precision rates. In contrast, in some cases, the genome classification system 106 labels genomic coordinates in low confidence regions with a low confidence classification (or a medium confidence classification), in part because low confidence regions exhibit lower recall and precision rates.

[0164] Regardless of how the genome classification system 106 determines or labels such ground truth classifications, in some cases the genome classification system 106 trains a genome location classification model 608 to determine variant confidence classifications for genomic coordinates based on such determined ground truth classifications as shown in Figure 6A for somatic nucleobase variants reflecting cancer or somatic mosaicism, or for germline nucleobase variants reflecting germline mosaicism. Thus, the genome classification system 106 can similarly utilize a trained version of the genome location classification model 608 to determine variant confidence classifications for a set of genomic coordinates and that are specific for somatic nucleobase variants reflecting cancer or somatic mosaicism, or specific for germline nucleobase variants reflecting germline mosaicism, as shown in Figure 6B. As a result, the genomic classification system 106 can also identify and display variant confidence classifications from a trained version of the genomic location classification model 608 for variant calls that correspond to the genomic coordinates of somatic nucleobase variants that reflect cancer or somatic mosaicism, or for germline nucleobase variants that reflect germline mosaicism, as shown in FIG. 6C.

[0165] As described above, to evaluate the performance of different embodiments of the genome location classification model, researchers measured variables and various accuracy metrics as evidenced by the confidence classification of the genome classification system 106. The following paragraphs describe some of the measurements as shown in Figures 7-10B. According to one or more embodiments, for example, Figures 7A-7G show graphs 700a-700g illustrating the sequencing metrics and sequencing metric derived input data that inform the genome location classification model for a particular variant type when trained from a logistic regression model. In particular, graphs 700a-700g show the logistic regression coefficients used by the genome location classification model for the top 23 sequencing metrics and sequencing metric derived input data to determine high or low confidence classifications for genome coordinates based on different nucleobase call variant types.

[0166] As shown in Figures 7A and 7B, for example, graphs 700a and 700b show logistic regression coefficients for genomic location classification models trained using ground truth classifications corresponding to either short deletions of 1-5 nucleobases in length (for graph 700a) or short insertions of 1-5 nucleobases in length (for graph 700b), respectively. Figures 7A and 7B show logistic regression models trained using the standardized depth with the highest magnitude coefficient compared to short deletion or short insertion weighted mapping quality metric (MAPQ) or other data inputs to determine a high or low confidence classification for a genomic coordinate or genomic region.

[0167] In particular, graph 700a in FIG. 7A shows that a logistic regression model trained for short deletions uses coefficients of greater than −1.5 and greater than 1.5 for mapping quality metrics to determine high and low confidence classifications for genomic coordinates or genomic regions, respectively. Graph 700b in FIG. 7B shows that a logistic regression model trained for short insertions uses coefficients of greater than −1.5 and greater than 1.5 for standardized depth metrics to determine high and low confidence classifications for genomic coordinates or genomic regions, respectively. Such standardized depth metrics are subject to standard deviation and may include forward-reverse depth metrics or normalized depth metrics.

[0168] In contrast, graph 700a in Figure 7A shows that the logistic regression model trained for short deletions uses coefficients of 0.0 and near 0.0 that are smaller in magnitude than other data inputs for short deletions for the forward fraction metric and the local average of the read reference mismatch metric (local_average_mismatch) to determine high and low confidence classifications for genomic coordinates. Graph 700b in Figure 7B shows that the logistic regression model trained for short insertions uses coefficients of 0.0 that are smaller in magnitude than other data inputs for short insertions for the higher negative insertion size metric to determine high and low confidence classifications for genomic coordinates.

[0169] As shown in Figures 7C and 7D, graphs 700c and 700d show logistic regression coefficients for genomic location classification models trained using ground truth classifications corresponding to either an intermediate deletion of 5-15 nucleobases in length (for graph 700c) or an intermediate insertion of 5-15 nucleobases in length (for graph 700d), respectively. Both graphs 700c and 700d show that the logistic regression models model a weighted mapping quality metric (MAPQ) that has the highest magnitude coefficient compared to other data inputs to determine a high or low confidence classification for a genomic coordinate or genomic region.

[0170] In particular, graph 700c in Figure 7C shows that a logistic regression model trained on intermediate deletions uses coefficients of magnitude approximately -0.8 and approximately 0.8 for the mapping quality metric to determine high and low confidence classifications for genomic coordinates, respectively. Similarly, graph 700d in Figure 7D shows that a logistic regression model trained on intermediate insertions uses coefficients of magnitude greater than -0.75 and greater than 0.75 for the mapping quality metric to determine high and low confidence classifications for genomic coordinates, respectively.

[0171] In contrast, graph 700c in Figure 7C shows that the logistic regression model trained for intermediate deletions uses coefficients of 0.0 that are smaller in magnitude than the other data inputs for intermediate deletions for both the binomial proportion test and the Bates distribution test to determine high and low confidence classifications for the genomic coordinates, respectively. Graph 700d in Figure 7D shows that the logistic regression model trained for intermediate insertions uses coefficients of 0.0 and near 0.0 that are smaller in magnitude than the other data inputs for intermediate insertions for the forward fraction metric and the higher negative insertion size metric to determine high and low confidence classifications for the genomic coordinates, respectively.

[0172] As shown in Figures 7E and 7F, graphs 700e and 700f show logistic regression coefficients for genomic location classification models trained using ground truth classifications corresponding to either long deletions greater than 15 nucleobases in length (for graph 700e) or long insertions greater than 15 nucleobases in length (for graph 700f), respectively. Figures 7E and 7F show that logistic regression models trained using a long deletion or long insertion weighted mapping quality metric (MAPQ) or depth clip metric determine a high or low confidence classification for a genomic coordinate or region with the highest magnitude coefficient compared to other data inputs.

[0173] In particular, graph 700e in Figure 7E shows that a logistic regression model trained on long deletions uses coefficients greater than -0.4 and greater than 0.4 for the mapping quality metric (MAPQ) to determine high and low confidence classifications for genomic coordinates or regions, respectively. Graph 700f in Figure 7F shows that a logistic regression model trained on long insertions uses coefficients greater than -0.4 and greater than 0.4 for the depth clip metric to determine high and low confidence classifications for genomic coordinates or regions, respectively.

[0174] In contrast, graph 700e in Figure 7E shows that the logistic regression model trained on the long deletion uses coefficients of 0.0 for both peak count and read position metrics, which are lower than other data inputs for the long deletion, to determine high and low confidence classifications for the genomic coordinates. Graph 700f in Figure 7F shows that the logistic regression model trained on the long insertion uses coefficients near 0.0 and 0.0, which are lower than other data inputs for the long insertion, for the local average of the read reference mismatch metric (local_mean_mismatch) and the binomial proportional test, to determine high and low confidence classifications for the genomic coordinates.

[0175] As shown in FIG. 7G, graph 700g shows the logistic regression coefficients of the genomic location classification model trained using the ground truth classification corresponding to the SNP. As shown in FIG. 7G, graph 700g shows that the logistic regression model trained on the SNP uses coefficients above -2.0 and above 2.0 higher than other data inputs for the SNP for the mapping quality metric (MAPQ) to determine high and low confidence classifications for the genomic coordinate or genomic region, respectively. In contrast, graph 700g shows that the logistic regression model trained on the SNP uses coefficients lower than other data inputs for the SNP for the deletion entropy metric to determine high and low confidence classifications for the genomic coordinate or genomic region.

[0176] To further evaluate the performance of the logistic regression model trained as a genome location classification model based on sequencing metrics, researchers determined the percentage of times such a genome location classification model correctly determines confidence classification. According to one or more embodiments, FIG. 8 shows a graph 800 with a receiver operating characteristic (ROC) curve that defines the area under the curve (AUC) for the percentage of times a logistic regression model trained as a genome location classification model (i) determines high or low confidence classifications in genome coordinates as true positive or false positive, and (ii) determines confidence classifications as true positive and false positive for genome coordinates with common deletions. As shown in FIG. 8, the genome classification system 106 inputs data derived or created from sequencing metrics into the genome location classification model to determine confidence classifications for genome coordinates.

[0177] As shown by graph 800, a logistic regression model trained as a genomic location classification model accurately determines a high confidence classification as true positive or false positive for a genomic coordinate with an AUC of 99.34% based on a comparison to a ground truth classification. As further shown by graph 800, such a genomic location classification model accurately determines a low confidence classification as true positive or false positive for a genomic coordinate with an AUC of 97.39% based on a comparison to a ground truth classification. Finally, such a genomic location classification model accurately determines a confidence classification as true positive or false positive for a genomic coordinate where a common deletion occurs with an AUC of 97.32% based on a comparison to a reference genome.

[0178] In addition to determining the ROC curve for the graph 800 shown in FIG. 8, the researchers also evaluated the precision, recall, and concordance (or reproducibility) with which a variant caller can identify SNVs and indels at genomic coordinates classified by a logistic regression model trained as a genomic location classification model. Various tests demonstrate that a logistic regression model trained as a genomic location classification model accurately classifies a larger portion of the human genome with high confidence coordinates (or regions) where SNVs and indels can be identified more accurately than those identified by GIAB. In fact, such a genomic location classification model can identify specific genomic coordinates (or regions) with high confidence classifications that GIAB identifies as being within difficult regions. For example, Table 2 below demonstrates that the genomic classification system 106 improves the precision with which existing sequencing systems identify the confidence with which a nucleic acid base can be determined at a specific genomic coordinate.

[0179] [Table 2]

[0180] As shown in Table 2, a logistic regression model trained as a genomic location classification model correctly classifies genomic coordinates in 90.3% of non-N autosomal human genomes. In contrast, GIAB identified genomic regions where variants could be accurately determined without difficulty in only 79-84% of non-N autosomal human genomes. As further shown by Table 2, such a logistic regression model accurately classifies genomic coordinates with approximately 99.9% precision, 99.9% recall, and 99.9% concordance based on ground truth classifications determined using SNV data. Similarly, such a logistic regression model accurately classifies genomic coordinates with approximately 99.0% precision, 99.5% recall, and 98.5% concordance based on ground truth classifications determined using indel data. For genomic coordinates labeled with medium or low confidence classifications by such logistic regression models, or genomic regions containing common deletions, such logistic regression models classify genomic coordinates based on ground truth data derived from SNVs or indels with lower precision, recall, and concordance, as further reported in Table 2.

[0181] To evaluate the performance of CNNs trained as genome location classification models based on context nucleic acid subsequences, researchers have determined the percentage of times such genome location classification models correctly determine confidence classifications. According to one or more embodiments, FIG. 9 shows a graph 900a with a ROC curve defining the AUC of a CNN trained as a genome location classification model that determines confidence classifications for genome coordinates based on ground truth classifications derived from indel data. FIG. 9 further shows a graph 900b with a ROC curve defining the AUC for a CNN trained as a genome location classification model that determines confidence classifications for genome coordinates based on ground truth classifications derived from data for single nucleotide polymorphisms (SNPs). As shown in FIG. 9, to determine confidence classifications for genome coordinates, the genome classification system 106 inputs data derived or prepared from context nucleic acid subsequences into the CNN trained as a genome location classification model.

[0182] In summary, graphs 900a and 900b demonstrate that a CNN trained as a genome location classification model accurately determines confidence classifications for genome coordinates as true positive or false positive based on ground truth data derived from indels or SNPs with AUCs of 77.9% to 91.7%, depending on the length of the context nucleic acid subsequence input to the genome location classification model. In particular, as shown by graph 900a, a genome location classification model trained for indels accurately determines confidence classifications for genome coordinates as true positive or false positive with AUCs of 81.4%, 87.4%, 87.6%, 88.2%, and 87.9%, respectively, based on context nucleic acid subsequences of 21 base pairs, 101 base pairs, 151 base pairs, 301 base pairs, and 801 base pairs. As shown by graph 900b, the genome location classification model trained for SNPs accurately determines confidence classifications for genome coordinates as true positive or false positive with AUCs of 77.9%, 88.8%, 90.0%, 91.2%, and 91.7% based on context nucleic acid subsequences of 21 base pairs, 101 base pairs, 151 base pairs, 301 base pairs, and 801 base pairs, respectively. Thus, for both indels and SNPs, the CNN trained as the genome location classification model more accurately determines confidence classifications for genome coordinates as the length of the context nucleic acid subsequence increases for the confidence classification.

[0183] To test the performance of the CNN trained as a genomic location classification model based on both sequencing metrics and context nucleic acid subsequences, the researchers also determined the rate at which such a genomic location classification model accurately determines a confidence classification using a test or holdout data set. According to one or more embodiments, Figures 10A and 10B show graphs 1002a-1002b, histograms 1004a-1004b, and confusion matrices 1006a-1006b illustrating the speed and confidence with which such a genomic location classification model accurately determines a confidence classification for a particular genomic coordinate based on ground truth classifications derived from indel and SNP data. As shown in Figures 10A and 10B, to determine a confidence classification for a genomic coordinate, the genomic classification system 106 inputs data derived (or created) from both sequencing metrics and context nucleic acid subsequences into the CNN trained as a genomic location classification model.

[0184] As shown by graph 1002a in FIG. 10A, the CNN trained on indels as a genomic location classification model accurately determines confidence classifications as true positive or false positive for genomic coordinates with an AUC of 97.8% based on a 101 base pair context nucleic acid subsequence. As shown by graph 1002b in FIG. 10B, the CNN trained on SNPs as a genomic location classification model accurately determines confidence classifications as true positive or false positive for genomic coordinates with an AUC of 99.7% based on a 101 base pair context nucleic acid subsequence. Thus, graphs 1002a and 1002b demonstrate that a CNN trained as a genomic location classification model as shown in FIGS. 10A and 10B can accurately determine confidence classifications for specific genomic coordinates at a very high rate when using both sequencing metrics and context nucleic acid subsequences as inputs.

[0185] Returning now to the histogram 1004a of FIG. 10A for indels. As shown by histogram 1004a, a CNN trained on indels as a genomic location classification model accurately determines a confidence classification as true positive in over 80,000 predictions at a confidence level of about 1.0 at genomic coordinates. In other words, based on a context nucleic acid subsequence of 101 base pairs, such a genomic location classification model determines a classification with high confidence at genomic coordinates where true positive indels are detected. As further shown by histogram 1004a, a CNN trained on indels as a genomic location classification model accurately determines a confidence classification as false positive in over 80,000 predictions at genomic coordinates at a confidence level of about 0.0. In other words, based on a context nucleic acid subsequence of 101 base pairs, such a genomic location classification model determines a classification with low confidence at genomic coordinates where false positive indels are detected.

[0186] Returning now to histogram 1004b of FIG. 10B for SNPs. As shown by histogram 1004b, the CNN trained on SNPs as the genomic location classification model accurately determines confidence classifications as true positive in nearly 800,000 predictions with a confidence of about 1.0 at the genomic coordinates. In other words, based on the 101 base pair context nucleic acid subsequence, the genomic location classification model determines classifications with high confidence at the genomic coordinates where the true positive SNPs are detected. As further shown by histogram 1004b, the CNN trained on SNPs as the genomic location classification model accurately determines confidence classifications as false positive in over 700,000 predictions with a confidence of about 0.0 at the genomic coordinates. In other words, based on the 101 base pair context nucleic acid subsequence, the genomic location classification model determines classifications with low confidence at the genomic coordinates where the false positive SNPs are detected.

[0187] Returning now to the confusion matrices 1006a and 1006b of Figures 10A and 10B, as shown by the confusion matrix 1006a of Figure 10A, a CNN trained on indels as a genomic location classification model correctly determines confidence classifications as true positive (e.g., high confidence classifications) or true negative (e.g., low confidence classifications) 92.322% of the time from all predictions at a genomic coordinate. In contrast, such a CNN sequencing system incorrectly determines confidence classifications as true positive or true negative only 7.678% of the time from all predictions at a genomic coordinate. As shown by the confusion matrix 1006b of Figure 10B, a CNN trained on SNPs as a genomic location classification model correctly determines confidence classifications as true positive or true negative 97.409% of the time from all predictions at a genomic coordinate. In contrast, such a CNN incorrectly determines confidence classifications as true positive or true negative only 2.591% of the time from all predictions at a genomic coordinate.

[0188] Referring now to FIG. 11A, this figure shows a flowchart of a series of acts 1100a for training a machine learning model to determine a confidence classification of a genomic coordinate, according to one or more embodiments. Although FIG. 11A illustrates acts according to one embodiment, alternative embodiments may omit, add, rearrange, and / or modify any of the acts shown in FIG. 11A. The acts of FIG. 11A may be performed as part of a method. Alternatively, a non-transitory computer-readable medium may include instructions that, when executed by one or more processors, cause a computing device to perform the acts set forth in FIG. 11A. In yet a further embodiment, a system includes at least one processor and a non-transitory computer-readable medium including instructions that, when executed by one or more processors, cause the system to perform the acts of FIG. 11A.

[0189] As shown in FIG. 11A, act 1100a includes act 1102 of determining one or more of a sequencing metric or a context nucleic acid subsequence. In particular, in some embodiments, act 1102 includes determining a sequencing metric for comparing the sample nucleic acid sequence to the genomic coordinates of the exemplary nucleic acid sequence. In some cases, act 1102 includes determining a context nucleic acid subsequence surrounding a variant-nucleobase call in the sample nucleic acid sequence at a genomic coordinate from the genomic coordinates of the reference genome from the exemplary nucleic acid sequence. In one or more embodiments, the sample nucleic acid sequence is determined using a single sequencing pipeline that includes a nucleic acid sequence extraction method, a sequencing device, and sequence analysis software. Relatedly, in certain embodiments, the exemplary nucleic acid sequence includes a reference genome or nucleic acid sequence of an ancestral haplotype.

[0190] As described above, in some cases, determining the sequencing metrics includes determining one or more of an alignment metric to quantify the alignment of the sample nucleic acid sequence with the genomic coordinates of the exemplary nucleic acid sequence, a depth metric to quantify the depth of the nucleic acid base calls for the sample nucleic acid sequence at the genomic coordinates of the exemplary nucleic acid sequence, or a call data quality metric to quantify the quality of the nucleic acid base calls for the sample nucleic acid sequence at the genomic coordinates of the exemplary nucleic acid sequence.

[0191] Relatedly, in certain implementations, determining the alignment metric includes determining one or more of a deletion size metric, a mapping quality metric, a positive insertion size metric, a negative insertion size metric, a soft clipping metric, a read position metric, or a read reference mismatch metric for the sample nucleic acid sequence, determining the depth metric includes determining one or more of a forward-reverse depth metric or a normalized depth metric, or determining the call data quality metric includes determining one or more of a nucleobase call quality metric or a callability metric for the sample nucleic acid sequence.

[0192] 11A, act 1100a includes act 1104 of training a genome location classification model to determine a confidence classification for the genome coordinate based on one or more of the sequencing metrics or the context nucleic acid subsequence. In particular, in some embodiments, act 1104 includes training a genome location classification model to determine a confidence classification for the genome coordinate based on the sequencing metrics and ground truth classification for the particular genome coordinate. Further, in some cases, act 1104 includes training a genome location classification model to determine a confidence classification for the genome coordinate based on the context nucleic acid subsequence and the ground truth classification for the genome coordinate.

[0193] As alluded to above, in certain embodiments, training the genome location classification model to determine the confidence classification comprises training a statistical machine learning model or a neural network to determine the confidence classification. Relatedly, in one or more embodiments, training the genome location classification model to determine the confidence classification comprises training a logistic regression model, a random forest classifier, or a convolutional neural network to determine the confidence classification.

[0194] Further, in some circumstances, the confidence classification indicates the degree to which a nucleobase can be accurately determined at a particular genomic coordinate. Relatedly, in some cases, determining the confidence classification includes determining a confidence classification for a single nucleotide variant, a nucleobase insertion, a nucleobase deletion, a portion of a structural variation, or a portion of a copy number variation in the genomic coordinate.

[0195] As further alluded to above, in one or more embodiments, training the genomic location classification model to determine a confidence classification includes, for the genomic coordinate, comparing the predicted confidence classification to a ground truth classification that reflects a Mendelian inheritance pattern or replicate concordance of the nucleic acid base calls at the genomic coordinate, determining a loss from the comparison of the predicted confidence classification to the ground truth classification, and adjusting parameters of the genomic location classification model based on the determined loss.

[0196] 11A , act 1100a includes act 1106 of determining a set of confidence classifications for the set of genomic coordinates. In particular, in certain implementations, act 1106 includes utilizing a genomic location classification model to determine a set of confidence classifications for the set of genomic coordinates based on a set of sequencing metrics for one or more sample nucleic acid sequences. In some cases, act 1106 includes utilizing a genomic location classification model to determine confidence classifications for the genomic coordinates based on the context nucleic acid subsequences.

[0197] For example, in one or more implementations, determining a belief classification from the set of belief classifications includes determining a belief classification for a genomic coordinate that includes a genetic or epigenetic modification. Relatedly, in some embodiments, determining a belief classification from the set of belief classifications includes determining a belief classification for a portion of a single nucleotide variant, a nucleobase insertion, a nucleobase deletion, or a structural change in the genomic coordinate.

[0198] Further, in some circumstances, determining a confidence classification from the set of confidence classifications includes determining at least one of a high, a medium, or a low confidence classification for the genomic coordinate. Additionally, or alternatively, determining a confidence classification from the set of confidence classifications includes determining a confidence score within a range of confidence scores that indicates the degree to which the nucleobase may be accurately determined at the genomic coordinate.

[0199] 11A, act 1100a includes act 1108 of generating at least one digital file including the set of confidence classifications. In particular, in certain implementations, act 1108 includes generating at least one digital file including the set of confidence classifications for the set of genomic coordinates. Similarly, in some embodiments, act 1108 includes generating a digital file including confidence classifications for the genomic coordinates of the variant-nucleobase calls.

[0200] In addition to acts 1102-1108, in a particular implementation, act 1100a includes determining a context nucleic acid subsequence surrounding the variant-nucleobase call from an example nucleic acid sequence, and training a genomic location classification model to determine a confidence classification for the genomic coordinate of the variant-nucleobase call based on the context nucleic acid subsequence, a subset of the sequencing metrics for the subset of genomic coordinates corresponding to the context nucleic acid subsequence, and a subset of the ground truth classifications for the subset of genomic coordinates corresponding to the context nucleic acid subsequence.

[0201] Referring now to FIG. 11B, this figure shows a flowchart of a series of acts 1100b for training a machine learning model to determine variant confidence classifications for genomic coordinates, according to one or more embodiments. Although FIG. 11B illustrates acts according to one embodiment, alternative embodiments may omit, add, rearrange, and / or modify any of the acts shown in FIG. 11B. The acts of FIG. 11B may be performed as part of a method. Alternatively, a non-transitory computer-readable medium may include instructions that, when executed by one or more processors, cause a computing device to perform the acts set forth in FIG. 11B. In yet a further embodiment, a system includes at least one processor and a non-transitory computer-readable medium including instructions that, when executed by one or more processors, cause the system to perform the acts of FIG. 11B.

[0202] As shown in Figure 11B, act 1100b includes act 1110 of determining sequencing metrics of sample nucleic acid sequences from the mixture of genomic samples. In particular, in some embodiments, act 1110 includes determining sequencing metrics for comparing sample nucleic acid sequences from the genomic sample to genomic coordinates of exemplary nucleic acid sequences. For example, in some cases, determining sequencing metrics includes determining mapping quality metrics, forward-reverse depth metrics, and nucleic acid base call quality metrics for the sample nucleic acid sequences. In one or more embodiments, the sample nucleic acid sequences are determined using a single sequencing pipeline that includes a nucleic acid sequence extraction method, a sequencing device, and sequence analysis software.

[0203] As further shown in FIG. 11B, act 1100b includes act 1112 of generating, for a variant-nucleobase call, a ground truth classification for the genomic coordinate based on one or more of the sequencing metrics. For example, act 1112 can include generating, for a particular variant-nucleobase call, a ground truth classification for the particular genomic coordinate based on one or more of the sequencing metrics or variant-call data for the mixture of genomic samples. As a further example, act 1112 can include generating a ground truth classification based on one or more of the sequencing metrics including a mapping quality metric, a forward-reverse depth metric, and a nucleobase call quality metric for the sample nucleic acid sequence.

[0204] As alluded to above, in certain embodiments, for a particular variant-nucleobase call, generating a ground truth classification for a particular genomic coordinate based on variant call data for the mixture of genomic samples includes determining one or more precisions or recalls for determining a set of variant-nucleobase calls for one or more sample nucleic acid sequences from the mixture of genomic samples at the particular genomic coordinate, and generating the ground truth classification based on the one or more precisions or recalls for determining the set of variant-nucleobase calls. Further, in some implementations, for a particular variant-nucleobase call, generating a ground truth classification for a particular genomic coordinate based on the variant call data for the mixture of genomic samples includes determining variant-allele frequencies of the set of variant-nucleobase calls for one or more sample nucleic acid sequences from the mixture of genomic samples, determining one or more of precision or recall for determining different variant-nucleobase calls for the one or more sample nucleic acid sequences from the mixture of genomic samples at the particular genomic coordinate and at different variant-allele frequencies from the variant-allele frequencies, and generating the ground truth classification based on the one or more of precision or recall for determining the different variant-nucleobase calls at the different variant-allele frequencies.

[0205] Relatedly, in some cases, for a particular variant-nucleobase call, generating a ground truth classification for a particular genomic coordinate based on variant-call data for a mixture of genomic samples includes determining a somatic quality metric for the nucleobase call from one or more sample nucleic acid sequences from the mixture of genomic samples, generating a somatic quality metric threshold for distinguishing between different ground truth classifications for the particular genomic coordinate, and generating a hierarchical ground truth classification for the particular genomic coordinate according to the somatic quality metric threshold. In some such cases, generating the hierarchical ground truth classification includes generating only a subset of the hierarchical ground truth classification according to the somatic quality metric threshold.

[0206] Further, in some embodiments, for a particular variant-nucleobase call, generating a ground truth classification for a particular genomic coordinate based on the variant call data for the mixture of genomic samples includes determining variant-allele frequencies of the set of variant-nucleobase calls for one or more sample nucleic acid sequences from the mixture of genomic samples, determining precision and recall for determining the set of variant-nucleobase calls for one or more sample nucleic acid sequences from the mixture of genomic samples at the particular genomic coordinate and at a variant-allele frequency different from the variant-allele frequency, determining an F-score for determining the different variant-nucleobase calls at the particular genomic coordinate based on the precision and recall, and generating the ground truth classification further based on the F-score for determining the different variant-nucleobase calls.

[0207] In addition to acts 1110 and 1112, in some embodiments, act 1100b further includes determining a context nucleic acid subsequence surrounding the variant-nucleobase call in the one or more sample nucleic acid sequences at one or more genomic coordinates from the one or more exemplary nucleic acid sequences. In certain implementations, the one or more exemplary nucleic acid sequences include a reference genome or nucleic acid sequence of an ancestral haplotype.

[0208] As further shown in Figure 11B, act 1100b includes act 1114 of training a genome location classification model to determine a variant confidence classification for the genome coordinate based on the ground truth classification. In particular, in some embodiments, act 1114 includes training a genome location classification model to determine a variant confidence classification for the genome coordinate based on the sequencing metrics and the ground truth classification for the variant-nucleobase call. Further, in some cases, act 1114 includes training a genome location classification model to determine a variant confidence classification for the genome coordinate based on the context nucleic acid subsequence and the ground truth classification for the variant-nucleobase call.

[0209] As alluded to above, in certain embodiments, the variant confidence classification indicates the degree to which somatic nucleobase variants reflecting cancer or somatic mosaicism can be accurately determined in genomic coordinates. In contrast, in some cases, the variant confidence classification indicates the degree to which germline nucleobase variants reflecting germline mosaicism can be accurately determined in genomic coordinates.

[0210] As further shown in FIG. 11B, act 1100b includes act 1116 of determining a set of variant confidence classifications for the set of genomic coordinates. In particular, in certain implementations, act 1116 includes utilizing a genomic location classification model to determine a set of variant confidence classifications for the set of genomic coordinates based on a set of sequencing metrics for one or more sample nucleic acid sequences. In some cases, act 1116 includes utilizing a genomic location classification model to determine a set of variant confidence classifications for the set of genomic coordinates based on a set of context nucleic acid subsequences that surround a corresponding set of variant-nucleobase calls. For example, determining the set of sequencing metrics may include determining a set of sequencing metrics for one or more sample nucleic acid sequences from one or more genomic samples.

[0211] As a further example, in some cases, act 1116 includes determining a variant confidence classification from the set of variant confidence classifications by determining a variant confidence classification for the genomic coordinate based on a context nucleic acid subsequence surrounding a somatic nucleobase variant that reflects cancer or somatic mosaicism. In contrast, in certain cases, act 1116 includes determining a variant confidence classification from the set of variant confidence classifications by determining a variant confidence classification for the genomic coordinate based on a context nucleic acid subsequence surrounding a germline nucleobase variant that reflects germline mosaicism. Furthermore, in one or more embodiments, act 1116 includes determining a variant confidence classification from the set of variant confidence classifications by determining a variant confidence score within a range of variant confidence scores that is indicative of the degree to which the nucleobase variant may be accurately determined at the genomic coordinate.

[0212] In addition to acts 1110-1116, in certain implementations, act 1100b includes determining an admixture of the genomic sample by determining a combination of a first subset of nucleic acid sequences from the first genomic sample and a second subset of nucleic acid sequences from the second genomic sample that together simulate a variant-allele frequency of a genomic sample having cancer or mosaicism. Similarly, in some cases, act 1100b includes determining an admixture of the genomic sample by determining a combination of a first percentage of nucleic acid sequences from the first naturally occurring genomic sample and a second percentage of nucleic acid sequences from the second naturally occurring genomic sample that together simulate a variant-allele frequency of a genomic sample having cancer or mosaicism.

[0213] Referring now to FIG. 12, this figure shows a flow chart of a series of acts 1200 for generating an indicator of confidence classification for genomic coordinates of variant-nucleobase calls from a digital file, according to one or more embodiments. Although FIG. 12 illustrates acts according to one embodiment, alternative embodiments may omit, add, rearrange, and / or modify any of the acts shown in FIG. 12. The acts of FIG. 12 may be performed as part of a method. Alternatively, a non-transitory computer-readable medium may include instructions that, when executed by one or more processors, cause a computing device to perform the acts set forth in FIG. 12. In yet a further embodiment, a system includes at least one processor and a non-transitory computer-readable medium including instructions that, when executed by one or more processors, may cause the system to perform the acts of FIG. 12.

[0214] As shown in Figure 12, act 1200 includes act 1202 of detecting a variant-nucleobase call at a genomic coordinate. In particular, in some embodiments, act 1202 includes detecting a variant-nucleobase call at a genomic coordinate within a sample nucleic acid sequence. As noted above, in some cases, detecting a variant-nucleobase call at a genomic coordinate includes detecting a single nucleotide variant, a nucleobase insertion, a nucleobase deletion, or a portion of a structural change.

[0215] 12, act 1200 includes act 1204 of identifying a confidence classification of the genomic coordinate according to the genomic location classification model. In particular, in some embodiments, act 1204 includes identifying, from the digital file, a confidence classification of the genomic coordinate according to the genomic location classification model.

[0216] As alluded to above, in certain embodiments, identifying a confidence classification for the genomic coordinate comprises identifying from the digital file a confidence classification indicative of the degree to which the nucleobase can be accurately determined at the genomic coordinate. Further, in some implementations, identifying a confidence classification from the digital file comprises identifying a confidence classification from an annotation or score for the genomic coordinate in the digital file. Relatedly, in one or more embodiments, identifying a confidence classification from the digital file comprises identifying at least one of a high confidence classification, a medium confidence classification, or a low confidence classification for the genomic coordinate.

[0217] 12, act 1200 includes an act of generating an indicator for a confidence classification 1206. In particular, in certain implementations, act 1206 includes generating an indicator of a confidence classification for the genomic coordinates of the variant-nucleobase call for display within a graphical user interface.

[0218] The methods described herein can be used in conjunction with a variety of nucleic acid sequencing techniques. Particularly applicable techniques are those in which the nucleic acids are attached to fixed positions within an array such that their relative positions do not change, and the array is imaged repeatedly. For example, embodiments in which images are obtained in different color channels that correspond to different labels used to distinguish one nucleotide base type from another are particularly applicable. In some embodiments, the process of determining the nucleotide sequence of the target nucleic acid can be an automated process. A preferred embodiment includes sequencing-by-synthesis ("SBS") techniques.

[0219] SBS technology generally involves the enzymatic extension of nascent nucleic acid strand by repeated addition of nucleotide to template strand.In the conventional method of SBS, a single nucleotide monomer can be provided to target nucleic acid in the presence of polymerase in each delivery.However, in the method described herein, multiple types of nucleotide monomers can be provided to target nucleic acid in the presence of polymerase during delivery.

[0220] SBS can utilize nucleotide monomers with terminator moieties or nucleotide monomers that lack any terminator moieties. Methods that utilize nucleotide monomers that lack terminators include, for example, pyrosequencing and sequencing using γ-phosphate-labeled nucleotides, as described in more detail below. In methods that use nucleotide monomers that do not contain terminators, the number of nucleotides added in each cycle is generally variable and depends on the template sequence and the mode of nucleotide delivery. In SBS techniques that utilize nucleotide monomers with terminator moieties, the terminator can be effectively irreversible under the sequencing conditions used, as in the case of conventional Sanger sequencing that utilizes dideoxynucleotides, or the terminator can be reversible, as in the case of the sequencing method developed by Solexa (now Illumina).

[0221] SBS techniques can use nucleotide monomers with or without a label moiety. Thus, incorporation events can be detected based on the properties of the label, such as the fluorescence of the label, the properties of the nucleotide monomer, such as the molecular weight or charge, the by-products of incorporation of the nucleotide, such as the release of pyrophosphate, and the like. In embodiments in which two or more different nucleotides are present in the sequencing reagent, the different nucleotides can be distinguishable from each other, or alternatively, the two or more different labels can be distinguishable under the detection technique used. For example, the different nucleotides present in the sequencing reagent can have different labels, which can be distinguished using appropriate optical systems, as exemplified by the sequencing method developed by Solexa (now Illumina).

[0222] Preferred embodiments include the technique of pyrosequencing, which detects the release of inorganic pyrophosphate (PPi) when a specific nucleotide is incorporated into a nascent strand (Ronaghi, M., Karamohamed, S., Pettersson, B., Uhlen, M. and Nyren, P. (1996) "Real-time DNA sequencing using detection of pyrophosphate release." Analytical Biochemistry 242(1), 84-9; Ronaghi, M. (2001) "Pyrosequencing sheds light on DNA sequencing." Genome Res., 11(1), 3-11; Ronaghi, M., Uhlen, M. and Nyren, P. (1998) "Real-time inorganic pyrophosphate-based sequencing." Science 281(5375),363, U.S. Patent Nos. 6,210,891, 6,258,568, and 6,274,320, the disclosures of which are incorporated herein by reference in their entirety). In pyrosequencing, the released PPi can be detected by its immediate conversion to adenosine triphosphate (ATP) by ATP sulfurase, and the level of ATP generated is detected via luciferase-generated photons. The nucleic acids to be sequenced can be attached to features in an array, and the array can be imaged to capture chemiluminescent signals generated by incorporation of nucleotides into the features of the array. Images can be obtained after treatment of the array with a particular nucleotide type (e.g., T, C, or G). Images obtained after addition of each nucleotide type differ with respect to which features in the array are detected. These differences in the images reflect the different sequence content of the features on the array. However, the relative position of each feature remains unchanged in the image. The images can be stored, processed, and analyzed using the methods described herein.For example, images obtained after treating the array with each different nucleotide type can be processed in the same manner as exemplified herein for images obtained from different detection channels for reversible terminator-based sequencing methods.

[0223] In another exemplary type of SBS, cycle sequencing is accomplished by stepwise addition of reversible terminator nucleotides containing cleavable or photobleachable dye labels, for example as described in WO 04 / 018497 and U.S. Pat. No. 7,057,026, the disclosures of which are incorporated by reference. This approach has been commercialized by Solexa (now Illumina Inc.) and is also described in WO 91 / 06678 and WO 07 / 123,744, each of which is incorporated by reference herein. The availability of fluorescently labeled terminators, both of which can be reversed and from which the fluorescent labels are cleaved, facilitates efficient cyclic reversible termination (CRT) sequencing. Polymerases can also be co-engineered to efficiently incorporate and extend from these modified nucleotides.

[0224] Preferably, in reversible terminator-based sequencing embodiments, the label does not substantially inhibit extension under SBS reaction conditions. However, the detection label may be removable, for example, by cleavage or degradation. Images can be taken after incorporation of the label into the arrayed nucleic acid features. In certain embodiments, each cycle involves simultaneous delivery of four different nucleotide types to the array, each nucleotide type having a spectrally distinct label. Four images can then be obtained, each using a detection channel selective for one of the four different labels. Alternatively, different nucleotide types can be added sequentially, and images of the array can be obtained during each addition step. In such embodiments, each image shows nucleic acid features that incorporate a particular type of nucleotide. Different features are present or absent in different images, since the sequence content of each feature is different. However, the relative positions of the features remain unchanged within the images. Images obtained from such reversible terminator-SBS methods can be stored, processed, and analyzed as described herein. Following the image taking step, the label can be removed, and the reversible terminator moiety can be removed for subsequent cycles of nucleotide addition and detection. Removal of the label after detection in a particular cycle and before the subsequent cycle has the advantage of reducing background signal and crosstalk between cycles. Examples of useful labeling and removal methods are described below.

[0225] In certain embodiments, some or all of the nucleotide monomers can include reversible terminators. In such embodiments, the reversible terminator / cleavable fluorophore can include a fluorophore attached to the ribose moiety via a 3' ester bond (Metzker, Genome Res. 15:1767-1776 (2005), which is incorporated herein by reference). Other approaches separate the terminator chemistry from the cleavage of the fluorescent label (Ruparel et al., Proc Natl Acad Sci USA 102:5932-7 (2005), which is incorporated herein by reference in its entirety). Ruparel et al. describe the development of reversible terminators that use a small amount of 3' allyl group to block extension, but can be easily deblocked by brief treatment with a palladium catalyst. The fluorophore was attached to the group via a photocleavable linker that can be easily cleaved by 30 seconds of exposure to long wavelength UV light. Thus, either disulfide reduction or photocleavage can be used as the cleavable linker. Another approach to reversible termination is the use of a natural terminus followed by placement of a bulky dye on the dNTP. The presence of a charged bulky dye on the dNTP can act as an effective terminator through steric and / or electrostatic hindrance. The presence of one incorporation event prevents further binding unless the dye is removed. Cleavage of the dye removes the fluor, effectively reversing the terminus. Examples of modified nucleotides are also described in U.S. Pat. Nos. 7,427,673 and 7,057,026, the disclosures of which are incorporated herein by reference in their entireties.

[0226] Additional exemplary SBS systems and methods that may be utilized with the methods and systems described herein are described in U.S. Patent Application Publication No. 2007 / 0166705, U.S. Patent Application Publication No. 2006 / 0188901, U.S. Pat. No. 7,057,026, U.S. Patent Application Publication No. 2006 / 0240439, U.S. Patent Application Publication No. 2006 / 0281109, WO 05 / 065814, U.S. Patent Application Publication No. 2005 / 0100900, WO 06 / 064199, WO 07 / 010,251, U.S. Patent Application Publication No. 2012 / 0270305, and U.S. Patent Application Publication No. 2013 / 0260372, the disclosures of which are incorporated herein by reference in their entireties.

[0227] Some embodiments may utilize detection of four different nucleotides using fewer than four different labels. For example, SBS may be performed using the methods and systems described in incorporated document US Patent Application Publication No. 2013 / 0079232. As a first example, pairs of nucleotide types may be detected at the same wavelength but may be distinguished based on differences in intensity for one member of the pair, or based on a change to one member of the pair (e.g., via making a chemical modification, photochemical modification, or physical modification) that results in the appearance or disappearance of a distinct signal compared to the signal detected for the other member of the pair. As a second example, three of the four different nucleotide types may be detected under certain conditions, while the fourth nucleotide type may have no detectable label under those conditions or may be minimally detected under those conditions (e.g., minimal detection due to background fluorescence, etc.). Incorporation of the first three nucleotide types into a nucleic acid may be determined based on the presence of their corresponding signals, and incorporation of the fourth nucleotide type into a nucleic acid may be determined based on the absence or minimal detection of any signal. As a third example, one nucleotide type may contain a label that is detected in two different channels, while the other nucleotide type is detected in one or less of the channels. The three exemplary configurations above are not considered mutually exclusive and may be used in various combinations. An exemplary embodiment that combines all three examples is a fluorescence-based SBS method that uses a first nucleotide type (e.g., dATP with a label that is detected in the first channel when excited by a first excitation wavelength), a second nucleotide type (e.g., dCTP with a label that is detected in the second channel when excited by a second excitation wavelength), a third nucleotide type (e.g., dTTP with at least one label that is detected in both channels when excited by the first and / or second excitation wavelength), and a fourth nucleotide type (e.g., unlabeled dGTP) that is not detected in any channel or that lacks a label that is minimally detected.

[0228] Furthermore, as described in incorporated document U.S. Patent Application Publication No. 2013 / 0079232, sequencing data can be obtained using a single channel. In such so-called one-dye sequencing methods, a first nucleotide type is labeled but the label is removed after the first image is generated, and a second nucleotide type is labeled only after the first image is generated. A third nucleotide type retains its label in both the first and second images, and a fourth nucleotide type remains unlabeled in both images.

[0229] Some embodiments may utilize sequencing by ligation techniques. Such techniques utilize DNA ligase to incorporate oligonucleotides and identify the incorporation of such oligonucleotides. The oligonucleotides typically have different labels that correlate with the identity of a particular nucleotide in the sequence to which the oligonucleotide hybridizes. As with other SBS methods, images can be obtained after treating an array of nucleic acid sequences with labeled sequencing reagents. Each image shows nucleic acid features that incorporate a particular type of label. Because the sequence content of each feature is different, different features are present or absent in different images, but the relative positions of the features remain unchanged within the images. Images obtained from ligation-based sequencing methods can be stored, processed, and analyzed as described herein. Exemplary SBS systems and methods that can be utilized with the methods and systems described herein are described in U.S. Pat. No. 6,969,488, U.S. Pat. No. 6,172,218, and U.S. Pat. No. 6,306,597, the disclosures of which are incorporated herein by reference in their entireties.

[0230] Some embodiments may utilize nanopore sequencing (Deamer, DW & Akeson, M. "Nanopores and nucleic acids: prospects for ultrarapid sequencing." Trends Biotechnol. 18, 147-151 (2000); Deamer, D. and D. Branton, "Characterization of nucleic acids by nanopore analysis." Acc. Chem. Res. 35:817-825 (2002); Li, J., M. Gershow, D. Stein, E. Brandin, and JA Golovchenko, "DNA molecules and configurations in a solid-state nanopore microscope." Nat. Mater. 2:611-615 (2003), the disclosures of which are incorporated herein by reference in their entireties). In such embodiments, the target nucleic acid passes through the nanopore. The nanopore may be a synthetic pore or a biological membrane protein, such as α-hemolysin. As the target nucleic acid passes through the nanopore, each base pair can be identified by measuring the fluctuations in the electrical conductance of the pore. (U.S. Pat. No. 7,001,792; Soni, GV & Meller, "A. Progress toward ultrafast DNA sequencing using solid-state nanopores." Clin. Chem. 53, 1996-2001 (2007); Healy, K. "Nanopore-based single-molecule DNA analysis." Nanomed. 2, 459-481 (2007); Cockroft, SL, Chu, J., Amorin, M. & Ghadiri, MR "A single-molecule nanopore device detects DNA polymerase activity with single-nucleotide resolution." J. Am Chem. Soc. 130, 818-820 (2008), the disclosures of which are incorporated herein by reference in their entireties).Data obtained from nanopore sequencing can be stored, processed, and analyzed as described herein. In particular, the data can be processed as images according to the exemplary processing of optical and other images described herein.

[0231] Some embodiments may utilize methods involving real-time monitoring of DNA polymerase activity. Nucleotide incorporation may be detected via fluorescence resonance energy transfer (FRET) interactions between fluorophore-containing polymerases and γ-phosphate-labeled nucleotides, for example, as described in U.S. Pat. No. 7,329,492 and U.S. Pat. No. 7,211,414, each of which is incorporated herein by reference, or nucleotide incorporation may be detected using zero-mode waveguides, for example, as described in U.S. Pat. No. 7,315,019, each of which is incorporated herein by reference, and fluorescent nucleotide analogs and engineered polymerases, for example, as described in U.S. Pat. No. 7,405,281 and U.S. Patent Application Publication No. 2008 / 0108082, each of which is incorporated herein by reference. Illumination can be restricted to a zeptoliter-scale volume around the surface-tethered polymerase so that incorporation of fluorescently labeled nucleotides can be observed with low background (Levene, MJ et al. "Zero-mode waveguides for single-molecule analysis at high concentrations." Science, 299, 682-686 (2003); Lundquist, PM et al. "Parallel confocal detection of single molecules in real time." Opt. Lett. 33, 1026-1028 (2008); Korlach, J. et al. "Selective aluminum passivation for targeted immobilization of single DNA polymerase molecules in zero-mode waveguide nano structures." Proc. Natl. Acad. Sci. USA 105, 1176-1181 (2008), the disclosures of which are incorporated herein by reference in their entireties). Images obtained from such methods can be stored, processed, and analyzed as described herein.

[0232] Some SBS embodiments include detection of protons released upon incorporation of a nucleotide into an extension product. For example, sequencing based on detection of released protons can be performed using electrical detectors and related technology available from Ion Torrent (Guilford, CT, a subsidiary of Life Technologies), or the sequencing methods and systems described in U.S. Patent Application Publication Nos. 2009 / 0026082(A1), 2009 / 0127589(A1), 2010 / 0137143(A1), or 2010 / 0282617(A1), each of which is incorporated herein by reference. The methods described herein for amplifying target nucleic acids using kinetic exclusion can be easily adapted to substrates used to detect protons. More specifically, the methods described herein can be used to generate clonal populations of amplicons used to detect protons.

[0233] The SBS method described above can be advantageously performed in a multiplex format, such that multiple different target nucleic acids are manipulated simultaneously. In certain embodiments, the different target nucleic acids can be processed in a common reaction vessel or on the surface of a particular substrate. This allows for convenient delivery of sequencing reagents, removal of unreacted reagents, and detection of incorporation events in a multiplexed manner. In embodiments using surface-bound target nucleic acids, the target nucleic acids can be in an array format. In an array format, the target nucleic acids can typically be bound to a surface in a spatially distinguishable manner. The target nucleic acids can be bound by direct covalent attachment, attachment to beads or other particles, or binding to a polymerase or other molecule attached to the surface. The array can include a single copy of the target nucleic acid at each site (also referred to as a feature), or multiple copies with the same sequence can be present at each site or feature. The multiple copies can be generated by amplification methods such as bridge amplification or emulsion PCR, which are described in more detail below.

[0234] The methods described herein can be used to detect, for example, at least about 10 features / cm 2, 100 features / cm 2 , 500 features / cm 2 , 1,000 features / cm 2 , 5,000 features / cm 2 , 10,000 features / cm 2 , 50,000 features / cm 2 , 100,000 features / cm 2 , 1,000,000 features / cm 2 , 5,000,000 features / cm 2 Arrays having any of a variety of densities of features, including 100, 150, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 1100, 1200, 1400, 1500, 1600, 1700, 1800, 1900, 2000,

[0235] An advantage of the methods described herein is that they provide rapid and efficient detection of multiple target nucleic acids in parallel. Thus, the present disclosure provides an integrated system that can prepare and detect nucleic acids using techniques known in the art, such as those exemplified above. Thus, the integrated system of the present disclosure can include fluidic components that can deliver amplification and / or sequencing reagents to one or more immobilized DNA fragments, the system including components such as pumps, valves, reservoirs, fluid lines, etc. A flow cell can be configured and / or used in the integrated system for detecting target nucleic acids. Exemplary flow cells are described, for example, in U.S. Patent No. 2010 / 0111768(A1) and U.S. Patent Application No. 13 / 273,666, each of which is incorporated herein by reference. As exemplified for the flow cell, one or more of the fluidic components of the integrated system can be used for amplification and detection methods. Taking the nucleic acid sequencing embodiment as an example, one or more of the fluidic components of the integrated system can be used for delivery of sequencing reagents in the amplification methods described herein and in the sequencing methods as exemplified above. Alternatively, an integrated system may include separate fluidic systems for performing the amplification method and for performing the detection method. Examples of integrated sequencing systems capable of producing amplified nucleic acids and sequencing the nucleic acids include, but are not limited to, the MiSeq™ platform (Illumina Inc., San Diego, Calif.) and the devices described in U.S. Patent Application No. 13 / 273,666, which is incorporated herein by reference.

[0236] The sequencing system described above sequences the nucleic acid polymers present in the sample received by the sequencing device. As defined herein, "sample" and its derivatives are used in the broadest sense and include any sample, culture, etc. suspected of containing a target. In some embodiments, the sample includes DNA, RNA, PNA, LNA, chimeric or hybrid forms of nucleic acid. A sample can include any biological, clinical, surgical, agricultural, air or water sample containing one or more nucleic acids. The term also includes any isolated nucleic acid sample, such as genomic DNA, fresh frozen or formalin-fixed paraffin-embedded nucleic acid samples. It is also envisioned that the sample can be derived from a single individual, a collection of nucleic acid samples from genetically related members, nucleic acid samples from genetically unrelated members, nucleic acid samples from a single individual such as a tumor sample and a normal tissue sample (matched), or a sample from a single source containing two different forms of genetic material such as maternal and fetal DNA obtained from a maternal subject, or the presence of contaminating bacterial DNA in a sample containing plant or animal DNA. In some embodiments, the source of nucleic acid material can include nucleic acid obtained from a newborn, for example, as typically used for newborn screening.

[0237] The nucleic acid sample can include high molecular weight material such as genomic DNA (gDNA). The sample can include low molecular weight material such as nucleic acid molecules obtained from FFPE or archived DNA samples. In another embodiment, the low molecular weight material includes enzymatically or mechanically fragmented DNA. The sample can include cell-free circulating DNA. In some embodiments, the sample can include nucleic acid molecules obtained from biopsies, tumors, scrapings, swabs, blood, mucus, urine, plasma, semen, hair, laser capture microdissection, surgical resection, and other clinical or laboratory obtained samples. In some embodiments, the sample can be an epidemiological, agricultural, forensic, or pathogenic sample. In some embodiments, the sample can include nucleic acid molecules obtained from animals, such as human or mammalian sources. In another embodiment, the sample can include nucleic acid molecules obtained from non-mammalian sources, such as plants, bacteria, viruses, or fungi. In some embodiments, the source of the nucleic acid molecule can be an archived or extinct sample or species.

[0238] Additionally, the methods and compositions disclosed herein may be useful for amplifying nucleic acid samples having low quality nucleic acid molecules, such as degraded and / or fragmented genomic DNA from forensic samples. In one embodiment, the forensic sample may include nucleic acid obtained from a crime scene, from a missing persons DNA database, from a laboratory associated with a forensic investigation, or may include forensic samples obtained by a law enforcement agency, one or more military services, or members thereof. The nucleic acid sample may be crude DNA, including purified samples or lysates, for example, from buccal swabs, paper, cloth, or other substrates that may be impregnated with saliva, blood, or other bodily fluids. Thus, in some embodiments, the nucleic acid sample may include small amounts of DNA, such as genomic DNA, or fragmented portions of DNA. In some embodiments, the target sequence may be present in one or more bodily fluids, including, but not limited to, blood, sputum, plasma, semen, urine, and serum. In some embodiments, the target sequence may be obtained from hair, skin, tissue samples, autopsies, or remains of victims. In some embodiments, the nucleic acid including one or more target sequences may be obtained from deceased animals or humans. In some embodiments, the target sequence may comprise nucleic acid obtained from non-human DNA, such as microbial, plant or entomological DNA. In some embodiments, the target sequence or the amplified target sequence is for human identification. In some embodiments, the disclosure generally relates to a method for identifying features of a forensic sample. In some embodiments, the disclosure generally relates to a human identification method using one or more target specific primers disclosed herein or one or more target specific primers designed using the primer design criteria outlined herein. In one embodiment, a forensic sample or human identification sample comprising at least one target sequence can be amplified using any one or more of the target specific primers disclosed herein or using the primer criteria outlined herein.

[0239] The components of the genome classification system 106 may include software, hardware, or both. For example, the components of the genome classification system 106 may include one or more instructions stored on a computer-readable storage medium and executable by a processor of one or more computing devices (e.g., user client device 108). When executed by one or more processors, the computer-executable instructions of the genome classification system 106 may cause the computing device to perform the bubble detection methods described herein. Alternatively, the components of the genome classification system 106 may include hardware, such as a dedicated processing device for performing a particular function or group of functions. Additionally or alternatively, the components of the genome classification system 106 may include a combination of computer-executable instructions and hardware.

[0240] Further, the components of the genome classification system 106 that perform the functions described herein with respect to the genome classification system 106 may be implemented, for example, as part of a stand-alone application, as a module of an application, as a plug-in of an application, as a library function(s) that may be called by other applications, and / or as a cloud computing model. Thus, the components of the genome classification system 106 may be implemented as part of a stand-alone application on a personal computing device or a mobile device. Additionally or alternatively, the components of the genome classification system 106 may be implemented in any application that provides sequencing services, including, but not limited to, Illumina BaseSpace, Illumina DRAGEN, or Illumina TruSight software. "Illumina", "BaseSpace", "DRAGEN", and "TruSight" are registered trademarks or trademarks of Illumina, Inc. in the United States and / or other countries.

[0241] Embodiments of the present disclosure may include or utilize special purpose or general purpose computers including, for example, computer hardware such as one or more processors and system memory, as discussed in more detail below. Embodiments within the scope of the present disclosure also include physical and other computer readable media for carrying or storing computer executable instructions and / or data structures. In particular, one or more of the processes described herein may be embodied in a non-transitory computer readable medium and implemented at least in part as instructions executable by one or more computing devices (e.g., any of the media content access devices described herein). In general, a processor (e.g., a microprocessor) receives instructions from a non-transitory computer readable medium (e.g., a memory, etc.) and executes those instructions, thereby performing one or more processes, including one or more of the processes described herein.

[0242] A computer-readable medium may be any available medium that can be accessed by a general-purpose or special-purpose computer system. A computer-readable medium that stores computer-executable instructions is a non-transitory computer-readable storage medium (device). A computer-readable medium that carries computer-executable instructions is a transmission medium. Thus, by way of example and not limitation, embodiments of the present disclosure may include at least two distinctly different kinds of computer-readable media: non-transitory computer-readable storage medium (device) and transmission media.

[0243] Non-transitory computer-readable storage media (devices) include RAM, ROM, EEPROM, CD-ROM, (e.g., RAM-based) solid-state drives (SSD), flash memory, phase-change memory (PCM), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer.

[0244] A "network" is defined as one or more data links that enable the transport of electronic data between computer systems and / or modules and / or other electronic devices. When information is transferred or provided to a computer over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless), the computer properly recognizes the connection as a transmission medium. A transmission medium may include a network and / or data links that may be used to carry desired program code means in the form of computer-executable instructions or data structures and that may be accessed by a general-purpose or special-purpose computer. Combinations of the above should also be included within the scope of computer-readable media.

[0245] Furthermore, upon reaching various computer system components, program code means in the form of computer executable instructions or data structures may be automatically transferred from the transmission medium to the non-transitory computer readable storage medium (device) (or vice versa). For example, computer executable instructions or data structures received over a network or data link may be buffered in a RAM in a network interface module (e.g., a NIC) and then eventually transferred to the computer system RAM and / or to a less volatile computer storage medium (device) in the computer system. It should therefore be understood that the non-transitory computer readable storage medium (device) may be included in computer system components that also (or even primarily) utilize a transmission medium.

[0246] Computer-executable instructions include, for example, instructions and data that, when executed by a processor, cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a certain function or group of functions. In some embodiments, computer-executable instructions are executed on a general-purpose computer to transform the general-purpose computer into a special-purpose computer that implements elements of the present disclosure. Computer-executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and / or methodological operations, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the described features or operations described above. Rather, the described features and operations are disclosed as example forms of implementing the claims.

[0247] Those skilled in the art will appreciate that the present disclosure may be implemented in a networked computing environment having many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, cell phones, PDAs, tablets, pagers, routers, switches, etc. The present disclosure may also be implemented in a distributed system environment in which both local and remote computer systems perform tasks that are linked through a network (either by hardwired data links, wireless data links, or a combination of hardwired and wireless data links). In a distributed system environment, program modules may be located in both local and remote memory storage devices.

[0248] Embodiments of the present disclosure may also be implemented in a cloud computing environment. In this specification, "cloud computing" is defined as a model for enabling on-demand network access to a shared pool of configurable computing resources. For example, cloud computing may be used in the market to provide ubiquitous and convenient on-demand access to a shared pool of configurable computing resources. The shared pool of configurable computing resources may be quickly configured through virtualization, exposed with low management effort or service provider interaction, and then scaled accordingly.

[0249] The cloud computing model may consist of various characteristics such as, for example, on-demand self-service, wide area network access, resource pooling, rapid elasticity, measured service, etc. The cloud computing model may also expose various service models such as, for example, Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). The cloud computing model may also be deployed using different deployment models such as private cloud, community cloud, public cloud, hybrid cloud, etc. In this specification and claims, a "cloud computing environment" is an environment in which cloud computing is employed.

[0250] FIG. 13 illustrates a block diagram of a computing device 1300 that may be configured to perform one or more of the processes described above. It will be understood that one or more computing devices, such as the computing device 1300, may implement the genome classification system 106 and the sequencing system 104. As shown by FIG. 13, the computing device 1300 may include a processor 1302, a memory 1304, a storage device 1306, an I / O interface 1308, and a communication interface 1310, which may be communicatively coupled by a communication infrastructure 1312. In certain implementations, the computing device 1300 may include fewer or more components than those shown in FIG. 13. The following paragraphs describe in more detail the components of the computing device 1300 shown in FIG. 13.

[0251] In one or more embodiments, the processor 1302 includes hardware for executing instructions, such as those that make up a computer program. By way of example and not limitation, to execute instructions for dynamically modifying a workflow, the processor 1302 may retrieve (or fetch) instructions from an internal register, an internal cache, memory 1304, or storage device 1306, decode them, and execute them. The memory 1304 may be a volatile or non-volatile memory used to store data, metadata, and programs for execution by the processor. The storage device 1306 includes storage devices, such as a hard disk, flash disk drive, or other digital storage device, for storing data or instructions for performing the methods described herein.

[0252] The I / O interface 1308 enables a user to provide input to, receive output from, or otherwise transfer data to or receive data from the computing device 1300. The I / O interface 1308 may include a mouse, a keypad or keyboard, a touch screen, a camera, an optical scanner, a network interface, a modem, other known I / O devices, or a combination of such I / O interfaces. The I / O interface 1308 may include one or more devices for presenting output to a user, including, but not limited to, a graphics engine, a display (e.g., a display screen), one or more output drivers (e.g., a display driver), one or more audio speakers, and one or more audio drivers. In certain implementations, the I / O interface 1308 is configured to provide graphical data to a display for presentation to a user. The graphical data may represent one or more graphical user interfaces and / or any other graphical content that may be useful in a particular implementation.

[0253] Communications interface 1310 may include hardware, software, or both. In any case, communications interface 1310 may provide one or more interfaces for communications (e.g., packet-based communications, etc.) between computing device 1300 and one or more other computing devices or networks. By way of example and not limitation, communications interface 1310 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wired-based network, or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as WI-FI.

[0254] Further, the communication interface 1310 can facilitate communication with various types of wired or wireless networks. The communication interface 1310 can also facilitate communication using various communication protocols. The communication infrastructure 1312 can also include hardware, software, or both that couple the components of the computing device 1300 to one another. For example, the communication interface 1310 can enable multiple computing devices connected by a particular infrastructure to communicate with one another to perform one or more aspects of the processes described herein using one or more networks and / or protocols. To illustrate, a sequencing process can enable multiple devices (e.g., a client device, a sequencing device, and a server device) to exchange information such as sequencing data and error notifications.

[0255] In the foregoing specification, the present disclosure has been described with reference to certain exemplary embodiments thereof. Various embodiments and aspects of the present disclosure are described with reference to the details discussed herein, and the accompanying drawings illustrate various embodiments. The above description and drawings are illustrative of the present disclosure and should not be construed as limiting the present disclosure. Numerous specific details are described to provide a thorough understanding of various embodiments of the present disclosure.

[0256] The present disclosure may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects as illustrative only and not restrictive. For example, the methods described herein may be performed with fewer or more steps / actions, or the steps / actions may be performed in a different order. Further, the steps / actions described herein may be repeated or performed in parallel with each other, or with different occurrences of the same or similar operations. The scope of the present application is therefore indicated by the appended claims, rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are intended to be embraced within their scope.

Claims

1. A system comprising: at least one processor; and when executed by the at least one processor, causing the system to determine a sequencing metric for comparing a sample nucleic acid sequence from different genomic samples to genomic coordinates of a reference genome; train a genomic position classification model to determine a coordinate-specific confidence classification indicating the accuracy with which a nucleic acid base can be determined at a specific genomic coordinate of the reference genome, based on sequencing metrics and coordinate-specific ground truth classifications from different genomic samples indicating the accuracy with which a nucleic acid base can be determined at the genomic coordinate of the reference genome; utilize the genomic position classification model to determine a set of coordinate-specific confidence classifications indicating the accuracy with which nucleic acid bases can be determined at a set of genomic coordinates of the reference genome, based on a set of sequencing metrics of sample nucleic acid sequences from a set of different genomic samples; generate at least one digital file comprising the set of coordinate-specific confidence classifications for the set of genomic coordinates; a non-transitory computer-readable medium comprising instructions; The system comprising the above.

2. The system of claim 1, wherein the coordinate-specific confidence classification indicates the degree to which variant nucleic acid base calls and invariant nucleic acid base calls can be accurately determined at the specific genomic coordinate of the reference genome.

3. The system of claim 1 or 2, wherein the sample nucleic acid sequence is determined using a single sequencing pipeline comprising a nucleic acid sequence extraction method, a sequencing device, and sequence analysis software.

4. The system of claim 1, further comprising instructions that, when executed by the at least one processor, cause the system to determine a coordinate-specific confidence classification from the set of coordinate-specific confidence classifications by determining a coordinate-specific confidence classification for genomic coordinates of the reference genome that include genetic or epigenetic modifications.

5. when executed by the at least one processor, causing the system to an alignment metric for quantifying an alignment between the sample nucleic acid sequence and the genomic coordinates of the reference genome, a depth metric for quantifying the depth of nucleic acid base calls for the sample nucleic acid sequence at the genomic coordinates of the reference genome, or A call data quality metric for quantifying the quality of a nucleobase call for the sample nucleic acid sequence at the genomic coordinates of the reference genome The system according to claim 1, further comprising instructions for determining the sequencing metric by determining one or more of the above.

6. When executed by the at least one processor, the system is caused to Determine the alignment metric by determining one or more of a deletion-entropy metric, a deletion-size metric, a mapping-quality metric, a positive-insertion size metric, a negative-insertion size metric, a soft-clipping metric, a read-position metric, or a read-reference mismatch metric for the sample nucleic acid sequence, Determine the depth metric by determining one or more of a forward-reverse depth metric, a normalization-depth metric, a depth-under metric, a depth-over metric, or a peak-count metric, or The system according to claim 5, further comprising instructions for determining the call data quality metric by determining one or more of a nucleobase call quality metric, a call likelihood metric, or a somatic quality metric for the sample nucleic acid sequence.

7. The system according to claim 1, further comprising instructions for determining a coordinate-specific confidence classification from the set of coordinate-specific confidence classifications by determining at least one of a high-confidence classification, a medium-confidence classification, or a low-confidence classification regarding the genomic coordinates of the reference genome when executed by the at least one processor.

8. The system according to claim 1, further comprising instructions for determining a coordinate-specific confidence classification from the set of coordinate-specific confidence classifications by determining a confidence score within a range of confidence scores indicating the probability that a nucleobase can be accurately determined at the genomic coordinates of the reference genome when executed by the at least one processor.

9. When executed by the at least one processor, the system further includes instructions for training the genomic position classification model to determine the coordinate-specific confidence classification by training a statistical machine learning model or a neural network to determine the coordinate-specific confidence classification. The system according to claim 1.

10. When executed by the at least one processor, the system determines from the reference genome a context nucleic acid subsequence surrounding the variant-nucleotide base call, trains the genomic position classification model to determine a coordinate-specific confidence classification for the genomic coordinates of the variant-nucleotide base call, the context nucleic acid subsequence, a subset of sequencing metrics for a subset of genomic coordinates of the reference genome corresponding to the context nucleic acid subsequence, and a subset of coordinate-specific ground truth classifications for a subset of genomic coordinates corresponding to the context nucleic acid subsequence based on which to determine. The system according to claim 1 further includes instructions.

11. When executed by at least one processor, on a computing device, detect a variant-nucleotide base call at a genomic coordinate of a reference genome within a sample nucleic acid sequence, using sequencing metrics from different genomic samples from a digital file, identify a coordinate-specific confidence classification indicating the accuracy with which a nucleotide base can be determined at the genomic coordinate of the reference genome according to the genomic position classification model, generate an indicator of the coordinate-specific confidence classification for the genomic coordinates of the reference genome for display within a graphical user interface. A non-transitory computer-readable medium storing instructions.

12. When executed by the at least one processor, the computing device further stores instructions for identifying the coordinate-specific confidence classification for the genomic coordinates by identifying the coordinate-specific confidence classification indicating the degree to which variant nucleotide base calls and invariant nucleotide base calls can be accurately determined at the genomic coordinates from the digital file. The non-transitory computer-readable medium according to claim 11.

13. When executed by the at least one processor, the computing device further stores instructions for identifying the coordinate-specific confidence classification from annotations or scores for the genomic coordinates in the digital file, thereby causing the coordinate-specific confidence classification to be identified from the digital file. The non-transitory computer-readable medium according to claim 11 or 12.

14. When executed by the at least one processor, the computing device further stores instructions for identifying the coordinate-specific confidence classification from the digital file by identifying at least one of a high-confidence classification, a medium-confidence classification, or a low-confidence classification for the genomic coordinates from the digital file. The non-transitory computer-readable medium according to claim 11.

15. A method comprising: determining, from a reference genome, a context nucleic acid subsequence surrounding a variant-nucleobase call in a sample nucleic acid sequence at a genomic coordinate from the genomic coordinates of the reference genome; training a genomic position classification model to determine a coordinate-specific confidence classification indicating the accuracy with which a nucleobase can be determined at the genomic coordinates of the reference genome, based on the context nucleic acid subsequence and the ground truth classification indicating the accuracy with which the nucleobase can be determined at the genomic coordinates of the reference genome; using the genomic position classification model to determine a coordinate-specific confidence classification indicating the accuracy with which a nucleobase can be determined at the genomic coordinates of the reference genome, based on the context nucleic acid subsequence; generating at least one digital file including the coordinate-specific confidence classification for the genomic coordinates of the reference genome; A method comprising the above steps.

16. Determining the coordinate-specific confidence classification includes determining the coordinate-specific confidence classification for a single nucleotide variant, a nucleobase insertion, a nucleobase deletion, a part of a structural change, or a part of a copy number change at a genomic coordinate. The method according to claim 15.

17. Determining the coordinate-specific confidence classification includes determining a confidence score within a range of confidence scores indicating the probability that a nucleobase can be accurately determined at a genomic coordinate. The method according to claim 15 or 16.

18. Training the genomic location classification model to determine the coordinate-specific confidence classification includes training a logistic regression model, a random forest classifier, or a convolutional neural network to determine the coordinate-specific confidence classification, the method according to claim 15.

19. Training the genomic location classification model to determine the coordinate-specific confidence classification includes for the genomic coordinates, comparing a predicted coordinate-specific confidence classification with a coordinate-specific ground truth classification that reflects the Mendelian inheritance pattern or replication concordance of the nucleobase calls at the genomic coordinates; determining a loss from the comparison of the predicted coordinate-specific confidence classification and the coordinate-specific ground truth classification; adjusting parameters of the genomic location classification model based on the determined loss; and the method according to claim 15.

20. The method according to claim 15, wherein the reference genome includes a path representing the nucleic acid sequence of an ancestral haplotype.

Images