Sequence variation detection using deep learning

Abstract

A method for sequence variant detection using deep learning is disclosed. This disclosure provides methods, systems, and computer program products for accurately and effectively detecting variants in biopolymer sequencing data, particularly suboptimal sequencing data, using embeddings of candidate variant information and deep learning models.

Description

Technical Field

[0001] This disclosure generally relates to methods and systems for detecting variations in biopolymer sequencing data using embedding and deep learning. Background Technology

[0002] The human genome is the complete set of nucleic acid sequences of humans, encoding DNA in the 23 chromosome pairs in the cell nucleus and in small DNA molecules in individual mitochondria. The human genome consists of 6 billion base pairs, including protein-coding DNA genes and non-coding DNA. The Human Genome Project released the first complete sequence of a single human genome in 2001. Currently, thousands of human genomes have been fully sequenced, and many more genome maps have been created at lower resolutions. The resulting data has been used globally in biomedicine, anthropology, forensics, and other scientific fields. It is widely expected that genome research will lead to advances in the diagnosis and treatment of diseases and bring new insights into many areas of biology, including human evolution.

[0003] However, understanding the genetic basis of disease requires accurate and rapid determination of an individual's genomic DNA sequence down to the single base pair level. This level of resolution in DNA sequencing allows for the identification of natural variations in sequences that occur between different individuals. These individual sites of sequence variation, often called single nucleotide variants (SNVs) or single nucleotide polymorphisms (SNPs), are present throughout an individual's genome and provide potential key to using genomic sequence information across a wide range of applications. An SNP refers to a variation in the location of a genomic sequence that occurs in different populations. For example, at a specific location in the human genome, nucleotide C may be present in most humans, but in a few individuals, A may be present at the same genomic location. For a specific location in the genome, these two different nucleotides are called alleles. It is estimated that an average of one SNP occurs every 300 base pairs (bp) throughout the human genome, resulting in an average of approximately 10 million SNPs in the human genome sequence relative to a reference genome.

[0004] SNPs generally refer to variations found in the genome of human germline cells. SNVs are a more general term that can include single nucleotide variations at specific sites in the genome of somatic cells. Cancer cells are a type of somatic cell that are extensively studied, and include SNVs considered crucial for their pathological phenotype and diagnosis.

[0005] Since the first determination of a single human genome in 2000, DNA sequencing technology has made tremendous progress, costing an estimated $2.7 billion. Currently, the most advanced high-throughput technology is commonly referred to as "next-generation sequencing" (NGS). NGS technology has enabled large-scale sequencing of plant and animal genomes, allowing the determination of an entire genome sequence to be completed in as little as a week at a cost of approximately $1,000.

[0006] NGS techniques typically work by simultaneously performing millions of individual overlapping sequence reactions, each producing a short sequence, or "reading," of a few hundred base pairs in length. Sequence identification requires multiple reads covering each base position, and a 30x redundancy read at each base position (i.e., "30X coverage") is generally expected. Therefore, NGS generates large datasets consisting of numerous sequence reads. However, the sequence reads provided by NGS have a relatively high error rate, approximately 0.1-10%. Therefore, NGS processing requires highly correlated statistical error analysis for each sample. The complexity of post-reaction processing to address and minimize erroneous NGS sequence reads poses a significant challenge to the correct identification or "recalling" of mutations or variations in the genomic sequence.

[0007] To distinguish true variants from errors present in NGS datasets, methods and software tools have been developed for this purpose. Two widely used software tools, GATK (available at software.broadinstitute.org / gatk / ) and SAMtools (Li et al., “The Sequence Alignment / Map format and SAMtools,” Bioinformatics 25(16): 2078-2079(2009)), can be used to retrieve variants (e.g., SNPs, SNVs), insertions, and deletions from NGS datasets. These widely used variant retrieval software tools use “classical” methods to align sequence reads and perform bioinformatics analysis and machine learning modeling on the alignment reads to retrieve variants. The “classical” bioinformatics and machine learning components of these software tools require labor-intensive “handcrafted” data features, which greatly limits their ability to generalize sequencing datasets obtained from different types of sequencers and / or data with different depth coverage. Furthermore, when applied to suboptimal (e.g., low coverage) sequencing datasets, the accuracy of variant invocation by classic tools such as GATK deteriorates significantly.

[0008] DeepVariant is a deep learning-based software tool developed to improve upon classic variant calling tools (e.g., see Poplin et al., “Creating a universal SNP and small indel variant caller with deep neural networks,” bioRxiv 092890; doi: doi.org / 10.1101 / 092890; published December 14, 2016). DeepVariant uses convolutional neural networks (CNNs) to call genetic variants in contrastive NGS reads by learning a statistical relationship between read stack images around hypothetical variant sites and ground-based true genotype calls. DeepVariant has been shown to outperform GATK at 30X sequencing depth on benchmark full genomes and generalizes better to sequencer data not used during training. The accuracy of DeepVariant in calling variants using low-coverage sequencing data remains unclear.

[0009] Therefore, there is still a need for improved methods that reduce costs, processing time, and sample requirements to enable high-precision detection and classification of variations found in sequencing data. Attached Figure Description

[0010] Figure 1 An overview flowchart of exemplary operations for detecting candidate variants in the embedding of sequencing data according to embodiments of the present disclosure is provided.

[0011] Figure 2 A block diagram depicts exemplary operations for generating embeddings of candidate variant information according to embodiments of the present disclosure, the candidate variant information comprising a plurality of biopolymer sequence reads.

[0012] Figure 3 A block diagram of an exemplary deep learning model architecture according to embodiments of the present disclosure is depicted, which can be used to process the embedding of candidate variant information, including biopolymer sequence reads.

[0013] Figure 4 A block diagram of an exemplary system according to embodiments of the present disclosure is depicted, which can be used to detect candidate variants in the embeddings of sequencing data using a deep learning model and training the deep learning model.

[0014] Figure 5 A system according to embodiments of the present disclosure is described, wherein a process, architecture, and / or function for using a deep learning model to detect candidate variants in the embeddings of sequencing data can be implemented. Detailed Implementation

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Generally, the terms used to describe the techniques and processes described herein are well understood and commonly used by one of ordinary skill in the art, such as common techniques and methods for high-throughput (or next-generation) sequencing and for calling mutations (e.g., SNPs or SNVs) in such sequencing data. Such common techniques and methods for sequence data acquisition and mutation analysis are described, for example, in Goodwin et al., “Coming of age: ten years of next-generation sequencing technology,” Nat RevGenet, June 2016, 17(6): 333-51, and in Nielsen et al., “Genotype and SNP calling from next-generation sequencing data,” Nat. Rev. Genet, June 2011, 12(6): 443-451.

[0016] For the purposes of this disclosure, terms used in the singular will also include the plural forms where appropriate, and vice versa. For the descriptions provided herein and in the appended claims, the singular forms “a” and “an” include a plural of objects unless the context explicitly indicates otherwise. The use of “comprising,” “consisting of,” “containing,” “including,” “covering,” and “including…includes” is interchangeable and not restrictive. It will be further understood that where the term “comprising” is used in the description of various embodiments, those skilled in the art will understand that in certain specific cases, the language “substantially consisting of” or “consisting of” may be used alternatively to describe embodiments. In the case of describing ranges of values, unless the context explicitly specifies otherwise, it should be understood that each intermediate integer of the value and each tenth of each intermediate integer of the value, unless the context explicitly specifies otherwise, the “maximum value” within the range between the upper and lower limits and any other value or intermediate value within that range are included within the invention. The upper and lower limits of these smaller ranges may be included independently within the smaller range and also within the invention, subject to any explicit exclusions within the range. Therefore, in cases where the range includes one or both of these limitations, the range excluding one or both of the included limitations is also included in this invention. For example, "1 to 50" includes "2 to 25", "5 to 20", "25 to 50", "1 to 10", etc.

[0017] Those skilled in the art will understand that the methods, systems, and computer-readable media products described herein offer numerous advantageous technical effects, including improved accuracy, speed, compactness, and overall efficiency in detecting variations in sequencing data, even with suboptimal or low-coverage data, using deep learning models. It should also be understood that the following description is not intended as a comprehensive overview; therefore, concepts have been simplified for clarity and brevity.

[0018] All publications, patents, patent applications and other documents cited in this disclosure are incorporated herein by reference in their entirety for all purposes, to the same extent as each individual publication, patent, patent application or other document individually indicated to be incorporated herein by reference for all purposes.

[0019] Classifying candidate variants using embeddings and deep learning

[0020] This disclosure provides methods, systems, and computer program products for detecting candidate variations in biopolymer sequences using embeddings representing biopolymer sequence information (e.g., sequence reads).

[0021] Robust neural networks are known to be obtained by training with dense vectors, where all values ​​help define the “object” of interest. However, many objects of interest do not naturally represent as dense vectors. For example, multiple biopolymer sequence reads obtained from high-throughput sequencing experiments are simply long strings of four letters representing the monomer units of the biopolymer (e.g., nucleic acid bases, A, C, G, and T). Embeddings are mappings of discrete objects to real-valued vectors. The magnitude and value of the embedding vectors are known, but they have no other inherent meaning. Typically, initial real-valued embeddings are randomly generated, and the final embedding values ​​are learned by training the neural network. It has been found that the overall position, pattern, and distance of vectors in some learned embeddings can provide useful information. For example, similarities in the vector space used to learn embeddings of large groups of words (e.g., Euclidean distance or angles between vectors) have been found to be robust and flexible measures of word similarity. Techniques and methods for generating and using embeddings of words and other discrete objects in deep learning models are well known in the art (see, for example, www.tensorflow.org / guide / embeddings).

[0022] Previously, learned embeddings of biopolymer sequence information have not been used to detect candidate variants in sequences, such as mutations in data or simple experimental errors. A surprising result of this disclosure is that multiple embeddings representing multiple biopolymer sequence reads can be used to accurately classify one or more candidate variants in a sequence. The methods, systems, and procedures described offer greater efficiency, accuracy, and speed in classifying candidate variants (e.g., referred to as SNPs) detected in multiple sequence reads typically generated by a series of sequencers in high-throughput sequencing of biopolymers. Furthermore, the methods and systems of this disclosure, which use embeddings of sequence reads to detect variants in sequences, can be processed using deep learning models to obtain accurate classifications of candidate variants even from suboptimal data (e.g., low-coverage NGS data). The effects of the methods and systems disclosed herein include a significant reduction in the time and cost of obtaining biopolymer sequences (e.g., germline and somatic genomes), and higher accuracy in detecting the presence and identity of variants in those biopolymer sequences.

[0023] As described elsewhere in this document, high-throughput biopolymer sequencing technologies provide very large, highly redundant datasets containing thousands or millions of sequence reads with high error rates. The increasing use of these high-throughput sequencing technologies has led to a range of data analysis challenges due to their massive datasets and error rates. One of the challenging problems when dealing with any high-throughput dataset of biopolymer sequence reads is the detection of true variants in the sequences. These variants have significant scientific value but are hidden among false variants in the sequencing data. Errors appear very similar to true variants and are difficult to characterize statistically, especially across multiple experiments and instruments. The methods and systems disclosed herein aim to address the problem of detecting whether a putative or candidate variant in a sequence read is a true difference in the sequence or simply an error. Utilizing embeddings and deep learning, the methods and systems of this disclosure can be applied to detect candidate variants from any type of biopolymer sequencing data. Currently, high-throughput sequencing technologies for nucleic acid sequencing are widely used. However, it is anticipated that the methods and systems of this disclosure can be applied to large datasets of sequence reads from any biopolymer, such as protein or carbohydrate sequence reads.

[0024] In some embodiments, candidate variants detected using the methods and systems of this disclosure may include sequence variations or mutations, such as SNPs, SNVs, and / or insertions and deletions, which are identified in large datasets of overlapping sequence reads corresponding to sequences of multiple different nucleic acid molecules synthesized from separate portions of a template being “sequentially sequenced.” The problem of classifying candidate variants is frequently encountered in large datasets of overlapping sequence reads obtained from NGS experiments. A range of NGS techniques are known in the art. For a review of NGS techniques, see, for example, Goodwin et al., “Coming of age: ten years of next generation sequencing technology,” Nat Rev Genet; June 2016; 17(6): 333-51. It is anticipated that the methods and systems of this disclosure can be used to classify variants from sequencing data obtained from any NGS technique known in the art.

[0025] In some embodiments, candidate variants detected using the methods and systems of this disclosure may include variations between sequences corresponding to copies of the same molecule. Such variations arise from errors in multiple sequencing processes of the same molecule, such as circular co-sequencing (or “CCS”). See, for example, Wenger et al., “Highly-accurate long-read sequencing improves variant detection and assembly of a human genome,” bioRxiv; published January 23, 2019 (doi: https: / / doi.org / 10.1101 / 519025). In short, CCS is an NGS technique that uses a large, single circular nucleic acid construct that is repeatedly sequenced, resulting in multiple sequence reads of the same molecule. These CCS sequence reads are typically longer (e.g., average length ~10,000 to 100,000 bp). The longer lengths offer advantages for certain types of genome sequencing analysis, but the error rates associated with reading these long sequences from CCS are relatively high. Error-induced variations between multiple sequence reads corresponding to the same sequence lead to problems in generating true shared sequences. Even though more measurements should theoretically provide higher shared sequence accuracy, increasing the number of CCS sequence reads per target sequence results in a decrease in the overall throughput of the CCS instrument. Our method and system, using embeddings and deep learning, can provide higher accuracy shared sequence generation from CCS datasets with fewer sequence reads. Typically, the same method and system utilizing embeddings and deep learning for classifying candidate variants can be used to address the problem of detecting candidate variants and generating accurate shared sequences from datasets containing multiple CCS sequence reads of the same large target nucleic acid molecule.

[0026] NGS technology works by performing millions of sequencing reactions in parallel, and each reaction results in millions of “sequence reads” (also referred to as “reads” or “read sequences” in this paper) that are typically a few hundred base pairs (bp) long, although some NGS technologies (such as Cyclic Concord Sequencing (or “CCS”)) can produce reads of an average length of 10,000 bp. A sequence read corresponds to a short portion of a larger sequence, such as an exome or genome, which can be millions or billions of bp long. The multiple read sequences generated in an NGS dataset may not all be contiguous portions of a larger target sequence, but rather overlapping portions covering the entire length of the target sequence. This overlapping nature of sequence reads leads to useful data redundancy in NGS experiments, but the level of redundancy varies across different parts of the sequence. Currently, the standard considered to be the level of redundancy required for accurate sequences is “30X coverage,” meaning that an average of 30 independent sequence reads cover every base position in the sampled genome. Unfortunately, large numbers of parallel sequence reads have a relatively high error rate, approximately 0.1% to 10%. Therefore, further post-processing analysis of the errors is required to provide an accurate overall sequence for each sample. Furthermore, the 30X coverage is an average across the entire sequencing dataset. Many locations in the dataset have far fewer reads (e.g., only 10), while others have far more (e.g., 1000). The high error rate across numerous sequence reads and the uneven coverage present a particularly challenging problem when attempting to use NGS sequencing data to accurately detect and confirm the presence of true mutations or variants in nucleic acid sequences (such as single nucleotide variants (SNVs), single nucleotide polymorphisms (SNPs), or insertions / deletions).

[0027] In high-throughput sequencing datasets, differences or variations typically occur between sequence reads and / or between sequence reads and a reference sequence. For example, millions of candidate variations (“genome” refers to the entire genome or a portion of the genome, such as the exome) are observed in typical high-throughput sequencing datasets spanning the genome. These observed sequence read variations may be due to actual variant sequences (e.g., mutations) or some kind of experimental error (e.g., random errors due to high noise). Observed but unconfirmed sequence variations relative to other sequence reads and / or relative to a reference sequence are referred to as “candidate variations.” “Candidate variation detection” or “candidate variation classification” constitutes determining whether an observed candidate variation is a real variation in the sequence (e.g., a variant or mutation) or an error in the sequence (e.g., experimental noise). In the case of candidate variation detection of real variations in the sequence, detection or classification may also include determining the type of variation (e.g., substitution, insertion, or deletion in the sequence). Optionally, in the case of nucleic acid sequencing, candidate variation detection may further include determining whether the sequence variation occurs on one, two, or more alleles in the genome or exome. Thus, in some embodiments, candidate variation detection or classification includes more than just a yes or no answer. For example, in the case where two alleles (e.g., allele A and allele B) correspond to gene sequences from a genomic NGS dataset, the detection of candidate variants can yield four possible answers, each corresponding to the probability of a true sequence variant being present in either or both of the two alleles—e.g., Yes. A Yes B No A Yes B Yes A / No B Or No A / No B .

[0028] The problem of classifying numerous candidate variants is amplified when the source sequencing datasets used to identify candidate variants are suboptimal. "Suboptimal" refers to quality below best quality, and "suboptimal candidate variants" are candidate variants obtained from sequencing datasets of suboptimal quality. Such datasets typically include suboptimal sequence reads and are therefore susceptible to a high ratio of undetected, falsely detected, and / or misclassified observed variants. Suboptimal candidate variant information can arise from many sources detrimental to sequencing data quality, including the use of fewer sequence reads (e.g., low-coverage datasets), a limited number of samples, and / or any source that can introduce noise or errors. However, the ability to utilize suboptimal candidate variant information (including less-than-ideal sequence reads) can be desirable because suboptimal datasets are readily available, enabling faster and cheaper sequencing experiments. Therefore, using suboptimal datasets can reduce time and cost, allowing for accurate identification of the complete human genome. Consequently, there will be significant gains in improved methods and systems for candidate variant classification.

[0029] This disclosure provides methods and systems for detecting candidate variants from high-throughput sequencing data using deep learning models. Specifically, the methods and systems can be used to generate embeddings of candidate variant information obtained from high-throughput sequencing datasets, including suboptimal sequencing datasets, such as low-coverage data obtained at sequencing depths less than 30X. By allowing accurate candidate variant detection using low-coverage data, the deep learning-based methods and systems of this disclosure can facilitate faster, more efficient, and more cost-effective nucleic acid sequencing resources. Therefore, faster and more efficient sequencing can enable nucleic acid sequencing to be used more broadly in medical applications, such as personalized medicine and diagnostics.

[0030] The deep learning models and related techniques used in the methods disclosed herein are based on the use of neural networks, such as convolutional neural networks (CNNs), in computer-based systems. Such deep learning models have been successfully used to extract features and classify data in a wide range of applications, particularly image, language, and speech recognition. To efficiently process candidate variants derived from high-throughput sequencing datasets using deep learning models, embeddings suitable for the sequencing data are generated, which facilitate the use of neural networks.

[0031] In some embodiments, this disclosure provides a method comprising: generating a plurality of embeddings comprising a plurality of biopolymer sequence reads; and detecting one or more candidate variants in the sequence reads based at least in part on the plurality of embeddings.

[0032] Figure 1A general flowchart illustrating exemplary operation of the methods and systems described herein is shown, which is used to generate multiple embeddings including sequence reads (particularly reads from suboptimal sequencing data) and to detect candidate variants of the sequence reads in part based on the embeddings. Candidate variant information 101 (also referred to herein as “CV information”) is generated from standard sequencing data using any software tools well known in the art for processing sequencing data and making variant calls, which includes multiple biopolymer sequence reads. Two of the most widely used software tools for nucleic acid sequence datasets are GATK (i.e., “Genome Analysis Toolkit”), available from the Broad Institute (software.broadinstitute.org / gatk / ) and SAMtools (Li et al., “The Sequence Alignment / Map format and SAMtools”, Bioinformatics 25(16): 2078-2079 (2009). These software tools perform “classical” variant invocation by aligning numerous sequence reads from the target sample to a reference genome, and then using bioinformatics analysis and machine learning modeling of the alignment reads to generate candidate variants.

[0033] Candidate variant information comprises multiple biopolymer sequence reads and is useful in the methods and systems of this disclosure. This candidate variant information can be generated from standard sequencing data using standard, well-known software for variant recall. For example, datasets of high-throughput NGS reads can be analyzed using appropriate reference sequences and classic variant recall software tools (e.g., GATK and SAMtools) to generate candidate variants. In some embodiments, the sequencing data used in this method is a low-coverage dataset with fewer than the optimal number of reads at each candidate variant location. Such suboptimal, low-coverage datasets are useful in the candidate variant classification methods of this disclosure, and in some embodiments, datasets with target genome sequence coverage of less than 30x, less than 20x, less than 15x, or even lower, may be included.

[0034] In some embodiments, the candidate variant information 101 used in the methods and systems of this disclosure includes a subset of standard information found in file formats (e.g., VCF and / or BAM files) generated by classic variant calling software tools (e.g., GATK and SAMtools). For example, candidate variant information can be extracted from a variant calling format (VCF) file. This VCF file contains information about the location and base calls of a particular candidate variant, its absolute location in the genome (e.g., detailed chromosomal sites), and the quality of base calls in the read sequence (BQ). The VCF file also includes a compressed sequence alignment / mapping format (BAM) file, which contains information about read mapping / alignment and its quality, also referred to herein as read mapping quality (MQ). The VCF and BAM file formats are well known in the art, and a complete description of each format is publicly available online at github.com / samtools / hts-specs.

[0035] As stated above, it is not necessary to use all candidate mutation information generated by the standard mutation invocation software tool in the deep learning method disclosed herein. In fact, the advantage of the currently disclosed method is that it only requires selected data representing candidate mutation information 101 to generate embeddings of candidate mutation information 102, including embeddings of multiple sequence reads, which can then be processed by the deep learning model 103 to generate output for detecting (or classifying) candidate mutations 104. Figure 2 The generation of the embedding representing candidate mutation information 102 is further described in detail. Figure 3 The architecture of the deep learning model is further explained in detail.

[0036] In some embodiments, the methods and systems of this disclosure use embeddings representing candidate variant information (e.g., multiple embeddings comprising multiple read sequences) as input to a deep learning model for classifying candidate variants. In some embodiments of the methods and systems, the embeddings may further include sequence quality information, sequence position information, reference sequence information, and / or variant hypothesis information.

[0037] Figure 2A block diagram of exemplary candidate variant information 101 is shown, which can be extracted and used to generate embeddings 102 of candidate variant information, such as embeddings of multiple biopolymer sequence reads. These embeddings can then be processed by a deep learning model 103 of this disclosure, as further described below. In some embodiments, the candidate variant information 102 extracted and used to generate embeddings includes multiple biopolymer sequence reads. In some embodiments, the candidate variant information extracted and used to generate embeddings includes a reference sequence 201, a read sequence 202, a quality value or score 203 of the read sequence (e.g., base quality or "BQ" value), positional information 204 associated with the reference and read sequences, and hypothesis information 205.

[0038] In some embodiments of the methods and systems disclosed herein, multiple biopolymer sequence reads are nucleic acid sequence reads. Such nucleic acid sequence reads (and corresponding reference sequences) are extracted from candidate variant information typically generated by “classic” sequence analysis software tools (such as GATK) and represented as base strings (e.g., A, C, G, T, or U), inserted, or deleted.

[0039] In some embodiments, the plurality of sequence reads 202 correspond to those reads that overlap with the position centered on the candidate variant site in the target or reference sequence. Similarly, the reference sequence 201 corresponds to the sequence encompassing the entire range of the variant site and the target sequence covered by the plurality of biopolymer sequence reads 202, typically equidistant upstream and downstream of the variant site. Figure 2 In the exemplary embodiment shown, the embedding of CV information 102 is generated by linking a reference sequence embedding 206 with each of a plurality of read sequence embeddings 207. As described elsewhere herein, the linking of the reference sequence embedding 206 with each of the plurality of read sequence embeddings 207 in the methods and systems of this disclosure results in more efficient and accurate single-read processing via a deep learning model.

[0040] The embeddings of reference sequence 206 and read sequence 207 are generated as fixed-dimensional real-valued vectors. The fixed dimension of the embeddings (sometimes referred to as the "channels" of the embeddings) can be chosen based on factors such as the expected efficiency of the deep learning model in processing. In some embodiments of this disclosure, the embeddings of reference sequence 206 and read sequence 207 each include at least 20 dimensions. However, it is contemplated that larger-dimensional embeddings of reference sequence 206 and read sequence 207 can be used in the methods and systems of this disclosure. Therefore, in some embodiments, the embeddings of the reference sequence and / or the read sequence include at least 20 dimensions, at least 50 dimensions, at least 100 dimensions, at least 200 dimensions, at least 300 dimensions, or even more.

[0041] The positional information 204 used for the reference sequence 201 and multiple sequence reads 202 varies depending on the length and position of the read sequence relative to the candidate variant position. The position of the candidate variant can be represented by a numerical "address" or "index," which indicates a specific location in the genome (or other reference sequence) where the candidate variant is located. The sequence positional information 204 provides important information for the deep learning model 103 to perform efficient and accurate training and inference. To enable the deep learning model to process the positional information 204 associated with each of the reference sequence 201 and multiple read sequences 202, embeddings representing positional information 208 are generated. The generated embeddings of positional information 208 have the same dimensions (or number of channels) as the embeddings of the reference sequence and read sequences (e.g., 20 dimensions). These positional information embeddings 208 are then summed and assigned (e.g., ...). Figure 2 The embeddings of the reference sequence 206 and the multiple read sequences 207 are shown in the "+=" function. Therefore, the positional information 204 associated with the reference sequence 201 and each read sequence 202 is effectively encoded into the embeddings of the candidate variation information 102, which serves as input to the deep learning model. For further explanation of the positional encoding of embeddings used in language translation in neural networks, see Vaswani, A. et al., "Attention Is All You Need", arXiv: 1706.03762v5 [cs.CL]6 Dec 2017; nlp.seas.harvard.edu / 2018 / 04 / 03 / attention.html.

[0042] Therefore, in some embodiments of the methods and systems disclosed herein, the generation of embeddings for multiple biopolymer sequence reads further includes: generating embeddings for sequence reads and embeddings for sequence position information of sequence reads, respectively, and adding the embeddings of sequence position information to assign to the embeddings for sequence reads.

[0043] Sequence quality information is typically associated with multiple reads generated by high-throughput NGS experiments and is found in candidate variant information generated by classic sequence retrieval tools and algorithms. Sequence quality information may include, for example, base quality (BG) scores and alignment quality scores. Sequence quality information is typically represented by scores, which are real-valued scalar values ​​(e.g., values ​​between 0 and 100). Therefore, such sequence quality information associated with a read sequence can be readily and directly associated with the embedding of the read sequence by simply concatenating another channel with the embedding of the scalar value containing the quality information. Thus, in some embodiments, sequence quality information 203 is concatenated with the embedding of the read sequence associated with it. For example, as... Figure 2As shown, the embedding of CV information 102 may include sequence quality information 203, such as BQ values, which correspond to each base of the read sequence linked to the embedding of each read sequence 207. For example, each BQ value is a single real number that can be linked to the embedding vector corresponding to each base. Including a BQ score representing each base position of the sequence read in the embedding provides an additional parameter that can help in the accurate detection of candidate variants.

[0044] Similarly, such as Figure 2 As shown, the linking of hypothesis information 205 with candidate variant information embedding 102 further facilitates the accurate and efficient processing of the neural network of deep learning model 103 in detecting candidate variants. In the method and system for classifying candidate variants disclosed herein, hypothesis information 205 corresponds to an additional channel linked to each base of read sequence embedding 207, which includes a 1 or 0 indicating true or false at the base position of the read sequence of a particular sequence hypothesis. For example, for a 201 bp nucleic acid sequence, if variant A is hypothesized to change to T at its central position 101, then hypothesis information 205 linked to sequence read embedding 207 corresponds to a channel containing 0 at each position and a channel containing 1 at position 101. In some embodiments, sequence hypothesis information 205 may include separate channels for each different type of sequence variant hypothesis, including but not limited to single base changes, insertions, or deletions of A, C, G, or T. For example, in the case of deleting multiple bases from positions 97 to 100 of the read sequence, the ligation hypothesis information 205 would include the channel containing 1 at positions 97-100 and all other channels containing 0.

[0045] Similarly, in some embodiments, to facilitate training and / or inference using the deep learning model 103, additional candidate variation information 101 can be included together with the reference 206 and / or the read sequence embedding 207 by further concatenating channels that encode additional information as 0s or 1s. For example, in some embodiments, channels are concatenated to the reference 206 and the read sequence embedding 207 that encodes start and / or end positions and / or chain directions associated with multiple read sequences.

[0046] like Figure 2As shown, in an exemplary embodiment that generates multiple embeddings comprising multiple biopolymer sequence reads, a dataset containing candidate variant information 101 of 100 nucleic acid sequence reads will result in 100 embeddings of candidate variant information 102. Assuming a 20-dimensional vector is used to represent each base, insertion, or deletion position in the multiple read sequences within the embeddings, the resulting multiple embeddings of candidate variant information 102 will comprise a vector of total dimension 41: (read base: dimension (dim) 20) + (BQ value: dim 1) + (reference base: dim 20). Furthermore, assuming the nucleic acid read sequence is 200 positions long, the resulting embeddings of candidate variant information 102 will comprise a 41×200 2D array representing real numbers for each read sequence. A complete set of embeddings 102 representing multiple 100 sequence reads will result in a 100×41×200 3D array with a total of 820,000 real numbers representing the candidate variant information ready for processing by the deep learning model 103. As described above, the size of the 3D array is greatly reduced by adding the embedding of the position information 208 to the embedding of the sequence reading 207.

[0047] The association between the reference sequence embedding 206 and each individual sequence read embedding 207 provides additional speed and efficiency in processing embeddings using the deep learning model 103 by allowing shortened processing of candidate variant information embeddings at locations with a relatively large number of redundant overlapping read sequences that can quickly converge to classification. For example, if 10 of the 25 available embeddings representing candidate variant information for 25 read sequences can be used to extract classifications via a neural network, then processing for that particular candidate variant classification can terminate 15 embeddings early and begin processing for the next candidate version, thus saving computational cost and time.

[0048] In a typical high-throughput sequencing dataset, the number of sequence reads representing each of millions of potential candidate variants can vary significantly, for example, from N=10 to N=10000. This variation in the number of reads is associated with different candidate variants that need to be classified in large sequencing datasets, which presents challenges for processing using neural networks. The advantage of the method and system disclosed herein is that a deep learning model can be used to efficiently process and classify variable-sized datasets of sequence reads associated with different candidate variants, as further described below.

[0049] Typically, deep learning models have an "architecture" or configuration that comprises a series of neural networks connected in series and / or parallel, each consisting of multiple neural layers. Therefore, a deep learning model can be described based on the order of the layers, the operations performed by each layer, and the connectivity between layers. A key feature of a neural network layer is its "kernels" (or "filters"), which perform computational transformations on the data input to that layer and produce a transformed output. Each kernel has associated, tunable weight parameters, and the combination of kernels and their weights determines the exact computational transformation performed by that layer. The weight values ​​of the kernels are "learned" or "trained" through iterative adjustments during the minimization of a loss function. The final learned values ​​of the weight parameters, set at the end of training, determine the architecture of the deep learning model that can be used for inference tasks, such as the classification of candidate variants identified in a high-throughput sequencing dataset.

[0050] In some embodiments of the methods and systems disclosed herein, the detection of candidate variants further includes processing multiple embeddings using a deep learning model, wherein the model detects (or classifies) candidate variants. In some embodiments, the processing using the deep learning model further includes: transforming the embeddings with a series of one-dimensional convolutional layers; max- and average-pooling the outputs of the one-dimensional convolutional layer series; and transforming the max- and average-pooled outputs through a series of fully connected layers that output the detection (or classification) of candidate variants.

[0051] Figure 3 A block diagram illustrating an exemplary architecture of a deep learning model 103 is depicted, which can be used to process embeddings 102 containing candidate variant information, including multiple biopolymer read sequences, and to detect candidate variant classifications 104. In some embodiments, the deep learning model is configured to perform at least the following operations: (1) transforming the embeddings using sequences of a one-dimensional convolutional layer 301, wherein the one-dimensional convolutional layer does not incorporate information between different read sequences; (2) transforming the output of the sequence of the one-dimensional convolutional layer using a max pooling / average pooling layer 302, wherein the max pooling / average pooling layer performs max pooling and average pooling that incorporate information between different read sequences; and (3) transforming the output of the max pooling / average pooling layer 302 with a series of fully connected layers 303, wherein the output of the fully connected layers 303 is the detection of candidate variants 104.

[0052] Figure 3An exemplary sequence of three one-dimensional (or “1D”) convolutional layers 301 (also referred to herein as “Conv1D layers”) is depicted. In some embodiments of deep learning models, the number of one-dimensional convolutional layers can be 3, 4, 5, 6, 10, 25, 50, or greater. The one-dimensional convolutional layers perform 1D convolutions on the input embeddings 102 of the candidate variants. This 1D convolution results in information sharing between different embedding channels of the read sequences; the 1D convolution does not merge (or share) information between different read sequences. That is, the transformation performed by the one-dimensional convolutional layers preserves information from the different individual embeddings representing different sequence reads. This use of Conv1D layers, which does not share information from different sequence reads, is significantly different from the use of 2D convolutional layers to extract features from a stacked image of read sequences (as in deep variant models). The use of one-dimensional convolutional processing in the deep learning models of this disclosure does not require maintaining a specific order of read sequences and allows for real-time single-read processing. As described elsewhere in this article, real-time single-read processing can provide more efficient deep learning processing for variable-sized sets of read sequences associated with each of the millions of potential candidate variants in the NGS dataset.

[0053] Typically, in the methods of this disclosure, stacks of one-dimensional convolutional layers can be used, where these layers have relatively small kernel sizes. However, it is contemplated that dilation methods can be used to increase the kernel size. In some embodiments, the 1D convolutional layer has a kernel size of at least 3 (or at least a kernel size of 3×1). It is contemplated that in some embodiments, particularly in the case of read sequence embeddings with more than 20 dimensions, kernel sizes greater than 3 can be used. Therefore, in some embodiments, the kernel size of the 1D convolutional layer is at least 3, at least 5, at least 8, at least 10, at least 15, at least 30, or even larger.

[0054] For example, such as Figure 3 As described, the output of the sequence from the 1D convolutional layer 301 is then processed by layer 302 (or more layers) performing max pooling (“MaxPool”) and mean pooling (“MeanPool”). The MaxPool / MeanPool layer 302 combines information between different individual sequence reads from multiple biopolymer sequence reads. The output of layer 302, a dimensionally reduced array of the averaged and maximized values ​​across multiple embeddings from different sequence reads, is input to the sequence from the fully connected layer 303. These fully connected layers 303 transform the input of the embeddings of candidate variant information 102 processed by the previous layer sequences and output the detection of candidate variants 104.

[0055] The output detection of candidate variant 104 may include an output value that confirms whether a true variant has been observed at a specific position relative to a reference sequence based on whether it meets a predetermined threshold. Additionally, as described elsewhere herein, the detection of candidate variants can provide further information, such as the type of variant (e.g., insertion, deletion, SNP) and / or whether the variant occurs on more than one allele in the genome. Therefore, in some embodiments, the detection of candidate variants not only provides confirmation that the candidate variant is a true sequence variant, but also, for example, provides a determination of whether it is heterozygous or homozygous.

[0056] As described above, post-sequencing processing of high-throughput sequencing datasets using classic variant recall tools (e.g., GATK) can potentially identify millions of candidate variants, each at a different genomic location, each with a different number of associated sequence reads, and each read with different mapping quality and / or base quality characteristics. The large number and distinct characteristics of each candidate variant in the dataset make accurate and efficient classification difficult using deep learning. Accordingly, in some embodiments, Figure 3 An exemplary architecture of the deep learning model 103 described herein may include additional optional features that provide more accurate and efficient classification of candidate variation information.

[0057] In some embodiments, including methods that involve processing with a deep learning model, the method further includes: performing average pooling on the output from at least one of a series of one-dimensional convolutional layers, and adding the average pooled value back to the input of a subsequent one-dimensional convolutional layer. For example, as... Figure 3 As described, in some embodiments, the deep learning model 103 is further configured to transform the output of at least one of the sequences from the one-dimensional convolutional layer 301 using an intermediate average pooling layer 304 (or "intermediate average pooling layer") and add the average pooling value of that intermediate layer back to the next input in the sequence of the one-dimensional convolutional layer 301. This optional intermediate average pooling layer 304 combines information from different read sequences to generate an average. By adding these averages from different read sequences back to the output of the one-dimensional convolutional layer, which does not combine information from different read sequences, the intermediate average pooling layer 304 effectively enables the neural network to learn early in the process of processing embeddings of "consensus" values ​​from all reads. This use of intermediate average pooling 304 in the deep learning model enhances processing efficiency and can allow for early detection of anomalies in read sequences containing candidate variant information.

[0058] Despite Figure 3As described, as a single layer accepting input from a first one-dimensional convolutional layer 301, it is conceivable that, in some embodiments, intermediate average pooling can be performed between subsequent one-dimensional convolutional layers in the sequence. Furthermore, in some embodiments, it is anticipated that more than one intermediate average pooling layer 304 may be used between sequences of one-dimensional convolutional layers 301. In some embodiments, the deep learning model may be configured with intermediate average pooling layers 304 located between each pair of one-dimensional convolutional layers in the sequence.

[0059] In some embodiments of the deep learning model 103, it is conceivable that the embedding of the candidate mutation information 102 is processed in parallel, i.e., directly through the one-dimensional convolutional sequence processing layer 301 while bypassing any intermediate average pooling layer 304, and also through the intermediate average pooling layer 304. In this parallel processing embodiment, the processing with and without intermediate average pooling can be compared and analyzed for a single read to identify potential anomalous reads in the dataset. Therefore, in some embodiments, the deep learning model will be used to eliminate anomalous reads identified early using intermediate average pooling layers from further processing of the dataset.

[0060] In some embodiments, including methods that use deep learning models for processing, the method further includes: transforming the output of at least one of a series of one-dimensional convolutional layers with a series of dimensionality reduction layers that directly output to a fully connected layer. For example, as... Figure 3 As described, in some embodiments, the deep learning model 103 is also configured such that at least one output of the one-dimensional convolutional layer sequence 301 is routed to a "highway" of a dimensionality reduction layer 305 that bypasses max and average pooling 302 and outputs directly to a fully connected layer 303. The dimensionality reduction layer 305 is able to take the output of the one-dimensional convolutional layer 301, which corresponds to a single read sequence embedding, and reduce the dimension of the embedding so that it can be processed by the fully connected layer 303. This direct processing by the fully connected layer prevents mixing between different individual read sequences. This "highway" processing in the deep learning model improves processing efficiency, where the candidate mutation dataset is already very good. For example, candidate mutation sites have many overlapping read sequences with little variation between them. Furthermore, because it bypasses the average / max pooling layers, the bottleneck layer "highway" maintains its order when processing reads. This order information used for read sequence processing can be useful in further analysis or processing of the dataset.

[0061] As described above, direct input to the fully connected layer 303 requires transformation, resulting in a significant reduction in size. Therefore, in some embodiments, the dimensionality reduction layer 305 may include a "bottleneck" layer followed by a linear mapping layer that takes the embedded 2D tensor output and transforms it into a single small vector. For example, in one embodiment, the dimensionality reduction layer 305 accepts input to an exemplary 201×41 tensor corresponding to an embedding that includes candidate change information for a single read and transforms it into a single vector of size 32. The single vector output of the dimensionality reduction layer 305 then bypasses the max pooling / average pooling layer 302 and is directly input to the fully connected layer 303.

[0062] Similar to optional intermediate mean pooling layers, in some embodiments of the deep learning model 103, it is conceivable that the processing of embedding candidate mutation information 102 via the “highway” of the dimensionality reduction layer 305 can be paralleled with the processing of the same information directly via a series of one-dimensional convolutional layers 301 and subsequent max / mean pooling 302. In such parallel processing embodiments, it is analyzed that a single read information processed via the highway of the bottleneck layer can quickly identify a set of read sequences with low error rates and easy classification, without processing the entire sequence of one-dimensional convolutional layers and mean / max pooling. This good parallel early classification of candidate mutation information thus provides a basis for the overall efficiency of the deep learning model.

[0063] Figure 3 The deep learning model architecture shown exhibits surprising technical effectiveness and advantages. Through multiple one-dimensional convolutional layers in parallel with intermediate average pooling layers and / or bottleneck layers, average / maximum pooling layers process the embedding of candidate variant information, providing accurate and efficient classification of candidate variants. Furthermore, this deep learning model is advantageous because it can achieve efficient and accurate classification even when starting with suboptimal (e.g., low coverage) sequencing datasets (e.g., average coverage less than 30X, less than 20X, less than 15X, or less than 10X).

[0064] As described above, before using a deep learning model for inference tasks such as classifying candidate variables, a training process is performed, in which the model's neural network "learns" an optimal set of parameters (e.g., adjustable weight factors) that results in an optimal match between the training dataset (e.g., a suboptimal sequencing dataset) and the model's reference dataset (e.g., a standard reference genome sequence). This learning process typically involves minimizing the space of adjustable weight factors through the neural network using iterative stochastic gradient descent. Therefore, the training process usually includes error computation, followed by backpropagation of the error through the network to adjust the weighting parameters.

[0065] As envisioned in this paper, training a deep learning model to classify candidate variants can include more time-consuming initial model training, as well as less time-consuming "transfer learning" training. Transfer learning is performed on a previously trained neural network, retraining it only with data that differs in some respects from the initial training data used to train the network. Therefore, in transfer learning, the tuning of kernel weighting parameters is typically more restricted compared to initial training. In transfer training, selected weights and / or selected kernels can remain constant, while others can be tuned. In this way, transfer learning can train a neural network to work better when classifying different types of datasets without requiring a complete retraining of the neural network.

[0066] In some embodiments of the methods and systems disclosed herein, transfer learning can be performed by training a neural network of a deep learning model that has been initially trained on embeddings of a suboptimal dataset. In such embodiments, the transfer learning process is typically performed using embeddings of new and / or slightly different types of high-throughput sequencing datasets. For example, a deep learning model initially trained using embeddings of candidate variant information (including multiple biopolymer sequence reads) from one type of sequencing instrument can be retrained, at least partially, using embeddings of information obtained from a different type of instrument. Thus, the learning from the initial training based on a first type of sequencing instrument will be transferred in the training of a neural network with embeddings of sequencing data from a second type of instrument. Because the weighting parameters do not need to be determined from scratch, the training process involved in transfer learning is greatly reduced. Therefore, it is contemplated that the method of this disclosure for detecting candidate variants based on embeddings (including sequence reads) of candidate variant information will further include training a deep learning model. In some embodiments, the further training corresponds to transfer learning using embeddings of candidate variant information and embeddings of model sequencing data from different types of samples, sequencing instruments, and / or candidate variant calling software used in the original training of the learning model.

[0067] In some embodiments of the methods and systems of this disclosure, including training a deep learning model, the training includes: generating multiple embeddings of suboptimal candidate variant information, wherein the information includes multiple sequence reads and a model reference sequence; processing the embeddings using a deep learning model to detect suboptimal candidate variants; and adjusting the parameters of the deep learning model to minimize the detection error of suboptimal candidate variants relative to true candidate variants of the model reference sequence.

[0068] In some embodiments, including methods and systems for training embeddings, the methods and systems further include suboptimal candidate mutation information, which includes read sequences, model reference sequences, and positional information of the read and model reference sequences.

[0069] In some embodiments of the methods and systems of this disclosure, including training deep learning models, the embedding process further includes: transforming the embedding with a series of one-dimensional convolutional layers; performing max and average pooling on the outputs of the one-dimensional convolutional layer series; and transforming the max and average pooled outputs through a series of fully connected layers to output a detection of suboptimal candidate mutations. In some embodiments of the methods and systems including training, the embedding process further includes: average pooling outputs from at least one of the series of one-dimensional convolutional layers, and adding the average pooling values ​​back to the inputs of subsequent one-dimensional convolutional layers. Additionally, in some embodiments of the methods and systems including training, the embedding process further includes: transforming the outputs of at least one of the series of one-dimensional convolutional layers with a series of dimensionality reduction layers that directly output to fully connected layers.

[0070] Various methods disclosed herein can also be embodied in the system. It is contemplated that any implementation of the methods, and various features useful in methods for generating embeddings of multiple biopolymer sequences, as described throughout, can be used to detect candidate variants and can be incorporated as features into a non-transitory computer-readable medium. Therefore, in some embodiments, this disclosure provides a system including a processor, a storage device, and a classification engine executable on the processor according to software instructions stored in the storage device, wherein the classification engine is configured to: generate multiple embeddings comprising multiple biopolymer sequence reads; and detect one or more candidate variants in the sequence reads based at least in part on the multiple embeddings. In some embodiments of the system, the system is configured to use a deep learning model to process the embeddings, and in some embodiments of the system, the deep learning model is configured to: transform the embeddings using a series of one-dimensional convolutional layers; perform max and average pooling on the outputs of the one-dimensional convolutional layer series; and transform the max and average pooled outputs using a series of fully connected layers that output the detection of candidate variants. Additionally, in some embodiments of the system, the deep learning model may be configured to: average pool the output of at least one of a series of one-dimensional convolutional layers and add the average pooled value back to the input of a subsequent one-dimensional convolutional layer, and / or be configured to transform the output of at least one of a series of one-dimensional convolutional layers with a series of dimensionality reduction layers that directly output to a fully connected layer.

[0071] In some embodiments, the system may further include a training engine that can be executed on a processor according to software instructions stored in a storage device, wherein the training engine is configured to: generate multiple embeddings of suboptimal candidate mutation information, wherein the information includes multiple sequence reads and embeddings of a model reference sequence; process the embeddings using a deep learning model to detect suboptimal candidate mutations; and minimize the detection error when dealing with suboptimal candidate mutations relative to true candidate mutations of the model reference sequence by adjusting the parameters of the deep learning model.

[0072] Figure 4 An exemplary system and workflow for training and then using a trained deep learning model to detect candidate variations are illustrated. Embeddings 401, comprising suboptimal candidate variant information from multiple biopolymer sequence reads, and embeddings 402, of model reference sequence data are generated and fed into a deep learning model training system 403. To train a deep learning model to detect variations from suboptimal (e.g., low coverage) sequencing data, the model reference sequences used in training should include true candidate variants at relevant positions in the sequences that identify the candidate variants. Typically, the model reference sequence is a fully characterized sequence determined using a high-coverage sequencing dataset and confirmed by independently conducted experiments, thus providing the true position and identity of each monomeric unit (e.g., base pair) of the sequence with an acceptable confidence level. The embeddings 402 of the model reference sequence can thus be used as the “true” set for training the deep learning model.

[0073] In some embodiments, model reference sequences useful in training methods can be obtained from high-quality sequencing datasets (e.g., 30X or greater coverage), where the true sequence at each location has been confirmed through repeated experiments. Such high-quality sequences suitable for use as model reference sequences can be prepared for training by performing high-quality NGS experiments and analyses on known samples. In some embodiments, the model reference sequence can be a “gold standard” genome sequence from sources such as publicly available benchmark human genomes available from the “Genome in Bottles” (available at: jimb.stanford.edu / giab) or mouse genomes available from the “Mouse Genome Project” (available at: www.broadinstitute.org / mouse / mouse-genome-project).

[0074] Typically, as described elsewhere in this document, suboptimal candidate variant information is obtained from low-coverage sequencing datasets (whether experimentally obtained or simulated) using standard software tools for generating candidate variants (such as GATK). Suboptimal candidate variant information can be obtained from sequencing datasets that are of lower quality in some respects compared to the sequencing data used as model reference sequences. For example, given that model reference sequences are typically generated from sequencing datasets with an average read coverage of 30x or higher across the entire sequence, suboptimal candidate variant information can be generated from sequencing datasets with average read coverage of less than 30X, less than 20X, less than 15X, less than 12X, or even less than 10X.

[0075] The final accuracy of a trained deep learning model depends at least in part on the quality level of the model reference sequence. In some embodiments, for training, the model reference sequence and suboptimal candidate variant information are obtained from the same experiment or under the same experimental conditions. In some embodiments, suboptimal candidate variant information can be prepared from the model reference sequence sequencing data by excluding certain portions of the dataset (e.g., portions of multiple read sequences), thereby effectively creating a low-quality dataset. For example, by randomly excluding half of the sequence reads from a high-quality 30X coverage NGS dataset, it can be effectively downgraded to a suboptimal 15X coverage dataset. Other methods known in the art can also be used to simulate low-quality data from a high-quality dataset, for example, by introducing noise.

[0076] like Figure 4 As shown, the embedding 401 of the suboptimal candidate mutation information, together with the embedding 402 of the model reference sequence, is used as input to the deep learning model training system 403. The embeddings of the suboptimal and model data used in training are described elsewhere in this paper (e.g., see...). Figure 2 The same method used for embedding (and description) is employed. Training system 403 is configured to train a deep learning model to accurately detect candidate variants identified in a suboptimal sequencing dataset. The regular training process uses embeddings 401 (including multiple biopolymer sequence reads) of suboptimal candidate variant information and embeddings 402 of the model reference sequence as input to deep learning model training system 403, which includes an untrained version of the deep learning model architecture. An initial output of candidate variant detection from the untrained model is then obtained, and the error of this initial detection (or classification) of candidate variants is determined relative to the ground truth candidate variant classification of the model reference sequence. In some embodiments, error determination is performed using a loss function or similar computation that quantifies the error between the detection of suboptimal candidate variants known from the model reference sequence and the true candidate variants.

[0077] Training is performed as an iterative process, where the error from each iteration is used to adjust the model parameters, and the error is determined to be minimized when the calculated error falls below a certain threshold at the end of the iteration, and the deep learning model is considered "trained" or "optimized" for use in the inference engine 420. Therefore, the training process involves minimizing the error of the classification of suboptimal candidate variants relative to the true candidate variants of the model reference sequence by adjusting the parameters of the deep learning model.

[0078] In some embodiments, the error is minimized by stochastic gradient descent, which involves iteratively adjusting weighted parameters within a deep learning model to produce a trained deep learning model 103. A range of loss functions useful in training neural networks, such as CNNs, are known in the art and can be used in the candidate mutation classification method and system disclosed herein.

[0079] The computations and data manipulations performed within deep learning models to classify candidate variants from sequencing data exceed human capabilities. Training and using neural networks typically require high-performance, energy-efficient computer processors and are often performed using graphics processing units (GPUs) or GPU farms. Therefore, in some embodiments, this disclosure provides a computing device configured to apply deep learning models in the process of classifying candidate variants from large amounts of digital data associated with genomic sequences.

[0080] Figure 4 A block diagram 400 of an exemplary computer-based system according to embodiments of the present disclosure is also shown. This exemplary computer-based system can be used to train and use a trained neural network system to classify candidate variants from an obtained sequencing dataset. The main computer-based components include a training engine 410 and an inference engine 420. As described above, the training engine is configured to accept inputs of embeddings 401 of suboptimal candidate variant information and embeddings 402 of a model reference sequence, and is connected to a storage device 430, which may include temporary and / or permanent storage. The training engine 410 executes computer-based instructions for configuring the neural network of the deep learning model training system 403 based on the training process. Deep learning model parameters during training can be stored in the storage device 430. The primary function of the training engine 410 in conjunction with the storage device 430 is to train and provide a final trained deep learning model, which can be accessed and used by the inference engine 420 for detecting (or classifying) candidate variables.

[0081] The computer-based implementation of the inference engine 420 is configured to accept an embedding 102 of candidate variant information as input, which includes multiple biopolymer sequence reads, and process it using a trained deep learning model 103, which ultimately outputs a detection 104 of the candidate variant. The input of the embedding information 102 and the output of the candidate variant detection result 104 can be stored in the storage device 430.

[0082] In some embodiments, the inference engine 420 may be expected to provide further computer-based processing prior to the input, such as generating an embedding of candidate variant information based on information obtained from other software tools (e.g., GATK or SAMtools). It is also expected that the inference engine may provide further processing of the candidate variant detection (or classification) output, such as reassembling the classified variant sequences to provide a full-length sequence as output, which includes variants based on a complete reference sequence.

[0083] For ease of understanding, Figure 4 The various computer-based components and the functions attributed to them shown are generally described. Those skilled in the art will recognize that one or more functions belonging to the various components can be performed by any of the other components and / or by components not shown in the figures. Furthermore, it is contemplated that these components can be configured to perform combinations of the various functions supported by the various embodiments described elsewhere herein. Therefore, the description of training engine 410, inference engine 420, and storage device 430 is intended to include any suitable combination of computing devices, including servers, interfaces, systems, databases, agents, nodes, engines, controllers, modules, or other types of computing devices that operate individually or collectively to perform the functions attributed to the various components. Furthermore, those skilled in the art will recognize that the functions described herein... Figure 4 One or more functions of the system can be performed in a client-server relationship environment, such as by one or more servers, one or more client devices (e.g., one or more user devices), and / or by one or more servers and / or a combination of servers.

[0084] Figure 5An exemplary system 500 is depicted, in which embodiments of a general candidate variant detection process 100, an embedding 200 for generating candidate variant information, a deep learning model architecture 300, and / or training engine and inference engine functions 400 can be implemented. System 500 includes at least one central processing unit 501 connected to a communication bus 502. The communication bus 502 can be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect), PCI-Express, AGP (Accelerated Graphics Port), HyperTransport, or any other bus or point-to-point communication protocol. System 500 also includes a main memory 504. Control logic (software) and data are stored in the main memory 504, which may include random access memory (RAM). System 500 also includes an input device 512, a graphics processor 506, and a display 508, such as a conventional CRT (Cathode Ray Tube), LCD (Liquid Crystal Display), LED (Light Emitting Diode) display, etc. User input can be received from the input device 512, which may include, but is not limited to, a keyboard, mouse, touchpad, microphone, etc. In one embodiment, the graphics processor 506 may include multiple shader modules, rasterization modules, etc. Each of these modules may even reside on a single semiconductor platform to form a graphics processing unit (GPU). As used herein, a single semiconductor platform may refer to a single, unique semiconductor-based integrated circuit or chip, or a multi-chip module with increased connectivity that operates on analog-on-a-chip. Depending on user needs, the various modules may also be placed separately or in various combinations of semiconductor platforms. System 500 may also include auxiliary storage devices 510, such as hard disk drives and / or removable storage drives, representing floppy disk drives, magnetic tape drives, optical disk drives, digital universal disk (DVD) drives, recording devices, and Universal Serial Bus (USB) flash memory.

[0085] Software instructions, computer programs, and / or computer control logic algorithms may be stored in the system's main memory 504 and / or auxiliary memory 510. Such software instructions, when executed, allow the system 500 to perform various functions. Main memory 504, auxiliary memory 510, and / or any other memory are examples of computer-readable media.

[0086] In one embodiment, various prior Figures 1-4 The functionality and / or architecture can be implemented in the context of a central processing unit 501, a graphics processor 506, an integrated circuit (not shown) having at least a portion of the capabilities of the central processing unit 501 and the graphics processor 506, a chipset (i.e., a set of integrated circuits designed to work and sold as a unit performing related functions), and / or any other integrated circuit.

[0087] In some embodiments, Figures 1-4 The functionality and / or architecture can be implemented within the context of a general-purpose computer system, a circuit board system, a circuit board system, an application-specific system, and / or any other desired system. For example, system 500 can take the form of a desktop computer, a laptop computer, a server, a workstation, an embedded system, and / or any other type of logic. In some embodiments, system 500 can take the form of a variety of other devices, including but not limited to personal digital assistant (PDA) devices, mobile phone devices, televisions, etc.

[0088] It is also anticipated that, in some embodiments, system 500 may be coupled to a network (e.g., a telecommunications network, a local area network (LAN), a wireless network, a wide area network (WAN) such as the Internet, a peer-to-peer network, a cable network, etc.) for communication.

[0089] In some embodiments, as described in this disclosure and by Figures 1-4 For example, methods and systems for classifying candidate variants using deep learning can be implemented, controlled, and / or used remotely via the Internet or other network systems. For instance, system 500 can be hosted on one or more servers accessible to remote users and used to classify candidate variants from user sequencing datasets. Thus, in some embodiments, users can upload their own sequencing datasets and / or candidate variant information for classification using a fully trained version of deep learning model 103 hosted on the server.

[0090] Additionally, in some embodiments, users may upload their own suboptimal sequencing datasets and / or suboptimal candidate variants for further training of a deep learning model hosted on a remote server (e.g., reference learning). The user then uses the further trained deep learning model hosted on the server to classify the candidate variants generated from their own sequencing dataset. Typically, the user downloads the output variant classification for further local use; however, in some embodiments, the hosting system for variant classification may include other analytical tools, such as a model or suboptimal candidate variant information database, model reference sequences, and / or other data that can be used with the various sequence analysis methods described herein.

[0091] Typically, computing devices useful to the deep learning-based mutation detection and classification processes and systems disclosed herein can include any suitable combination of computing devices, including servers, interfaces, systems, databases, agents, nodes, engines, controllers, modules, or other types of computing devices operating individually or in combination. At least one processor of the computing device (e.g., GPU, CPU, ASIC, FPGA, DSP, x86, ARM, etc.) is configured (or programmed) to execute processes involved in the training method and classify mutations using a deep learning model stored on a computer-readable tangible, non-transitory medium (e.g., hard disk drive, solid-state drive, RAM, flash memory, ROM, etc.).

[0092] In various embodiments of the non-transitory computer-readable medium, various features useful in the methods and systems for classifying candidate variants described elsewhere herein (including, among other things, the use of embeddings, deep learning models for processing embeddings, and training methods (including transfer learning)) may also be incorporated as features into the non-transitory computer-readable medium.

[0093] Therefore, in some embodiments, this disclosure provides a non-transitory computer-readable medium including instructions for detecting candidate variants, the instructions causing a processor, when executed by a processor, to perform one or more steps, including: generating a plurality of embeddings comprising a plurality of biopolymer sequence reads; and detecting one or more candidate variants in the sequence reads at least in part based on the plurality of embeddings. In some embodiments of the non-transitory computer-readable medium, the embeddings further include sequence quality information, sequence position information, reference sequence information, and / or variant hypothesis information. In some embodiments of the non-transitory computer-readable medium, detection includes processing the embeddings using a deep learning model, and in some embodiments, processing the embeddings using a deep learning model includes: transforming the embeddings using a series of one-dimensional convolutional layers; performing max and average pooling on the outputs of the one-dimensional convolutional layer series; and transforming the max and average pooled outputs by a series of fully connected layers through the detection of candidate variants using the outputs. Additionally, in some embodiments of the non-transitory computer-readable medium, processing the embeddings using a deep learning model further includes: performing average pooling on the outputs from at least one of the series of one-dimensional convolutional layers, and adding the average pooled value back to the input of a subsequent one-dimensional convolutional layer. In some embodiments of non-transitory computer-readable media, using a deep learning model to process embeddings further includes transforming the outputs from at least one of a series of one-dimensional convolutional layers with a series of dimensionality reduction layers that directly output into a fully connected layer.

[0094] Standard programming tools can be used to prepare software instructions for configuring or programming computing devices to provide candidate variant classification capabilities. For example, specifications for VCF and BAM file formats, along with online software tools such as the SAMtools repository at github.com / samtools, can be used to program the extraction and generation of embeddings for candidate variant information from high-throughput sequencing data. Software tools such as Keras (v.2.1.3) and Tensorflow (v.1.4.0) can be used to program the architecture of deep learning models and the configuration of neural networks. Typically, software instructions are embodied as a computer program product comprising a non-transitory tangible computer-readable medium storing the instructions, which cause the processor of the computing device to perform the noise reduction steps disclosed herein.

[0095] Additionally, standard protocols and algorithms can be used to execute the methods of this disclosure for data exchange between servers, systems, databases, or interfaces during implementation. For example, protocols based on HTTP, HTTPS, AES, public-private key exchange, web service APIs, financial transaction protocols, or other electronic information exchange methods can be used for data exchange over packet-switched networks, circuit-switched networks, the Internet, LANs, WANs, VPNs (or other types of networks) for implementing the methods and systems of this disclosure.

[0096] Although the foregoing disclosure of the invention has been described in detail by way of example and illustration for purposes of clarity and understanding, the present disclosure, including the examples, descriptions, and embodiments described herein, is illustrative and should not be construed as limiting the present disclosure. Additional embodiments of the present disclosure are set forth in the appended claims.

[0097] It will also be apparent to those skilled in the art that various modifications or alterations can be made to the examples, descriptions, and embodiments described herein, and that these modifications or alterations are included within the scope of this disclosure and the appended claims. Furthermore, those skilled in the art will recognize many methods and processes equivalent to those described herein. All such equivalents should be understood to be within the scope of this disclosure and covered by the appended claims.

Claims

1. A method for sequence variation detection using deep learning, comprising: Generate multiple embeddings that include reads of multiple biopolymer sequences; Detecting one or more candidate variants in the sequence read, at least in part based on the plurality of embeddings, wherein the detection includes processing the embeddings using a deep learning model; The embedding is transformed using a series of one-dimensional convolutional layers; Max pooling and average pooling are performed on the outputs of the series of one-dimensional convolutional layers; The output of the detection of the candidate variants is transformed by a series of fully connected layers using the output of max pooling and average pooling. The output from at least one of the series of one-dimensional convolutional layers is transformed using a series of dimensionality reduction layers that directly output to the fully connected layer. The method includes training the deep learning model, wherein training includes: Multiple embeddings are generated for suboptimal candidate mutation information, wherein the information includes multiple sequence reads and the embedding of the model reference sequence; The embedding is processed using a deep learning model, which detects the suboptimal candidate variants; and By adjusting the parameters of the deep learning model, the error in detecting the suboptimal candidate variants relative to the true candidate variants of the model reference sequence is minimized.

2. The method according to claim 1, wherein the embedding further includes sequence quality information, sequence position information, reference sequence information and / or mutation hypothesis information.

3. The method according to claim 1, wherein the method further comprises: The output from at least one of the series of one-dimensional convolutional layers is averaged and pooled, and the averaged pooled value is added back to the input of the subsequent one-dimensional convolutional layer.

4. The method of claim 1, wherein processing the embedding during training comprises: The embedding is transformed using a series of one-dimensional convolutional layers; Max pooling and average pooling are performed on the outputs of the series of one-dimensional convolutional layers; as well as The outputs of max pooling and average pooling are transformed by a series of fully connected layers that use the outputs of the detection of the suboptimal candidate mutations.

5. A system for sequence variation detection using deep learning, comprising a processor, a storage device, and a classification engine executable on the processor according to software instructions stored in the storage device, wherein the classification engine is configured to: Generate multiple embeddings that include reads of multiple biopolymer sequences; and One or more candidate variants in the sequence read are detected, at least in part based on the plurality of embeddings; The system is configured to process the embedding using a deep learning model, wherein the deep learning model is configured to: The embedding is transformed using a series of one-dimensional convolutional layers; Max pooling and average pooling are performed on the outputs of the series of one-dimensional convolutional layers. The output of the detection of the candidate variants is transformed by a series of fully connected layers using the output of max pooling and average pooling. The output from at least one of the series of one-dimensional convolutional layers is transformed using a series of dimensionality reduction layers that directly output to the fully connected layer. The system further includes a training engine executable on the processor according to software instructions stored in the storage device, wherein the training engine is configured to: Multiple embeddings of suboptimal candidate variant information are generated, wherein the information includes multiple sequence reads and embeddings of the model reference sequence; The embedding is processed using a deep learning model, which detects the suboptimal candidate variants. and By adjusting the parameters of the deep learning model, the error in detecting the suboptimal candidate variant is minimized relative to the true candidate variants of the model reference sequence.

6. The system of claim 5, wherein the embedding further includes sequence quality information, sequence position information, reference sequence information, and / or mutation hypothesis information.

7. The system of claim 5, wherein the deep learning model is configured to: The output from at least one of the series of one-dimensional convolutional layers is averaged and pooled, and the averaged pooled value is added back to the input of the subsequent one-dimensional convolutional layer.

8. The system of claim 5, wherein in the training engine, the deep learning model is configured to: The embedding is transformed using a series of one-dimensional convolutional layers; Max pooling and average pooling are performed on the outputs of the series of one-dimensional convolutional layers; and The outputs of max pooling and average pooling are transformed by a series of fully connected layers that use the outputs of the detection of the suboptimal candidate mutations.

9. A non-transitory computer-readable medium comprising instructions for detecting candidate variants, the instructions, when executed by a processor, causing the processor to perform one or more steps, including: Generate multiple embeddings that include reads of multiple biopolymer sequences; Based at least in part on the plurality of embeddings, one or more candidate variants are detected in the sequence read, wherein the detection includes processing the embeddings using a deep learning model. The use of deep learning models to process the embeddings includes: The embedding is transformed using a series of one-dimensional convolutional layers; Max pooling and average pooling are performed on the outputs of the series of one-dimensional convolutional layers. The output of the detection of the candidate variants is transformed by a series of fully connected layers using the output of max pooling and average pooling. The output from at least one of a series of one-dimensional convolutional layers is transformed using a series of dimensionality reduction layers whose outputs are directly fed into a fully connected layer. The medium includes instructions for training the deep learning model, wherein training includes: Multiple embeddings are generated for suboptimal candidate mutation information, wherein the information includes multiple sequence reads and the embedding of the model reference sequence; The embedding is processed using a deep learning model, which detects the suboptimal candidate variants; and By adjusting the parameters of the deep learning model, the error in detecting the suboptimal candidate variant is minimized when the true candidate variant is compared to the model reference sequence.

10. The non-transitory computer-readable medium of claim 9, wherein the embedding further includes sequence quality information, sequence position information, reference sequence information, and / or variation hypothesis information.

11. The non-transitory computer-readable medium of claim 9, wherein processing the embedding using a deep learning model further comprises: The output from at least one of the series of one-dimensional convolutional layers is averaged and pooled, and the averaged pooled value is added back to the input of the subsequent one-dimensional convolutional layer.

Images