Systems and methods for grouping and collapsing sequencing reads

Abstract

To provide systems and methods for collapsing sequencing reads and identifying similar sequencing reads.SOLUTION: A method includes: generating a plurality of first identifier subsequences from a first identifier sequence of each nucleotide sequencing read; generating a first signature for the nucleotide sequencing read by applying hashing to the plurality of first identifier subsequences; assigning the nucleotide sequencing read to a first particular bin of a first data structure on the basis of the first signature; and determining a nucleotide sequence for each first particular bin of the first data structure with one or more nucleotide sequencing reads assigned thereto.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 62 / 753,786, filed October 31, 2018, the contents of which are incorporated by reference in their entirety.

[0002] Copyright Notice A portion of the disclosure of this patent document contains material subject to copyright protection. The copyright owner has no objection to anyone copying the patent document or the patent disclosure as it appears in the Patent and Trademark Office patent files and records, but otherwise reserves all copyright rights whatsoever.

[0003] background Field The present disclosure relates generally to the field of processing nucleotide sequencing data, and more particularly, to collapsing nucleotide sequencing data using locality sensitive hashing. [Background technology]

[0004] 2. Description of Related Art Read collapsing is a computational method of identifying nucleotide sequencing reads that are output from a sequencing system as originating from the same source deoxyribonucleic acid (DNA) molecule. The sequencing system can be a next-generation sequencing (NGS) system, such as the NextSseq instrument from Illumina, Inc. (San Diego, CA). Read collapsing can include using statistical methods to reduce spurious errors found in these read sets. The in silico error reduction resulting from read collapsing can be useful for applications within next-generation sequencing (NGS), such as detection of variants with ultra-low allele fractions, and in enabling enhanced variant calling specificity for clinical applications. Summary of the Invention [Means for solving the problem]

[0005] Abstract Disclosed herein is a system and method for collapsing sequencing reads and identifying similar nucleotide sequences in a plurality of different sequencing reads. In one embodiment, the system includes a non-transient memory configured to store runnable instructions and a first hash data structure for storing nucleotide sequencing reads in a plurality of bins. The system may also include a hardware processor programmed with the executable instructions to perform a method comprising: receiving a plurality of nucleotide sequencing reads (e.g., nucleotide sequencing read 1 of paired-end sequencing reads); for each nucleotide sequencing read: generating a plurality of first identifier subsequences from the first identifier sequence of the nucleotide sequencing read; generating a first signature for the nucleotide sequencing read by applying hashing to the plurality of first identifier subsequences; and assigning the nucleotide sequencing read to at least one first particular bin of the first hash data structure based on the first signature; and determining a nucleotide sequence for each first particular bin of the first hash data structure together with the assigned one or more nucleotide sequencing reads.

[0006] Another embodiment of the present invention is a computer-implemented method comprising: receiving a plurality of nucleotide sequencing reads (e.g., nucleotide sequencing reads); for each nucleotide sequencing read: generating a plurality of first identifier subsequences from a first identifier sequence of the nucleotide sequencing read; generating a first signature for the nucleotide sequencing read by applying hashing to the plurality of first identifier subsequences; and assigning the nucleotide sequencing read to a first particular bin of a first data structure based on the first signature; and determining a nucleotide sequence for each first particular bin of the first data structure together with the assigned one or more nucleotide sequencing reads.

[0007] Yet another embodiment includes a system and method for identifying similar nucleotide sequencing reads. In one example, the system includes a non-transient memory configured to store: executable instructions, a first hash data structure and a second hash data structure for storing a plurality of pairs of sequencing reads; and a hardware processor programmed by the executable instructions to perform a method including: receiving a pair of a first query nucleotide sequencing read and a second query nucleotide sequencing read; generating a plurality of first query identifier subsequences and a plurality of second query identifier subsequences from the first query nucleotide sequencing read and the second query nucleotide sequencing read, respectively. The first and second query nucleotide sequencing reads can be a pair of paired-end sequencing reads. The method includes generating a first query signature and a second query signature for the first nucleotide sequencing read and the second nucleotide sequencing read by applying hashing to the plurality of first query identifier subsequences and the plurality of second query identifier subsequences, respectively; extracting one or more first stored pairs and one or more second stored pairs from the first hashed data structure and the second hashed data structure, respectively, to generate the first query signature and the second query signature for the first nucleotide sequencing read and the second nucleotide sequencing read. The method may include recovering using a query signature, wherein each of the first pair and the second pair includes a first stored nucleotide sequencing read and a second stored nucleotide sequencing read; and determining each pair of a first stored nucleotide sequencing read and a second stored nucleotide sequencing read that are present in both the first stored pair and the second stored pair as a first sequencing read and a second sequencing read that are similar to the first query sequencing read and the second query sequencing read, respectively.

[0008] Another embodiment includes a method for generating a first query nucleotide sequencing read and a second query nucleotide sequencing read pair, the method comprising: receiving a pair of a first query nucleotide sequencing read and a second query nucleotide sequencing read; generating a plurality of first query identifier subsequences and a plurality of second query identifier subsequences from the first query nucleotide sequencing read and the second query nucleotide sequencing read, respectively; generating a first query signature and a second query signature for the first nucleotide sequencing read and the second nucleotide sequencing read, respectively, from the plurality of first query identifier subsequences and the plurality of second query identifier subsequences. and retrieving one or more first stored pairs from a first hash data structure, storing a plurality of pairs of sequencing reads using the first query signature and the second query signature, wherein each of the first pairs comprises a first stored nucleotide sequencing read and a second stored nucleotide sequencing read that are similar to the first query nucleotide sequencing read and the second query nucleotide sequencing read.

[0009] Details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. Neither this summary nor the following detailed description purports to define or limit the scope of the inventive subject matter. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram of collapsing sequencing reads. [Diagram 2]2A-2D show schematics of read grouping and collapsing based on locality sensitive hashing. The schematics include shingling (FIG. 2A), min hashing (FIG. 2B), locality sensitive hashing (LSH) insertion (FIG. 2C), and LSH query (FIG. 2D). Given a query sequencing read (not shown), a min hash or signature 232 of the query sequencing read can be generated, and the min hash 232 can be split into two chunks 232a, 232b that are used to query hash tables 1 and 2 (224a, 224b). The query sequencing read is similar, but not identical, to the sequence ACTGGAC 204 stored in hash tables 1 and 2 (224a, 224b). The minimum hash 232 of the query sequencing read is generated similarly to how the minimum hash 212 is generated for the sequencing read ACTGGAC (204) illustrated in Figures 2A and 2B. Because hash table 1 (224a) does not include one of the chunks 232a as a key 212a of the existing bin 228a, no sequencing reads similar to the query sequencing read are found in hash table 1 (224a). Because hash table 2 (224b) includes one of the chunks 232b as a key 212b of the existing bin 228b, the query sequencing read is similar to the sequencing read 204 associated with the existing bin 228b. The sequence ACTGGAC (204) stored in the existing bin 228b is then returned as a sequencing read similar to the query sequencing read. [Diagram 3] FIG. 3 shows a schematic diagram of generating a virtual universal molecular index (vUMI) for read 1 and read 2 of paired-end sequencing reads. [Figure 4] 4A and 4B show schematic diagrams for generating a k-mer (FIG. 4A) and a tiled k-mer (FIG. 4B) from a virtual UMI. [Diagram 5]FIG. 5 shows a flow diagram of binning sequencing reads via their sequences and their hashes. Each physical UMI barcode can be selected from one of 120 possible physical UMI barcodes, so that there are 120×120=14,400 combinations of physical UMI barcodes for each pair of sequencing reads 1 and 2. In the flow diagram, each pair of sequencing reads 1 and 2 can be assigned to each of the 14,400 UMI bins based on the combination of the physical UMI barcodes of the pair. The read collapsing method of the present disclosure can be applied to the sequencing reads of each UMI bin. [Figure 6] 6A-6F are exemplary plots showing that read collapsing with locality-sensitive hashing and alignment-based read collapsing have similar performance. [Figure 7] FIG. 7 is a flow diagram illustrating an exemplary method for read collapsing using locality-sensitive hashing. [Figure 8] FIG. 8 is a flow diagram illustrating an exemplary method for identifying similar reads using locality-sensitive hashing. [Figure 9] FIG. 9 is a block diagram of an exemplary computing system configured to perform read collapsing and querying with locality-sensitive hashing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Detailed Description In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, like symbols typically identify like components unless the context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made without departing from the spirit or scope of the subject matter presented herein. It is readily understood that the aspects of the present disclosure, as generally described herein and illustrated in the drawings, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations. All of these configurations are expressly contemplated herein and form part of the disclosure herein.

[0012] Overview Read collapsing is a computational method that identifies nucleotide sequencing reads that originate from the same source deoxyribonucleic acid (DNA) molecule, and then uses statistical methods to reduce false errors found in these read sets. With reference to FIG. 1, given all the reads 104+r1, 104+r2, 104-r1, 104-r2 of the same DNA molecule 108 with a plus strand 108a and a minus strand 108b, read collapsing can include grouping together those reads 104+r1, 104+r2, 104-r1, 104-r2. Read collapsing can include using statistical voting to reduce false errors, for example, using simple collapsing to determine the nucleotide sequence of a nucleotide strand (e.g., the sequence of the plus strand 108a of the DNA molecule 108). Read collapsing may involve inferring the sequence of the original DNA molecule 108 with high fidelity, for example, using double collapsing to determine the nucleotide sequence of the DNA molecule 108 from both the sequence of the plus strand 108a and the sequence of the minus strand 108b. The systems and methods disclosed herein may utilize locality sensitive hashing (LSH) and virtual identifier sequences (vID sequences) for read collapsing.

[0013] Read collapsing may generate high quality reads. Read collapsing may require samples to be sequenced with identifier sequences (ID sequences) 112a, 112b', 112a', 112b. Such identifier sequences are also referred to herein as "physical identifier sequences" (pID sequences). These identifier sequences may be Universal Molecular Index (UMI) barcodes. Although such identifier sequences 112a, 112b', 112a', 112b allow for increased resolution when distinguishing reads and molecules that may otherwise appear very similar, read collapsing may be performed without such identifier sequences under certain circumstances. Read collapsing may result in in-silica error reduction. Such error reduction may be useful for many applications within next generation sequencing (NGS).

[0014] One application of this process is the detection of variants that only exist in ultra-low allele fractions (e.g., in circulating tumor DNA (ctDNA)). Another application is enhanced variant calling specificity for clinical applications. Because read collapsing effectively combines all of the overlapping observations of DNA fragments (e.g., PCR copies of DNA fragments) into a single representative, read collapsing has the advantage of significantly reducing the amount of data that needs to be processed downstream. Removing overlapping observations, or reads, can result in a 10-fold or greater reduction in data size.

[0015] The naive read collapsing method can involve exhaustive pairwise sequence comparison, which has a runtime of O(n 2 ) is required. 2 ) is insurmountable for NGS data. For example, approximately 600 million read pairs (6 × 10 8 ) can be generated from a sample. Exhaustive pairwise sequence comparisons yield 3.6×10 17 Even at a rate of one comparison per nanosecond, this would require 3.6×10 8seconds (which is equivalent to roughly 4,167 days) are required to compute the pairwise comparisons. Even with 56 processing cores, it still takes more than two months of computation time to analyze one sample.

[0016] Conventional read collapsing methods may use a combination of alignment position and UMI barcode information to identify groups of overlapping reads. One negative aspect of these methods of read collapsing is that such methods require input reads that are already aligned and sorted. There are challenges associated with pre-processing the aligned and sorted reads before read collapsing. First, conventional read processing requires pre-processing of O(n×log(n)). Aligning "n" reads may require a runtime of O(n). Sorting "n" reads requires a runtime of O(n×log(n)). Pre-processing of aligning and sorting "n" reads requires a runtime of O(n×log(n)).

[0017] The entire process of collapsing reads using traditional methods has a lower bounded runtime of O(n×log(n)). This superlinear function means that the runtime grows faster than the input size. As larger amounts of raw sequencing data are generated, the computational cost associated with traditional read collapsing grows faster than the amount of sequencing data being processed. Furthermore, traditional read collapsing is constrained to well-characterized references. Basing read collapsing on nucleotide alignments constrains its usage to applications where high-quality reference sequences exist, because these reference sequences are necessary to enable good alignment and subsequent accurate collapsing. For example, reasonably good alignments may not be available for structural variants, repeat expansions, and repetitive genomic regions. The dependency on alignments makes it difficult to use read collapsing techniques (and UMI techniques) for novel or unknown species, limiting the generality of the error reduction techniques that UMI has enabled. Read collapse can have a large impact in these settings because there are no high-quality "references" to help identify sequencing errors.

[0018] A system and method for read collapsing using locality sensitive hashing (LSH) is disclosed herein. Conventional read collapsing approaches require alignment information because there was no good alternative for subgrouping reads that already share UMI barcodes. The LSH-based read collapsing method as disclosed herein addresses this shortcoming and removes the dependency of collapsing on having alignment, allowing for significant gains in algorithmic complexity and general applicability of read collapsing and related techniques.

[0019] In one embodiment, the LSH-based read collapsing method does not have a runtime of O(n×log(n)) and does not require well-characterized references. For example, the read collapsing method may not require any alignment information, much less sorted alignments. In one implementation, the method relies on a physical identifier sequence (pID sequence) (e.g., a physical UMI barcode (pUMI barcode)) and a virtual identifier sequence (vID sequence) (e.g., a virtual UMI barcode (vUMI barcode)) to identify groups of overlapping reads present in different nucleotide fragments. The physical identifier sequence is also referred to herein as an "identifier sequence" (ID sequence). The physical UMI barcode is also referred to herein as a "UMI barcode". A virtual identifier sequence may be a subsequence of a read that acts as a "virtual identifier sequence" to identify a group of overlapping reads. A vUMI barcode may be a subsequence of a read that acts as a "virtual" barcode to identify a group of overlapping reads. A physical identifier sequence or physical UMI barcode may be an identifier sequence or barcode that is added to a nucleotide fragment during sequencing library preparation.

[0020] In one embodiment, the method groups similar reads together and does not require any reference sequence or exhaustive sequence comparison. The method may include determining a naive grouping of a first pass of reads into bins defined by UMI barcodes (e.g., physical UMIs, virtual UMIs, or a combination thereof). The method combines virtual UMIs with locality-sensitive hashing. The method allows reads with similar sequences to be grouped together without their alignment information, and the method separates the process of read collapsing from alignment constraints. The method may include determining similar sequences, including checking other sequencing reads in the bin.

[0021] The method can be used to collapse reads from any sample (e.g., DNA or RNA), regardless of the organism from which the sample originates. Furthermore, since hashing is an O(1) constant time operation, and hashing needs to be done a fixed number of times for each of n reads, the method allows read collapsing to run in O(n) runtime. For example, hashing must be done once, twice, three times, or more times for each of n reads. Such read collapsing runtime reduces the processing time required for the increasingly large sets of data generated in NGS. The O(n) runtime allows for a significant reduction in the runtime complexity of secondary analyses, allowing for versatile application of read collapsing to any sample.

[0022] In one embodiment, a DNA sequencing instrument can implement the LSH-based read collapsing method disclosed herein. For example, the method can be implemented as an on-instrument method for in silico error reduction, because the method does not require a reference sequence for read collapsing. The method can achieve a greatly reduced error rate for all sequencing reads by leveraging the overlap rate of NGS to perform error reduction. The method can also significantly reduce the amount of sequencing data that users must process, thus increasing the accessibility of genomic analysis. The method can be utilized as an on-sequencer technique that outputs fewer high-quality reads for customers and reduces the complexity of downstream analysis.

[0023] In one embodiment, the sequencing reads are not associated with or are generated using UMI barcodes. For example, locality-sensitive hashing can be performed on the virtual UMI to group nucleotide reads. As another example, a "tiered" virtual UMI strategy can be used to mimic the binning functionality provided by physical UMI barcodes. The method can generate two types of virtual UMIs, one used as a mimic of the physical UMI and one used as the virtual UMI.

[0024] Duplicate marking is a bioinformatics method for reducing bias introduced by PCR. Disclosed herein are systems and methods for grouping similar read sequences together and creating duplicates with locality-sensitive hashing.

[0025] Lead Collapsing Disclosed herein includes systems and methods for read collapsing. In one embodiment, the method uses a virtual identifier (vID) sequence (e.g., a virtual universal molecular index (vUMI)) with locality-sensitive hashing to enable reference-free grouping of similar reads without exhaustive pairwise comparison. A virtual identifier sequence (e.g., a virtual UMI) of a sequencing read refers to any substring or subsequence within the sequencing read itself, including potentially discontinuous substrings. A virtual identifier sequence is distinct from a physical identifier (pID) sequence. A physical identifier sequence (e.g., a physical UMI (pUMI)) refers to an identifier sequence or UMI barcode that is added during sequencing library preparation.

[0026] Locality-sensitive hashing (LSH) is a computational method that places "similar" data into the same computational "bin" without performing exhaustive pairwise comparisons. Data similarity can be calculated with metrics such as Levenshtein distance, Hamming distance, or Jaccard distance. It refers to the sequence similarity of the reads. The LSH function above can "hash" the virtual UMI associated with each read, and use the result to place each read into a bin next to reads with similar virtual UMIs. LSH as applied to sequencing reads as disclosed herein allows sequencing reads with erroneous virtual UMIs to be grouped together based on the virtual UMI. Sequencing techniques often do not generate error-free sequencing reads. Therefore, being able to bin together and quickly find similar sequences that may have small variations is important to perform the read grouping required to perform read collapsing. Since small mutations in sequencing reads are often difficult to predict, a general method for grouping similar sequencing reads may be more useful than a specific method for grouping similar sequencing reads that assumes a specific mutation pattern. In some implementations, the error tolerant properties of the methods disclosed herein come into play in approximately 20% of sequencing reads. If left uncorrected, these sequencing reads may significantly affect collapsing accuracy and subsequently manifest themselves as excessive false positives in variant calling.

[0027] In one embodiment, similar sequencing reads can be identified based on virtual UMIs generated from sequencing reads, for example, by sorting the virtual UMIs lexicographically. A read collapsing method based on sorting virtual UMIs may not account for mutations in the virtual UMIs and may have a runtime complexity of O(n×log(n)). In another embodiment, similar sequencing reads can be identified using naive canonical hashing with the virtual UMIs. A read collapsing method based on naive canonical hashing may have similar performance compared to LSH-based read collapsing methods. Such read collapsing methods do not have the error-free property. In one embodiment, similar sequencing reads can be identified by clustering UMIs associated with sequencing reads. A clustering-based read collapsing method may handle slightly mismatched virtual UMI barcodes, but may have a runtime complexity of O(n 2 ) pairwise comparisons, which is significantly worse than the O(n × log(n)) runtime of traditional read collapsing methods and the O(n) runtime of LSH-based read collapsing methods.

[0028] LSH is a probabilistic data process. The probability that LSH places similar read data in the same bin is high. The probability that similar data does not fall into the same bin is small and non-zero. The likelihood that similar data does not fall into the same bin or that different data falls into the same bin may be small. In one embodiment, LSH is designed to minimize the probability that LSH places similar read data in the same bin. In one embodiment, the LSH-based read collapsing method can be configured to perform maximum recall and alignment-based checks on each item (usually less than 5 items) in each bin.

[0029] Locality-sensitive hashing applied to arrays Locality-sensitive hashing (LSH) passes each piece of data (e.g., the data's virtual UMI) through a "hash" function, the results of which are used to place the data into a bin. With LSH, similar data falls into the same (or nearby) bin, allowing for very fast queries about similar data. LSH-based read collapsing can include roofing, minimum hashing, and locality-sensitive hashing. Roofing involves collapsing the input data into overlapping sets or shingles of length k characters. Minimum hashing involves passing each "shingle" through a set of hash functions to generate a fingerprint or signature of the data. Locality-sensitive hashing involves using the fingerprints to place data into "bins" where similar data is likely to share a similar binning scheme. For example, sequencing reads can be collapsing into subsequences of length k of the sequencing read. Each subsequence can be passed through a hash function to generate a signature of the sequencing read. The signature can be used to place the sequencing read into one or more bins where similar sequencing reads are likely to share similar or identical signatures.

[0030] 2A-2D show schematics of locality sensitive hash based read grouping and collapsing, including roofing (FIG. 2A), min hashing (FIG. 2B), locality sensitive hash (LSH) insertion (FIG. 2C), and LSH query (FIG. 2D). Roofing involves moving a sliding window of k bases by m base pair increments, thus consolidating the virtual UMI 204 into k-mer "singles" 208a-208d (FIG. 2A). For example, a sliding window of 4 bases may be moved by 1 base pair increments to consolidate the virtual UMI 204 into 4-mer singles. FIG. 2A shows four singles of a sequence. The first single is GGAC (208a), the second single is TGGA (208b), the third single is CTGG (208c), and the fourth single is ACTG (208d).

[0031] Min hashing involves generating a hash "signature" 212 for a k-mer set of shingles by passing the set through several hash functions 216 and taking the minimum hash (MinHash). FIG. 2B illustrates that a set of four shingles, GGAC, TGGA, CTGG, and ACTG, may be passed through eight hash functions 216 to generate eight-element outputs 220a-d of the eight hash functions for each shingle, respectively. The number of hash functions may be 8, 16, 32, 64, 128, 256, 512, 1024, or more. The minimum number of corresponding elements of the hash outputs 220a-d may be taken to calculate the minimum number of min hashes 212. The Jaccard distance of the min hashes is an approximation of the true Jaccard distance. The more hash functions, the better the approximation.

[0032] 2C, the sequencing read 204 may be inserted into hash tables 224a, 224b based on the minimum hash 212, or a subsequence 212a, 212b of the minimum hash 212 calculated from the sequencing read 204. FIG. 2C illustrates that the sequencing read 204 may be inserted into hash tables 224a, 224b based on a subsequence 212a, 212b of the minimum hash 212. The LSH insertion of the sequencing read 204 consumes a hash "signature" 212, which then splits the signature 212 into chunks or subsequences 212a, 212b. The chunks 212a, 212b are then used as keys in the hash tables 224a, 224b, and in particular, in the bins 228a, 228b of the hash tables 224a, 224b. This partitioning and hashing scheme is adjustable for bin "wideness" and for higher recall or higher specificity. As shown in Figure 2C, the same sequencing read 204 is placed into two different bins in two different hash tables 224a, 224b. To the extent that two sequencing reads share or are stored in one or more bins, the sequencing reads can be considered similar.

[0033] FIG. 2D illustrates using LSH to determine whether a query sequencing read is similar or identical to a sequencing read stored in a hash table. Given a query sequencing read, the system passes the query sequencing read through min hashing and queries all hash tables for the min hash. The query sequence signature 232 may be split into two chunks 232a, 232b, which are queried against all hash tables 224a, 224b. Since the hash table 2 (224b) contains one of the chunks 232b as a key 212b for an existing bin 228b, the query sequence is similar or identical to the sequencing read 204 associated with (e.g., stored in) the existing bin 228b.

[0034] Locality-sensitive hashing applied to arrays A virtual UMI is a subsequence of the sequencing read itself. A virtual UMI can span the entire nucleotide read and can be a contiguous or non-contiguous subsequence. For example, a virtual UMI of a sequencing read can be 25 base pairs (bps) from the 5' end of the sequencing read. Figure 3 shows a schematic diagram of generating a virtual universal molecular index (vUMI) for read 1 (R1) and read 2 (R2) of a paired-end sequencing read. Read 1 304r1 and read 2 304r2 of a paired-end sequencing read that correspond to the plus or minus strand of the DNA fragment 308 being sequenced can be processed to generate virtual UMIs 312r1, 312r2.

[0035] 4A and 4B show schematic diagrams of generating k-mers (FIG. 4A) and tiled k-mers (FIG. 4B) from a virtual UMI. The Jaccard similarity of two sets of k-mers is the number of k-mers that both sets contain. The Jaccard similarity of two identical virtual UMIs may be 7 / 7 if seven k-mers 412 are generated for the virtual UMI. As illustrated in FIG. 4A, the Jaccard similarity of virtual UMIs 404 and 408 that are the same except for the insertion 408, i, may be 2 / 12 because all k-mers 412a that may include the insertion 408 or 3' of the insertion are affected. The use of overlapping k-mers across the virtual UMIs allows for better tolerance of insertions and deletions. As illustrated in Figure 4B, with the same insertion illustrated in Figure 4A, the Jaccard similarity of the virtual UMI 404 and the same virtual UMI 404i except for the insertion 408 is still high at 14 / 18 when 16 k-mers 412 are generated for the virtual UMI 404, 404i. The Jaccard similarity is still very high because only the k-mer 412a containing the insertion 408 is affected by the insertion 408. The greater similarity after the roofing results in more similar MinHash signatures, which in turn helps to join altered sequences in the same LSH bin.

[0036] In one embodiment, the LSH itself does not directly store or index the group of reads to be collapsed. Rather, the LSH assists in finding similar reads for a given query read. Once a similar read is found, it is stored in a conventional hash table whose key is the "centroid" read that originally started the group. In other words, the LSH data structure contains the same "keys" as the conventional hash table described above (and the two are updated in lockstep). These "keys" now are reads that act as "group anchors" to which other reads are assigned if they are similar. The LSH allows incoming reads to quickly find the keys / anchors that may match. As a result, no exhaustive checking or comparison is required. The actual groups or sequencing reads may be stored in the conventional hash table.

[0037] Alignment Score Check An alignment score check may be performed after checking the LSH bins for similar sequences. The above alignment score check ensures that dissimilar virtual UMIs are not considered equivalent or similar. A minimum alignment score for the virtual UMI may be required for the virtual UMI to be considered equal. When performing an alignment check for LSH matches, the best match may be used to determine which alignment score is above a threshold (i.e., with the highest alignment score). More mismatches (e.g., single nucleotide variants (SNVs) and insertions and deletions (indels)) will reduce the alignment score. "Sliding" the alignment reduces the number of matches, which in turn reduces the number of mismatches allowed. In one embodiment, a full alignment of the virtual UMI may be performed, which may be computationally expensive.

[0038] Dual-bin LSH structure for vUMI matching The two virtual UMIs from each read pair are "independent" barcodes, or independent measurements of the identity of the same DNA fragment. By maintaining two separate LSH data structures for each virtual UMI, false positives can be reduced. Because fewer MinHash signatures are placed into each bin, the chance of unintended mismatches is lower. By intersecting the results of the two independent queries, most of the remaining false positive hits can be removed without losing any accurate hits.

[0039] Independent LSH data structures may be used for the virtual UMI for Read 1 and / or Read 2. The LSH may be configured for very high recall with less than optimal specificity. To improve specificity, crossover between two orthogonal queries may be employed to improve specificity without a large impact on sensitivity. Separating the virtual UMI from Read 1 and Read 2 also improves specificity in more repetitive regions.

[0040] In one embodiment, there may be 64 hashes for each k-mer. Read 1 has a 64-element minHash signature and Read 2 has a 64-element minHash signature. The Read 1 and Read 2 pairs may be stored in two hash tables of a dual-bin LSH structure based on the Read 1 and Read 2 minHash signatures, respectively. The Read 1 minHash signature may be split into subsequences, so that the Read 1 and Read 2 pairs may be stored in multiple bins of one hash table and multiple bins of the other hash table. The number of hashes may vary for different implementations (e.g., 8, 64, 256, 1024, and more). More hashes may be used for more accurate data structure performance at the expense of slightly slower runtime, and fewer hashes may be used for slightly less accurate data structure performance at the expense of faster runtime. Every k-mer is hashed the same number of times, whether that be 64 times, 8 times, or n times, to ensure that the minhash signature is a consistent size.

[0041] In one embodiment, read 1 and read 2 are considered an atomic, inseparable unit, which is referenced by both the MinHash signature of the virtual UMI from read 1 and the MinHash signature of the virtual UMI from read 2 (vUMI 1 and vUMI 2). Within each hash table, the MinHash signature may be split into portions such that the atomic read 1 / read 2 pair is stored in multiple bins, once for each chuck of the signature.

[0042] An exemplary pseudocode for a dual-bin LSH structure is shown below.

number

[0043] An example pseudocode for using a dual-bin LSH structure in LSH is shown below. By inserting a pair of virtual UMIs where a similar virtual UMI is not stored in the dual-bin LSH structure, it is possible for future queries to find the inserted pair. In essence, a new "seed" is created for the family that can be matched next time.

number

[0044] Dual-bin LSH structure for vUMI matching FIG. 5 shows a flow diagram of binning of sequencing reads through their sequences and their hashes. The binning process illustrated in FIG. 5 is parallelizable, which can result in linear speedup with thousands of threads. The binning process illustrated requires intermediate caching with minimal memory usage. Each physical UMI barcode can be selected from one of 120 possible physical UMI barcodes, so that there are 120×120=14,400 combinations of physical UMI barcodes, and each pair of sequencing reads 1 and 2 can have one of the 14,400 combinations of physical UMI barcodes. LSH can be applied in parallel to the sequencing reads associated with each pair of physical UMIs.

[0045] Simple collapsing refers to collapsing all sequences that share the same physical + virtual UMI pairing in the same order. Since these sequences have their barcodes in the same order, this indicates that these sequences are not only derived from the same DNA molecule, but also from the same strand of the DNA molecule. In simple collapsing, every read in a group / family has the same first UMI, the same second UMI, the same first virtual UMI, and the same second virtual UMI. Multiple reads may meet this condition, in which case they are all considered to be reads that are derived from the same strand of the same molecule.

[0046] After simple collapsing has been performed, double collapsing can be performed. In double collapsing, given a collapsed read pair, an attempt is made to find another simple molecule with the same physical and virtual UMI pairing in reverse order - this is analogous to finding the opposite strand of the same DNA molecule. If such a double match is found, double collapsing is performed.

[0047] Since double collapsing is performed following simple collapsing, all strand-specific overlaps have been removed in simple collapsing. In double collapsing, already collapsed reads from opposite strands that are derived from the same molecule are found. For example, given a collapsed read pair with a first UMI x, a second UMI y, a first virtual UMI a, and a second virtual UMI b, double collapsing looks for the opposite strand read pair whose first UMI is y, whose second UMI is x, whose first virtual UMI is b, and whose second virtual UMI is a. The reverse complement of the opposite strand read pair may not be required due to the semantics of how reads are reported in output files such as "fastq" files. This double-layered single strand, then cross-strand collapsing allows for some advanced variant calling techniques in downstream analysis.

[0048] result The read collapsing results between the conventional alignment-based method and the above LSH / virtual UMI-based method are found to be comparable both at the level of alignment summary metrics and in terms of variant calling (e.g., structural variant calling and small variant calling). Other items in the bin can be checked to find similar nucleotide sequences. The virtual UMI-based method disclosed herein can therefore be used to collapse sequencing reads for variant calling.

[0049] 6A-6F are exemplary plots showing that read collapsing with locality-sensitive hashing and alignment-based read collapsing have similar performance on NextSeq® (Illumina, Inc., San Diego, Calif.) performed on formalin-fixed paraffin-embedded (FFPE) samples. FIG. 6A shows highly similar target region coverage produced by alignment-based read collapsing and LSH-based read collapsing (also referred to herein as "fastq-based collapsing" or "fastq read collapsing"). FIG. 6B shows noise AF, which determines the proportion of genomic loci with non-reference evidence and is an indicator of small variant calling performance. LSH-based read collapsing had similar error detection and correction capabilities as alignment collapsing. FIG. 6C shows the percentage of reads with non-linear alignment, an indicator of SV calling performance. LSH-based read collapsing generated fewer (~7%) chimeric reads. The fewer chimeric reads generated is evidence that LSH-based read collapsing can generalize better to noisy reads and therefore produces cleaner alignments. Figure 6D shows tumor mutation burden (TMB) determining mutations per megabase. This figure shows that small variant calls were highly concordant. LSH-based read collapsing has no problem separating mutation-carrying reads from "wild-type" reads even without genomic guidance. Figure 6E shows microsatellite instability determining mutations in highly repetitive regions of the genome. Highly repetitive regions are difficult regions to handle due to low sequence complexity / uniqueness. LSH-based read collapsing worked even in these low complexity regions. Figure 6F shows that LSH-based read collapsing worked well even in regions with variable copy numbers.Table 1 shows that fusion calling showed dramatically improved specificity with LSH-based read collapsing in these samples, as predicted. The above fusion calling results suggest improved handling of nonlinear reads.

[0050] Table 2 shows the improvements to fusion call specificity compared to NovaSeq TM (Illumina, Inc.) demonstrated to not negatively affect run recall. Table 1. Fusion calling false positives [Table 1] Table 2. Fusion Call Recall [Table 2]

[0051] Overall, these data show that LSH-based read collapsing compares favorably to alignment-based collapsing, and matches or exceeds existing performance in summary-level metrics and variant calling.

[0052] Lead collapsing method FIG. 7 is a flow diagram illustrating an exemplary method 700 of read collapsing using locality-sensitive hashing. Method 700 may be embedded in a set of executable program instructions stored in a computer-readable medium (e.g., one or more disk drives) of a computing system. For example, a computing system 900 shown in FIG. 9 and described in more detail below may execute a set of executable program instructions to perform method 700. When method 700 is initiated, the executable program instructions may be loaded into memory (e.g., RAM) and executed by one or more processors of computing system 900. Although method 700 is described with respect to computing system 900 shown in FIG. 9, the description is illustrative only and not intended to be limiting. In some embodiments, method 700 or portions thereof may be performed in series or in parallel by multiple computing systems. Computing system 900 may include a first hash data structure (e.g., a hash table) to sort nucleotide sequencing reads in multiple bins.

[0053] After the method 700 begins at block 704, the method 700 proceeds to block 708, where the computing system receives a plurality of first nucleotide sequencing reads. The plurality of first nucleotide sequencing reads may be associated with the same physical identifier sequence. The plurality of first nucleotide sequencing reads may be associated with any physical identifier sequence.

[0054] The method 700 proceeds from block 708 to block 712, where the computing system generates a plurality of first identifier subsequences from a first identifier sequence of each first nucleotide sequencing read. The generation of the plurality of first identifier subsequences may include generating a plurality of k-mers from a first identifier sequence of the sequencing read. The subsequences may include nucleotide insertions, nucleotide deletions, nucleotide substitutions, or combinations thereof. Two consecutive first identifier subsequences may overlap. For example, the two consecutive first identifier subsequences overlap by k-1 nucleotides. For example, the plurality of first identifier subsequences includes a plurality of 4-mers, where the first identifier sequence includes about 25 nucleotides. The first identifier sequence may be a subsequence of the sequencing read 1. The subsequence may be a contiguous subsequence of the sequencing read 1. The subsequence may be a non-contiguous subsequence of the sequencing read 1.

[0055] After generating the first identifier subsequence at block 712, the method 700 proceeds to block 716, where the computing system generates a first signature for the first nucleotide sequencing read by applying hashing to the plurality of first identifier subsequences. The first signature may match a key for a first particular bin of the first hash data structure. The first signature and the key for a first particular bin of the first hash data structure may be identical.

[0056] Generating the first signature may include determining a plurality of hashes for each first identifier subsequence. Generating the first signature may include determining each first element of the first signature from a corresponding hash of the plurality of first identifier subsequences. Each first element of the first signature may be a minimum of the corresponding hashes of the plurality of first identifier subsequences. Each first element of the first signature is a minimum, average, median, or maximum of the corresponding hashes of the plurality of first identifier subsequences.

[0057] The method proceeds from block 716 to block 720, where the computing system assigns the first nucleotide sequencing read to at least one first specific bin of the first hash data structure based on the first signature. In one embodiment, the step of assigning the first nucleotide sequencing read comprises determining a plurality of subsequences of the first signature from the first signature of the first nucleotide sequencing read; and assigning the first nucleotide sequencing read to a first specific bin of each first hash data structure of a plurality of first hash data structures based on the subsequences of the first signature. In another embodiment, the step of assigning the first nucleotide sequencing read comprises determining a plurality of subsequences of the first signature from the first signature of the first nucleotide sequencing read; and assigning the first nucleotide sequencing read to a plurality of first specific bins of the first hash data structure based on the subsequences of the first signature. The method 700 ends at block 728.

[0058] In one example, the first specific bin is a bin present in the first hashed data structure, where the alignment score of the first nucleotide sequencing read and the signature of another first nucleotide sequencing read assigned to the first specific bin of the first hashed data structure is above an alignment score threshold. In another example, the first specific bin is a bin present in the first hashed data structure, where the highest alignment score of the first nucleotide sequencing read and the signature of any first nucleotide sequencing read assigned to the first specific bin of the first hashed data structure is above an alignment score threshold. In another example, the first specific bin is a new bin of the first hashed data structure, where the alignment score of the first nucleotide sequencing read and the signature of any first nucleotide sequencing read assigned to any existing bin of the first hashed data structure is below an alignment score threshold.

[0059] After the first nucleotide sequencing reads are assigned to the first specific bin in block 720, the method 700 proceeds to block 724, where the computing system determines a nucleotide sequence for each first specific bin of the first hash data structure together with the assigned one or more first nucleotide sequencing reads. The step of determining the nucleotide sequence may include determining a consensus sequence of the one or more first nucleotide sequencing reads assigned to the first specific bin. The step of determining the consensus sequence may include determining the most frequent first nucleotide sequencing read assigned to the first specific bin as a consensus sequence of the first specific bin. The consensus sequence may include the most frequent nucleotide base for each corresponding position of the first nucleotide sequencing reads assigned to the first specific bin. The step of determining the consensus sequence may comprise determining the first nucleotide sequencing read with the highest quality score assigned to the first specific bin as the consensus sequence of the first specific bin. The highest quality score may be determined based on the quality score of each base on the first nucleotide sequencing read with the highest quality score. The step of determining the nucleotide sequence may comprise selecting the sequence of one or more first nucleotide sequencing reads assigned to the first specific bin as a representative sequence of the first specific bin. The step of determining the nucleotide sequence may comprise determining that the alignment score of two of the one or more first nucleotide sequencing reads assigned to the first specific bin exceeds an alignment score threshold.

[0060] Paired-end sequencing reads Each first nucleotide sequencing read can be associated with a second nucleotide sequencing read.The first nucleotide sequencing read and the second nucleotide sequencing read can form a paired-end nucleotide sequencing read.The computing system can determine a plurality of second identifier subsequences from the second identifier sequence of the second nucleotide sequencing read; and determine a second signature of the second nucleotide sequencing read by applying hashing to the plurality of second identifier subsequences.

[0061] The step of assigning the first nucleotide sequencing read may be different in different implementations. For example, the step of assigning the first nucleotide sequencing read may include assigning a pair of sequencing reads, including the first nucleotide sequencing read and the second nucleotide sequencing read, to a first specific bin of the first hash data structure based on the first signature. As another example, the step of assigning the first nucleotide sequencing read includes assigning the second nucleotide sequencing read to a second specific bin of the first hash data structure based on the second signature. As yet another example, the step of assigning the first nucleotide sequencing read includes assigning a pair of sequencing reads, including the first nucleotide sequencing read and the second nucleotide sequencing read, to a second specific bin of the second hash data structure based on the second signature.

[0062] As an example, the step of assigning the first nucleotide sequencing read comprises assigning a pair of sequencing reads including the first nucleotide sequencing read and the nucleotide sequencing read to a first specific bin of the first hashed data structure and a second specific bin of a second data structure based on a plurality of subsequences of a first signature of the first nucleotide sequencing read and a plurality of subsequences of a second signature of the second nucleotide sequencing read, respectively. The computing system may store the first data structure and the second data structure for storing bin keys of the first hashed data structure and bin keys of the second hashed data structure, respectively. The step of assigning the pair of sequencing reads may include determining that the first signature and the second signature are stored in the first data structure and the second data structure; and assigning the pair of sequencing reads to a first specific bin of the first hash data structure and a second specific bin of the second hash data structure using the first stored key and the second stored key, respectively. The alignment score of the pair of sequencing reads and the pair including the first sequencing read associated with the first stored key and the second sequencing read associated with the second stored key exceeds an alignment score threshold.

[0063] The step of assigning the pair of sequencing reads includes the steps of: determining one or more first keys of the first hash data structure stored in the first data structure and associated with the first signature; determining one or more second keys of the second hash data structure stored in the second data structure and associated with the second signature; determining whether a pair including a first sequencing read associated with a first stored key and a second sequencing read associated with a second stored key is associated with any first sequencing read associated with any first stored key; determining, together with said pair of sequencing reads, that the first sequencing read associated with said first stored key has the highest alignment score of any pair including said first sequencing read associated with said pair of sequencing reads and any second stored key; and assigning said pair of sequencing reads to a first specific bin of said first hash data structure and a second specific bin of said second hash data structure, respectively, with said highest alignment score using said first stored key and said second stored key associated with said pair of first sequencing read and second sequencing read. The first sequencing read associated with said first stored key may have the highest alignment score of the first sequencing read associated with any first stored key with said first signature. The second sequencing read associated with said second stored key may have the highest alignment score of the second sequencing read associated with any second stored key with said second signature.

[0064] Lead Identification FIG. 8 is a flow diagram illustrating an exemplary method 800 of identifying similar reads using locality-sensitive hashing. Method 800 may be embedded in a set of executable program instructions stored in a computer-readable medium (e.g., one or more disk drives) of a computing system. For example, a computing system 900 shown in FIG. 9 and described in more detail below may execute a set of executable program instructions to perform method 800. When method 800 is initiated, the executable program instructions may be loaded into memory (e.g., RAM) and executed by one or more processors of computing system 900. Although method 800 is described with respect to computing system 900 shown in FIG. 9, the description is intended to be illustrative only and not limiting. In some embodiments, method 800 or portions thereof may be performed in series or in parallel by multiple computing systems. The computing system may store a first hash data structure and a second hash data structure to store multiple pairs of sequencing reads.

[0065] Each pair of sequencing reads may include a first nucleotide sequencing read and a second nucleotide sequencing read, where each pair of sequencing reads is assigned to one of a plurality of first bins of the first hash data structure based on a first signature of the paired first nucleotide sequencing read generated by hashing a first identifier subsequence of the first identifier sequence of the first nucleotide sequencing read. Each pair of sequencing reads may be assigned to one of a plurality of second bins of the second hash data structure based on a second signature of the paired second nucleotide sequencing read generated by hashing a second identifier sequence of the second nucleotide sequencing read.

[0066] After method 800 begins at block 804, method 800 proceeds to block 808, where a computing system receives a pair of a first query nucleotide sequencing read and a second query nucleotide sequencing read. From block 808, method 800 proceeds to block 812, where the computing system generates a plurality of first query identifier subsequences and a plurality of second query identifier subsequences from the first query nucleotide sequencing read and the second query nucleotide sequencing read, respectively. After generating the query identifier subsequences at block 812, method 800 proceeds to block 816, where the computing system generates a first query signature and a second query signature for the first nucleotide sequencing read and the second nucleotide sequencing read by applying hashing to the plurality of first query identifier subsequences and the plurality of second query identifier subsequences, respectively. The computing system may perform the operations at blocks 808-816 as described with reference to blocks 708-716 described with reference to FIG.

[0067] After block 816, the method may include an orthogonal query. For example, the method proceeds from block 816 to block 820, where the computing system retrieves one or more first stored pairs and one or more second stored pairs from the first hash data structure and the second hash data structure, respectively, using the first query signature and the second query signature, where each of the first pair and the second pair includes a first stored nucleotide sequencing read and a second stored nucleotide sequencing read. After retrieving pairs of sequencing reads in block 820, the method 800 proceeds to block 824, where the computing system determines each pair of a first stored nucleotide sequencing read and a second stored nucleotide sequencing read that are present in both the first stored pair and the second stored pair as sequencing read 1 and sequencing read 2, as similar to the query sequencing read 1 and the query sequencing read 2. The method 800 ends at block 828.

[0068] Each pair of sequencing reads can be associated with a first identifier sequence and a second identifier sequence. The computing system can determine that the first identifier sequence and the second identifier sequence of the first pair of sequencing reads, and the second identifier sequence and the first identifier sequence of the second pair of sequencing reads, are identical; and can determine the nucleotide sequences of the first pair of sequencing reads and the second pair of sequencing reads.

[0069] In one embodiment, method 800 may include receiving a first query nucleotide sequencing read at block 808. Receiving the first query nucleotide sequencing read may include receiving a pair of the first query nucleotide sequencing read and the second query nucleotide sequencing read. Method 800 may include generating a plurality of first query identifier subsequences from the first query nucleotide sequencing read at block 812. Generating the plurality of first query identifier subsequences may include generating a plurality of second query identifier subsequences from the second nucleotide sequencing read. Method 800 may include generating a first query signature for the first nucleotide sequencing read by applying hashing to the plurality of first query identifier subsequences at block 816. The step of generating the first query signature may include generating a second query signature for the second nucleotide sequencing read by applying hashing to the plurality of second query identifier subsequences. The method 800 may include retrieving one or more first stored nucleotide sequencing reads from a first hashed data structure using the first query signature at block 820. Each of the first stored nucleotide sequencing reads may be similar to the first query nucleotide sequencing read. The step of retrieving one or more first stored nucleotide sequencing reads may include retrieving one or more first stored pairs from the first hashed data structure, storing a plurality of pairs of sequencing reads using the first query signature and the second query signature. Each of the first pairs may include a first stored nucleotide sequencing read and a second stored nucleotide sequencing read that are similar to the first query nucleotide sequencing read and the second query nucleotide sequencing read, respectively.

[0070] Execution environment FIG. 9 shows the general structure of an exemplary computing device 900 configured to execute the metabolite, annotation and gene integration system disclosed herein. The general structure of the computing device 900 shown in FIG. 9 includes an arrangement of computer hardware and software components. The computing device 900 may include more (or less) elements than those shown in FIG. 9. However, it is not necessary that all of these generally conventional elements be shown to provide an enabling disclosure. As shown, the computing device 900 includes a processing unit 940, a network interface 945, a computer-readable medium drive 950, an input / output device interface 955, a display 960, and an input device 965, all of which may communicate with each other by a communication bus. The network interface 945 may provide a connection to one or more networks or computing systems. The processing unit 940 may thus receive information and instructions over a network from other computing systems or services. The processing unit 940 may also communicate to and from memory 970, and may further provide output information to an optional display 960 via an input / output device interface 955. The input / output device interface 955 may also accept input from an optional input device 965 (e.g., a keyboard, a mouse, a digital pen, a microphone, a touch screen, a gesture recognition system, a voice recognition system, a game pad, an accelerometer, a gyroscope, or other input device).

[0071] Memory 970 may include computer program instructions (grouped as modules or components in some embodiments) that the processing unit 940 executes to perform one or more embodiments. Memory 970 typically includes RAM, ROM, and / or other persistent, secondary or non-transitory computer readable media. Memory 970 may store an operating system 972 that provides computer program instructions for use by the processing unit 940 in the general management and operation of the computing device 900. Memory 970 may further include computer program instructions and other information for performing aspects of the disclosure.

[0072] For example, in one embodiment, memory 970 includes a locality sensitive hash based read collapsing module 974 for collapsing sequencing reads using locality sensitive hashing (e.g., read collapsing method 700 described with reference to FIG. 7). Memory 970 may additionally or alternatively include a locality sensitive hash query module 976 for identifying similar nucleotide sequencing reads of a query sequencing read (e.g., identification method 800 described with reference to FIG. 8). Additionally, memory 970 may include or be in communication with a data store 990 and / or one or more other data stores that store data and results relating to read collapsing and / or similar nucleotide sequencing read identification.

[0073] Hardware Acceleration In some embodiments, the disclosed methods for grouping and collapsing sequencing reads are implemented in application-specific hardware designed or programmed to compute the disclosed methods more efficiently than a general-purpose computer processor. For example, the processing unit 940 can be a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

[0074] In one example, the locality-sensitive hashing (LSH) operation may be accelerated by an FPGA. In some embodiments, the acceleration of the LSH operation by an FPGA may depend on the memory required to build and query a hash table per UMI, and on how close the memory bandwidth is to the bottleneck in the software. If the clustering of UMIs associated with sequencing reads is performed in an FPGA, it may also be beneficial to accelerate the read collapsing method in the same hardware.

[0075] In some embodiments, one or more application-specific integrated circuits (ASICs) can be programmed to perform one or more functions of each genomic analysis module described herein, or other computers. ASICs include integrated circuits that include one or more programmable logic circuits, similar to FPGAs described herein, in that the digital logic gates of the ASICs are programmable using a hardware description language such as VHDL. However, ASICs differ from FPGAs in that ASICs are programmable only once and cannot be dynamically reconfigured once programmed. Furthermore, aspects of the present disclosure are not limited to using FPGAs or ASICs to perform grouping and collapsing of sequencing reads. Instead, any of the genomic analysis modules of the processing unit 940, or other computers, can be implemented using one or more central processing units (CPUs), graphics processing units (GPUs), or any combination thereof, that perform grouping and collapsing of sequencing reads through the execution of software instructions.

[0076] In some implementations, the use of an integrated circuit such as an FPGA, an ASIC, a CPU, a GPU, or a combination thereof to perform grouping and collapsing of sequencing reads may include a single FPGA, a single ASIC, a single CPU, a single GPU, or any combination thereof. Alternatively, or in addition, the use of an integrated circuit such as an FPGA, an ASIC, a CPU, a GPU, or a combination thereof to perform grouping and collapsing of sequencing reads may include multiple FPGAs, multiple ASICs, multiple CPUs, or multiple GPUs, or any combination thereof. The use of additional integrated circuits such as multiple FPGAs to perform grouping and collapsing of sequencing reads may reduce the amount of time it takes to perform secondary analysis operations such as mapping, alignment, P-HMM probability calculation, and variant calling. In some implementations, the use of an FPGA to perform these secondary analysis operations may reduce the time it takes to complete these secondary analysis operations from 24 hours (or more) to as little as 30 minutes (or less). In some implementations, the use of multiple FPGAs to perform these secondary analysis operations can result in completion of these secondary analysis operations in as little as five minutes.

[0077] Terminology With respect to the use of virtually any plural and / or singular term herein, those of skill in the art may interpret the plural to the singular and / or from the singular to the plural as appropriate to the context and / or application, and the various singular / plural permutations may be expressly set forth herein for purposes of clarity.

[0078] It will be understood by those of ordinary skill in the art that the terms used herein in general, and in the appended claims in particular (e.g., the body of the appended claims) are generally intended to be "open system" terms (e.g., the term "including" should be interpreted as "including but not limited to", the term "having" should be interpreted as "having at least", the term "includes" should be interpreted as "includes but is not limited to", etc.). It will be further understood by those of ordinary skill in the art that where a particular number of introduced claim recitations is intended, such intent will be expressly set forth in the claim, and in the absence of such recitation, no such intent exists. For example, as an aid to understanding, the following appended claims may include the use of the introductory phrases "at least one" and "one or more" to introduce the claim recitations. However, the use of such phrases should not be interpreted as meaning that the introduction of a claim recitation with the indefinite article "a" or "an" limits any particular claim that includes such an introduced claim recitation to embodiments that include only one such recitation, even if that same claim also includes the introductory phrase "one or more" or "at least one" and an indefinite article (e.g., "a" or "an" should be interpreted as meaning "at least one" or "one or more"); the same is true with respect to the use of definite articles used to introduce claim recitations.Furthermore, even if a particular number in an introduced claim recitation is explicitly recited, one of ordinary skill in the art will recognize that such recitation should be interpreted to mean at least the recitation number (e.g., a minimum recitation of "two recitations" means at least two recitations, or two or more recitations, without other modifiers). Furthermore, where a convention similar to "at least one of A, B, and C, etc." is used, such syntax is generally intended in the sense that one of ordinary skill in the art would understand the convention (e.g., "a system having at least one of A, B, and C" includes, but is not limited to, systems having only A, only B, only C, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). Where a convention similar to "at least one of A, B, or C, etc." is used, such syntax is generally intended in the sense that one of ordinary skill in the art would understand that convention (e.g., "a system having at least one of A, B, or C" includes, but is not limited to, systems having only A, only B, only C, A and B together, A and C together, B and C together, and / or A, B and C together, etc.). It will be further understood by one of ordinary skill in the art that virtually any disarticulated word and / or phrase expressing two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibility of including one of the terms, either of the terms, or both terms. For example, the phrase "A or B" is understood to include the possibilities of "A" or "B" or "A and B."

[0079] Moreover, when features or aspects of the disclosure are described in terms of a Markush group, those skilled in the art will recognize that the disclosure is also thereby described with respect to any individual members or subgroups of members of that Markush group.

[0080] As will be understood by those skilled in the art, for any and all purposes (e.g., for providing a written description), all ranges disclosed herein also include any and all possible subranges and combinations thereof. Any recited range can be easily recognized as fully described and allowing that same range to be at least equally separated into 1 / 2, 1 / 3, 1 / 4, 1 / 5, 1 / 10, etc. As a non-limiting example, each range discussed herein can be easily separated into a lower 1 / 3, a middle 1 / 3, and an upper 1 / 3, etc. As will be understood by those skilled in the art, all phrases such as "up to," "at least," "greater than," "less than," etc., refer to a range that includes the recited numbers and can be subsequently separated into subranges as discussed above. Finally, as will be understood by those skilled in the art, a range includes each individual member. Thus, for example, a group having 1 to 3 items refers to a group having 1, 2, or 3 items. Similarly, a group having 1 to 5 items refers to groups having 1, 2, 3, 4 or 5 items, and so forth.

[0081] While various aspects and embodiments are disclosed herein, other aspects and embodiments will be apparent to those of ordinary skill in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims

1. 1. A system for determining a nucleotide sequence from a nucleotide sequencing read, the system comprising: a non-transitory memory configured to store executable instructions and a first hash data structure for storing nucleotide sequencing reads into a plurality of bins; and A hardware processor programmed with the executable instructions to perform a method including: receiving a plurality of first nucleotide sequencing reads; For each first nucleotide sequencing read: generating a plurality of first identifier subsequences from the first identifier sequences of the first nucleotide sequencing reads; generating a first signature for the first nucleotide sequencing read by applying a hash to the plurality of first identifier subsequences; and assigning the first nucleotide sequencing read to at least one first particular bin of the first hash data structure based on the first signature; and determining a nucleotide sequence for each first particular bin of the first hash data structure along with one or more assigned first nucleotide sequencing reads; A system including:

2. The step of assigning the first nucleotide sequencing read comprises: determining a plurality of subsequences of the first signature from the first signature of the first nucleotide sequencing reads; and assigning the first nucleotide sequencing read to a first particular bin of each first hash data structure of a plurality of first hash data structures based on a subsequence of the first signature; The system of claim 1 .

3. The step of assigning the first nucleotide sequencing read comprises: determining a plurality of subsequences of the first signature from the first signature of the first nucleotide sequencing reads; and assigning the first nucleotide sequencing read to a plurality of first particular bins of the first hash data structure based on the plurality of subsequences of the first signature; The system of claim 1 .

4. 2. The system of claim 1, wherein the first particular bin is a bin present in the first hashed data structure, and an alignment score of the first nucleotide sequencing read and another first nucleotide sequencing read assigned to the first particular bin of the first hashed data structure is above an alignment score threshold.

5. 2. The system of claim 1, wherein the first particular bin is a bin present in the first hashed data structure, and a highest alignment score of the first nucleotide sequencing read and any first nucleotide sequencing read assigned to the first particular bin of the first hashed data structure is above an alignment score threshold.

6. 2. The system of claim 1, wherein the first particular bin is a new bin in the first hash data structure, and an alignment score of the first nucleotide sequencing read and any first nucleotide sequencing read assigned to any existing bin in the first hash data structure is below an alignment score threshold.

7. The system of claim 1 , wherein the first signature matches a key of the first particular bin of the first hash data structure.

8. The system of claim 1 , wherein the key for the first particular bin of the first signature and the first hash data structure are identical.

9. 2. The system of claim 1, wherein each first nucleotide sequencing read is associated with a second nucleotide sequencing read, and the first nucleotide sequencing read and the second nucleotide sequencing read form a paired-end nucleotide sequencing read.

10. 2. The system of claim 1, wherein the step of determining a nucleotide sequence comprises determining a consensus sequence of the one or more first nucleotide sequencing reads assigned to the first particular bin.

11. 11. The system of claim 10, wherein determining the consensus sequence comprises determining a first nucleotide sequencing read having a highest quality score assigned to the first particular bin as the consensus sequence for the first particular bin.

12. 2. The system of claim 1, wherein the step of determining the nucleotide sequence comprises selecting a sequence of the one or more first nucleotide sequencing reads assigned to the first particular bin as a representative sequence for the first particular bin.

13. 2. The system of claim 1, wherein the step of determining the nucleotide sequence comprises determining that an alignment score of two of the one or more first nucleotide sequencing reads assigned to the first particular bin is above an alignment score threshold.

14. 10. The system of claim 1, wherein the plurality of nucleotide sequencing reads are associated with an identical physical identifier sequence.

15. 2. The system of claim 1, wherein the plurality of nucleotide sequencing reads are not associated with any physical identifier sequences.

16. 1. A computer-implemented method for determining a nucleotide sequence from nucleotide sequencing reads, the method comprising: receiving a plurality of first nucleotide sequencing reads; For each first nucleotide sequencing read: generating a plurality of first identifier subsequences from the first identifier sequences of the first nucleotide sequencing reads; generating a first signature for the first nucleotide sequencing read by applying a hash to the plurality of first identifier subsequences; and assigning the first nucleotide sequencing read to a first particular bin in a first data structure based on the first signature; and determining a nucleotide sequence for each first particular bin of the first data structure along with one or more assigned first nucleotide sequencing reads; The method includes:

17. 17. The method of claim 16, wherein generating the plurality of first identifier subsequences comprises generating a plurality of k-mers from the first identifier sequences of the sequencing reads.

18. 18. The method of claim 17, wherein the subsequence comprises a nucleotide insertion, a nucleotide deletion, a nucleotide substitution, or a combination thereof.

19. The method of claim 17 , wherein two consecutive first identifier subsequences overlap.

20. 20. The method of claim 17, wherein the plurality of first identifier subsequences comprises a plurality of 4-mers, and the first identifier sequence comprises about 25 nucleotides.

21. 20. The method of claim 17, wherein the first identifier sequence is a subsequence of sequencing read 1.

22. 20. The method of claim 17, wherein generating the first signature comprises determining a plurality of hashes for each first identifier subsequence.

23. The method of claim 17 , wherein the first data structure comprises a hash table.

24. 1. A system for identifying similar nucleotide sequencing reads, the system comprising: Non-transient memory configured to store: Executable instructions, a first hash data structure and a second hash data structure for storing a plurality of pairs of sequencing reads; and A hardware processor programmed with the executable instructions to perform a method including: receiving a pair of a first query nucleotide sequencing read and a second query nucleotide sequencing read; generating a plurality of first query identifier subsequences and a plurality of second query identifier subsequences from said first query nucleotide sequencing reads and said second query nucleotide sequencing reads, respectively; generating first and second query signatures for said first and second nucleotide sequencing reads by applying hashing to said plurality of first query identifier subsequences and said plurality of second query identifier subsequences, respectively; retrieving one or more first stored pairs and one or more second stored pairs from the first and second hashed data structures, respectively, using the first and second query signatures, wherein each of the first and second pairs includes a first stored nucleotide sequencing read and a second stored nucleotide sequencing read; and determining each pair of a first stored nucleotide sequencing read and a second stored nucleotide sequencing read that are present in both the first stored pair and the second stored pair as a sequencing read 1 and a sequencing read 2 similar to the query sequencing read 1 and the query sequencing read 2, respectively; Including, the system.

25. 25. The system of claim 24, wherein each pair of sequencing reads includes a first nucleotide sequencing read and a second nucleotide sequencing read, and each pair of sequencing reads is assigned to one of a plurality of first bins of the first hash data structure based on a first signature of a first nucleotide sequencing read of the pair generated by hashing a first identifier subsequence of a first identifier sequence of the first nucleotide sequencing read, and each pair of sequencing reads is assigned to one of a plurality of second bins of the second hash data structure based on a second signature of a second nucleotide sequencing read of the pair generated by hashing a second identifier sequence of the second nucleotide sequencing read.

26. The hardware processor includes: For each pair of sequencing reads: generating a plurality of first identifier subsequences from a first identifier sequence of the first nucleotide sequencing read of the pair of sequencing reads; generating a first signature for the first nucleotide sequencing read by applying a hash to the plurality of first identifier subsequences; and assigning the pair of sequencing reads to at least one first particular bin of the first hash data structure based on the first signature; and determining a nucleotide sequence for each first particular bin of the first hash data structure along with one or more pairs of first and second nucleotide sequencing reads assigned from the first and second nucleotide sequencing reads of the one or more pairs; 26. The system of claim 25, programmed by the executable instructions to perform a method comprising:

27. Each pair of sequencing reads is associated with a first identifier sequence and a second identifier sequence, and the hardware processor: determining that the first identifier sequence and the second identifier sequence of a first pair of sequencing reads, and the second identifier sequence and the first identifier sequence of a second pair of sequencing reads, are identical; and determining nucleotide sequences of said first pair of sequencing reads and said second pair of sequencing reads; 26. The system of claim 25, programmed by the executable instructions to perform a method comprising:

28. 1. A method for identifying similar nucleotide sequencing reads, the method comprising: receiving a first query nucleotide sequencing read; generating a plurality of first query identifier subsequences from said first query nucleotide sequencing reads; generating a first query signature for said first nucleotide sequencing read by applying a hash to said plurality of first query identifier subsequences; and retrieving one or more first stored nucleotide sequencing reads from a first hashed data structure using the first query signature, wherein each of the first stored nucleotide sequencing reads is similar to the first query nucleotide sequencing read; The method includes:

29. The step of recovering the first query nucleotide sequencing read comprises receiving a pair of the first query nucleotide sequencing read and a second query nucleotide sequencing read, the step of generating a plurality of first query identifier subsequences comprises generating a plurality of second query identifier subsequences from the second nucleotide sequencing reads, and the step of generating a first query signature comprises generating a second query signature for the second nucleotide sequencing read by applying a hash to the plurality of second query identifier subsequences.

30. The method of claim 28, wherein the step of retrieving one or more first stored nucleotide sequencing reads comprises retrieving one or more first stored pairs from the first hash data structure; storing a plurality of pairs of sequencing reads using the first query signature and the second query signature, each of the first pairs comprising a first stored nucleotide sequencing read and a second stored nucleotide sequencing read similar to the first query nucleotide sequencing read and the second query nucleotide sequencing read, respectively.

Images