Sequence Read Deduplication for Lossless Text-Based Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for compressing sequencing data generated by next-generation sequencing technologies are inefficient, creating large binary files that are not human-readable and often lossy, posing a bottleneck for storage and transfer, which hinders the ability of clinics and labs to perform genetic carrier screening and other research services.
Innovation Solution
A method that identifies and stores only unique sequence information from redundant sequence read files, reducing storage space and transfer costs by creating lossless, human-readable text files that can be easily processed and analyzed, using algorithms to compress FASTA and FASTQ files by storing only unique sequence reads and their meta-information in a master file.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If existing compression methods are used for sequencing data, then storage space is reduced, but the files become binary and not human-readable, and information is lost
Solution Approach 1:
The patent extracts only the unique sequence information from redundant sequencing reads, storing just one copy of each unique sequence while removing all duplicative reads. This extraction approach reduces storage space while preserving all unique genetic information without loss, as the unique sequences are stored in a reference file that can be used for all downstream analyses.
Solution Approach 2:
Instead of storing redundant copies of identical sequences, the patent creates a single master copy of each unique sequence in a reference file, and stores only metadata pointers to these unique sequences in the compressed data file. This copying strategy eliminates storage of duplicative information while maintaining data integrity and accessibility.
2Quantity of substance
If existing compression methods are used for sequencing data, then file size is reduced, but the files become specialized and wrapped into alignment programs
Solution Approach 1:
The patent creates a universal compression format that stores unique sequences in a reference file and metadata in a separate data file, which can be used with any alignment or analysis program. This multi-functional approach allows the compressed data to be compatible with various bioinformatics tools and workflows, eliminating the limitation of being wrapped into specialized alignment programs while maintaining small file sizes.
3Loss of information
If all sequence reads are stored in original format, then complete information is preserved, but storage costs and transfer bandwidth requirements increase significantly
Solution Approach 1:
The patent extracts and stores only the unique sequence information from millions of redundant reads, eliminating storage of duplicative data. By identifying and keeping just one copy of each unique sequence while removing all identical duplicates, the method preserves complete genetic information while reducing storage space requirements by eliminating redundant copies.
4Reliability
If redundant sequence reads are stored, then coverage depth is maintained, but storage and transfer costs increase
Solution Approach 1:
The patent creates a single master copy of each unique sequence in the reference file, which serves as the authoritative source for that sequence. The coverage depth is maintained through metadata that records how many times each unique sequence appears in the original data, allowing analytical programs to reconstruct the original coverage information without storing redundant physical copies of the sequences.
Data Source
AI summary
The present invention generally relates to storing sequence read data. The invention can involve obtaining a plurality of sequence reads from a sample, identifying one or more sets of duplicative sequence reads within the plurality of sequence reads, and storing only one of the sequence reads from each set of duplicative sequence reads in a text file using nucleotide characters.