Genomic Data Compression via Secondary Character Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current genomic data compression techniques are inefficient for large volumes of genomic data, particularly in fastq format, due to high compression and decompression times, memory requirements, and lack of optimality for variable read-length sequences, which hinders quick storage, retrieval, and transmission.
Innovation Solution
A method and system for compressing genomic data files by identifying and removing secondary characters, modifying quality scores, and encoding primary characters as bits, while using run-length encoding for quality sequences, resulting in a lossless compressed file with improved compression ratios and processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional compression techniques are used for genomic data, then storage space is reduced, but compression time and decompression time increase significantly
Solution Approach 1:
The patent segments the genomic data processing into distinct phases: identifying primary characters (A, C, G, T), removing secondary characters, encoding primary characters as bits, and applying run-length encoding to quality sequences. This segmentation allows each processing stage to be optimized independently, achieving high compression ratios without excessive compression time
Solution Approach 2:
The patent changes the representation parameters of genomic data by converting character-based sequences into bit-based sequences (reducing from 8 bits per character to 2 bits per nucleotide), and transforms quality scores using run-length encoding. These parameter changes enable more efficient storage while maintaining fast processing speeds
2Quantity of substance
If conventional compression techniques are used for genomic data, then storage space is reduced, but memory requirements increase
Solution Approach 1:
The patent extracts and removes secondary characters from genomic sequences, keeping only the essential primary characters (A, C, G, T) for compression. This extraction reduces the data volume that needs to be processed and stored in memory, lowering memory requirements while still achieving effective compression of the essential genomic information
3Quantity of substance
If genomic data is compressed for storage, then storage costs are reduced, but decompression speed decreases
Solution Approach 1:
The patent performs preliminary organization of the compressed data structure by separating primary character data from quality score data and applying different encoding schemes to each. This preliminary structuring enables faster decompression because the decoding process can efficiently reconstruct the original data without complex processing, achieving both high compression ratios and fast decompression speeds
4Ease of manufacture
If standard compression formats are used for genomic data, then data storage is simplified, but compression optimality for variable read-length sequences is poor
Solution Approach 1:
The patent applies different compression strategies to different parts of the genomic data: primary characters are encoded as bits with 2 bits per nucleotide, while quality sequences undergo run-length encoding. This local quality approach optimizes compression for each data type independently, achieving superior overall compression ratios compared to uniform compression methods, while maintaining straightforward implementation
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for compression of a genomic data file are described herein. In one embodiment, genomic sequences, sequence headers, and quality sequences associated with a plurality of data streams provided in a genomic data file are identified. Each of the genomic sequences includes at least one of primary characters and secondary characters. Further, the secondary characters from each of the genomic sequences may be removed to obtain an intermediate genomic sequence file and a quality score corresponding to the secondary character may be modified in quality sequences to obtain an intermediate quality sequence file. Based on the intermediate genomic sequence file and the intermediate quality sequence file, a modified genomic sequence file and a modified quality sequence file, respectively are generated. A compressed genomic data file is obtained using at least the modified genomic sequence and the modified quality sequence.