FASTQ Sequence Reordering for Stable Lossless Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing genome sequencing data compression technologies struggle to achieve high compression ratios and stability across varying data sizes and types due to the increasing complexity and diversity of sequencing data formats, particularly in FASTQ data, which includes nucleotide sequences and quality scores, leading to high storage costs and inefficiencies.
Innovation Solution
A method and apparatus for compressing FASTQ data through character frequency-based sequence reordering, separating data into components, measuring character frequencies, producing scores, and reordering based on these scores to improve compression ratios while preserving quality information without loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If general-purpose compression techniques are applied to FASTQ data, then compression is performed, but the compression ratio is degraded due to the special format of genome sequencing data
Solution Approach 1:
The patent segments FASTQ data into four distinct components (identifier, nucleotide sequence, separator, quality score) and processes each component separately through dedicated compression modules. This segmentation allows each module to optimize compression for its specific data type while maintaining overall format compatibility, resolving the contradiction between achieving high compression ratios and adapting to the special FASTQ format structure.
Solution Approach 2:
The patent applies different compression strategies to different parts of the FASTQ data based on their local characteristics. For example, nucleotide sequences use one compression approach while quality scores use another, with each part processed according to its specific properties. This local quality approach enables optimized compression for each component while preserving the overall data structure integrity.
2Reliability
If existing genome sequencing compression programs are used, then compression is performed for specific data types, but they fail to operate stably when data size and production platform vary
Solution Approach 1:
The patent creates a universal compression framework that can handle multiple data types and production platforms through a unified four-component processing architecture. Each compression module is designed to work with its corresponding data component regardless of the specific data type or platform origin, enabling the system to adapt to various genome sequencing data formats while maintaining stable operation. The framework processes identifiers, sequences, separators, and quality scores through consistent methods that work across different data sources.
3Quantity of substance
If quality value compression is applied to increase compression ratio, then compression ratio improves, but quality information is lost and cannot be decompressed
Solution Approach 1:
The patent creates compressed representations (copies) of the quality score data that preserve all original information while reducing storage requirements. The compression process generates alternative encodings of the quality values that can be perfectly reconstructed during decompression, ensuring no quality information is lost. This copying approach allows the system to achieve high compression ratios while maintaining complete fidelity of the quality information through lossless compression techniques.
Data Source
AI summary
A method and apparatus for decompressing FASTQ data through character frequency-based sequence reordering implemented by a computer apparatus, the method including separating genome sequencing data into components of an identifier, a nucleotide sequence read, and prediction quality information; measuring character frequency for the entire data of each of the nucleotide sequence read and the prediction quality information; producing a score by applying the measured character frequency for the nucleotide sequence read and the prediction quality information; reordering the nucleotide sequence read and the prediction quality information based on a condition that is preset based on the score; and compressing at least one of information of the identifier, an identifier of the nucleotide sequence read, and an identifier of the prediction quality information through a compression program by including the reordered nucleotide sequence read and the reordered prediction quality information and generating compressed genome sequencing data.