Genomic Read Compression With Class-Based Random Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for representing genome sequencing data, such as CRAM, face limitations including poor compression performance, lack of data indexing and random access, and inefficient encoding mechanisms, leading to increased storage needs and slow data processing.
Innovation Solution
A method that classifies and partitions genome sequencing data into distinct classes, using entropy coding and binarization techniques to reduce information entropy, enable selective access, and support incremental updates, with embedded indexing and context-aware encoding to improve compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If general purpose compression algorithms such as gzip are used to compress FASTQ and SAM formats, then compression is achieved, but the compression ratio is poor and the resulting monolithic binary data structure is difficult to navigate and manipulate
Solution Approach 1:
The patent segments genomic data into distinct classes (perfectly matching reads, substitution-only reads, indel reads, unmapped reads) and encodes each class separately using specialized algorithms. This segmentation enables both high compression ratios for each class and efficient random access to specific read types through embedded indexes, resolving the contradiction between compression efficiency and data accessibility.
Solution Approach 2:
The patent applies different encoding strategies tailored to each read class characteristics. For example, perfectly matching reads use position-only encoding, substitution reads use differential encoding, and indel reads use specialized coding. This local optimization of encoding quality for each data type achieves superior compression while maintaining easy access through class-specific indexes.
2Quantity of substance
If CRAM format is used with differential encoding against a reference, then compression efficiency improves, but the format lacks data indexing, random access capability, and support for incremental updates
Solution Approach 1:
The patent divides compressed data into separately indexable segments corresponding to different read classes. Each segment has its own embedded index, enabling random access to specific read types without decompressing the entire file. This segmentation also facilitates incremental updates by allowing individual class segments to be updated independently.
Solution Approach 2:
The patent creates a universal compressed genomic data format that simultaneously provides high compression efficiency, random access capability, incremental update support, and selective access to read classes. The embedded index structure and class-based organization make the format adaptable to multiple operational requirements, resolving the limitation of CRAM's single-function design.
3Device complexity
If all reads are encoded in a single monolithic structure, then encoding is simplified, but data access and processing become slow due to the need to handle large rigid data structures
Solution Approach 1:
The patent segments reads into distinct classes and encodes each class separately with its own optimization strategy. This segmentation is accompanied by embedded indexes that enable fast access to specific read types. The modular structure improves processing speed by allowing operations to target only relevant data segments rather than scanning entire monolithic structures.
4Reliability
If CRAM records encapsulate all elements necessary to reconstruct each read, then complete information is stored, but selective access to reads with specific features requires inspection of every complete record
Solution Approach 1:
The patent segments reads into classes based on their features (perfectly matching, substitutions, indels, unmapped). Each class is encoded separately with class-specific optimization. Embedded indexes allow direct access to reads with specific features by class, eliminating the need to inspect every complete record while maintaining data completeness through class-appropriate encoding.
Solution Approach 2:
The patent performs preliminary classification and organization of reads into distinct classes before encoding. This preliminary action creates an indexed structure that enables efficient selective access. Reads are pre-grouped by their characteristics, so when selective access is needed, the system can directly navigate to the relevant class without examining unrelated records.
Data Source
AI summary
Method and apparatus for the compression of genome sequence data produced by genome sequencing machines. Sequence reads are coded by aligning them with respect to pre-existing or constructed reference sequences, the coding process is composed of a classification of the reads into data classes followed by the coding of each class in terms of a multiplicity of genomic descriptors. Genomic descriptors of the same type are organized in blocks which are compressed by applying successive transformation stages, binarization and entropy coding. Specific source models and entropy coders are used for each data class and for each associated descriptor.