Genomic Read Compression with Indexed Class-Based Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current genome sequencing data representation methods, such as CRAM, face challenges with poor compression ratios, lack of data indexing and random access, inefficient data classification, and redundant information due to their monolithic and general-purpose compression approaches, which hinder efficient storage and processing of large genomic datasets.
Innovation Solution
A novel method that classifies and partitions genome sequencing data into distinct classes, using integrated data indexing and context-aware encoding to minimize redundant information, enabling selective access and incremental updates within the compressed domain.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If general purpose compression algorithms such as gzip are used to compress SAM format, then compression is achieved, but compression performance is poor and data structures become difficult to navigate and manipulate
Solution Approach 1:
The patent segments compressed genomic data into distinct data blocks, each representing specific types of reads (e.g., perfectly matching reads, reads with substitutions, reads with indels). This segmentation enables selective access to specific data types without decompressing the entire dataset, resolving the contradiction between compression efficiency and data manipulability.
Solution Approach 2:
The patent introduces a new dimension of organization by creating an integrated index structure that operates alongside the compressed data blocks. This index enables random access to specific genomic regions and data types, transforming the monolithic compressed structure into a multi-dimensional accessible format that maintains both compression and ease of operation.
2Quantity of substance
If CRAM format with differential encoding is used, then compression efficiency improves, but data indexing and random access capabilities are lacking
Solution Approach 1:
The patent performs preliminary classification and organization of reads into distinct data blocks before compression. This preliminary action creates a structured foundation that enables subsequent random access and indexing operations without requiring full decompression, thereby adding versatility to the compressed format while maintaining compression efficiency.
Solution Approach 2:
The patent introduces an integrated index structure as an intermediary between the compressed data blocks and the user. This index mediates access requests by translating high-level access patterns into efficient block-level operations, enabling random access and querying capabilities without compromising the compressed representation.
3Device complexity
If all reads are encoded in a monolithic format, then simplicity is maintained, but selective access to specific read classes is inefficient
Solution Approach 1:
The patent segments the monolithic encoded format into distinct data blocks, each containing reads with specific characteristics (e.g., Class 0 for perfectly matching reads, Class 1 for reads with substitutions). This segmentation enables selective access to specific read classes by operating on individual blocks rather than the entire dataset, dramatically reducing access time while maintaining encoding simplicity through consistent block-level operations.
4Ease of operation
If redundant information is retained in compressed format, then data reconstruction is simplified, but storage requirements increase
Solution Approach 1:
The patent applies local quality by encoding different types of reads with locally optimized representations. Each data block uses encoding strategies tailored to its specific content characteristics (e.g., simpler encoding for perfectly matching reads, more detailed encoding for reads with indels). This approach minimizes storage requirements by avoiding uniform redundant encoding across all read types while ensuring each block can be independently reconstructed when needed.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Method and apparatus for the compression of genome sequence data produced by genome sequencing machines. Sequence reads are coded by aligning them with respect to pre-existing or constructed reference sequences, the coding process is composed of a classification of the reads into data classes followed by the coding of each class in terms of a multiplicity of genomic descriptors. Genomic descriptors of the same type are organized in blocks which are compressed by applying successive transformation stages, binarization and entropy coding. Specific source models and entropy coders are used for each data class and for each associated descriptor.