Genomic Data Compression with Descriptor Blocks and Random Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for representing genome sequencing data, such as CRAM, face limitations including poor compression performance, lack of data indexing and random access, and inefficient data structures, which hinder efficient genomic analysis and storage.
Innovation Solution
A method that classifies and partitions genome sequencing data into distinct classes and encodes them using separate blocks of descriptors, integrating data indexing into the encoding process, reducing information source entropy, and enabling selective access and incremental updates in the compressed domain.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If CRAM format is used for compression, then compression efficiency is improved, but data indexing and random access capability deteriorates
Solution Approach 1:
The patent divides the compressed genomic data into multiple partitions, each containing data from a specific genomic region. Each partition includes its own index structure, enabling independent access to specific regions without decompressing the entire dataset. This segmentation allows random access to individual genomic regions while maintaining overall compression efficiency.
Solution Approach 2:
The patent introduces a hierarchical indexing structure that adds a new dimension to data access. Instead of linear sequential access, the system implements multi-level indexing (partition level, block level, and record level) that enables direct navigation to specific genomic regions, effectively transforming the access pattern from one-dimensional to multi-dimensional.
2Ease of manufacture
If general purpose compression algorithms like gzip are used, then ease of implementation is improved, but compression performance deteriorates
Solution Approach 1:
The patent applies different compression strategies to different types of genomic data based on their local characteristics. For example, highly repetitive regions use reference-based compression, while variable regions use different encoding schemes. This localized approach optimizes compression for each data type while maintaining overall system simplicity.
Solution Approach 2:
The patent combines multiple compression techniques into a composite compression system. It integrates reference-based compression, differential encoding, and context-adaptive arithmetic coding in a unified framework. This composite approach achieves superior compression ratios while building upon well-understood individual techniques.
3Loss of substance
If all genomic data is compressed into a single monolithic structure, then storage efficiency is improved, but data access and manipulation speed deteriorates
Solution Approach 1:
The patent divides the compressed genomic dataset into multiple partitions, each representing a specific genomic region. Each partition is independently compressed and includes its own index structures. This allows the system to access only the required partition when querying specific genomic regions, dramatically improving access speed while maintaining high overall compression ratios.
Solution Approach 2:
The patent pre-computes and stores index structures for each partition during the compression phase. These indexes include information about data locations, lengths, and characteristics, enabling rapid navigation and access without requiring full decompression. This preliminary organization of data accelerates subsequent access operations.
4Reliability
If CRAM record structure is used, then data representation is improved, but selective access to specific data classes deteriorates
Solution Approach 1:
The patent segments the data into distinct partitions based on genomic region and data type. Each partition contains homogeneous data with consistent characteristics, enabling efficient selective access. The index structures for each partition track the specific data classes present, allowing queries to target only relevant partitions and data types without inspecting unrelated records.
Data Source
AI summary
Method and apparatus for the compression of genome sequence data produced by genome sequencing machines. Sequence reads are coded by aligning them with respect to pre-existing or constructed reference sequences, the coding process is composed of a classification of the reads into data classes followed by the coding of each class in terms of a multiplicity of descriptors blocks. Specific source models and entropy coders are used for each data class in which the data is partitioned, and each associated descriptor block.