Compressed Genomic Sequence Indexing for Direct Similarity Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current genomic data analysis techniques face computational bottlenecks due to the exponential growth of genomic data, as they require decompression for analysis, which does not effectively utilize redundancy in similar genomes, leading to inefficient processing times for similarity searches.
Innovation Solution
The approach involves compressing genomic data to preserve structural similarities and differences, allowing direct computation on the compressed data, using a two-step process of pre-processing to create a compressed database and search operation that performs coarse and fine searches without full decompression, leveraging redundancy for computational acceleration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If genomic data is compressed to reduce storage space, then storage efficiency is improved, but computational analysis requires full decompression which increases processing time
Solution Approach 1:
The patent segments the compressed genomic data into distinct components: a compressed index structure and compressed sequence data. This segmentation allows different parts of the data to be processed differently - the index can be searched in compressed form while only specific sequence portions need decompression, thus resolving the contradiction between storage efficiency and processing speed.
Solution Approach 2:
The patent performs preliminary compression of the genomic data into an optimized format with an index structure before analysis. This preliminary action creates a data structure that enables efficient searching and retrieval without requiring complete decompression, allowing both storage efficiency and computational speed to be improved.
2Speed
If compression schemes are designed to accelerate inexact search, then search speed is improved, but the compression scheme must respect similarity metrics which increases complexity
Solution Approach 1:
The patent applies local quality by using different compression strategies for different parts of the genomic data. The index structure uses one type of compression optimized for search operations, while the sequence data uses another compression method. This localized approach enables efficient inexact search without requiring a single complex compression scheme for the entire dataset.
Solution Approach 2:
The patent changes parameters of the compression scheme to optimize for inexact search by preserving similarity metrics in the compressed representation. By adjusting compression parameters to maintain distance relationships and similarity information, the system enables fast search operations while keeping the compression scheme manageable in complexity.
3Loss of information
If computational analysis runs on full genomic library, then analysis completeness is improved, but processing time scales linearly with library size which decreases efficiency
Solution Approach 1:
The patent extracts and separates the essential searchable features into a compressed index structure, removing redundant information from the full genomic library. This extraction allows analysis to be performed on the condensed index representation, maintaining analysis completeness for the target queries while dramatically improving processing efficiency by avoiding redundant computations on identical sequence regions.
Solution Approach 2:
The patent performs preliminary organization of the genomic library into a compressed indexed structure that pre-processes the data for efficient querying. This preliminary action creates a data structure that enables complete analysis to be performed on compressed representations, maintaining information completeness while achieving sub-linear scaling with library size.
Data Source
Figure 1~2
Figure 3
Figure 4a~4c
AI summary
The redundancy in genomic sequence data is exploited by compressing sequence data in such a way as to allow direct computation on the compressed data using methods that are referred to herein as "compressive" algorithms. This approach reduces the task of computing on many similar genomes to only slightly more than that of operating on just one. In this approach, the redundancy among genomes is translated into computational acceleration by storing genomes in a compressed format that respects the structure of similarities and differences important to analysis. Specifically, these differences are the nucleotide substitutions, insertions, deletions, and rearrangements introduced by evolution. Once such a compressed library has been created, analysis is performed on it in time proportional to its compressed size, rather than having to reconstruct the full data set every time one wishes to query it.