Genomic Sequence Encoding With Dynamic Segmentation Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Storing and analyzing genomic information has become impractical due to increasing data sizes, processing time, and the need for custom decoders in conventional compression techniques, especially as the number of sequenced genomes and catalogued variations grows.
Innovation Solution
The system employs dynamic alphabet coding (DAC) to encode genomic data by dividing it into segments based on unique characters, using the smallest number of bits per character necessary, and embedding the length value within the encoded information, allowing for efficient compression and preservation of accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If genomic data is stored using conventional compression techniques, then storage space is reduced, but custom decoders are required and processing time increases
Solution Approach 1:
The patent segments genomic data into fixed-size blocks (e.g., 64 or 128 bytes) and applies independent Huffman coding to each block. This segmentation eliminates the need for complex custom decoders while maintaining compression efficiency, as each block can be decoded independently using simple lookup tables.
Solution Approach 2:
The patent changes the encoding parameter from variable-length codes requiring complex state management to fixed-block Huffman coding with predefined lookup tables. This parameter change simplifies the decoder architecture while preserving compression ratios, directly addressing the contradiction between storage efficiency and decoder complexity.
2Adaptability or versatility
If genomic data is stored using 8-bit per character format to include all characters, then all characters can be represented, but storage efficiency decreases
Solution Approach 1:
The patent applies local quality by using fixed 8-bit blocks for storage (ensuring all characters can be represented) while applying Huffman compression locally within each block to achieve efficient storage. This allows the system to maintain full character representation capability while significantly improving storage efficiency through local compression.
3Loss of information
If more genomes are sequenced and variations are catalogued, then genomic information completeness increases, but processing time and storage requirements become impractical
Solution Approach 1:
The patent segments large genomic datasets into manageable fixed-size blocks, enabling parallel processing and efficient storage. This segmentation allows the system to handle increasingly complete genomic information without proportionally increasing processing time, as each block can be processed independently and efficiently.
4Quantity of substance
If conventional compression techniques are used, then data size is reduced, but prior knowledge of byte requirements is needed
Solution Approach 1:
The patent performs preliminary action by pre-defining Huffman coding lookup tables for fixed-size blocks during system initialization. This preliminary preparation eliminates the need for runtime calculations of byte requirements, making the compression and decompression operations simple and straightforward while maintaining efficient data sizing.
Data Source
AI summary
Genomic data is written to disk in a compact format by dividing the data into segments and encoding each segment with the smallest number of bits per character necessary for whatever alphabet of characters appears in that segment. A computer system dynamically chooses the segment boundaries for maximum space savings. A first one of the segments may use a different number of bits per character than a second one of the segments. In one embodiment, dividing the data into segments comprises scanning the data and keeping track of a number of unique characters, noting positions in the sequence where the number increases to a power of two, calculating a compression that would be obtained by dividing the genomic data into one of the plurality of segments at ones of the noted positions, and dividing the genomic data into the plurality of segments at the positions that yield the best compression.