Encoded Matrix Index for Genomic Data Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Relational databases face inefficiencies in storing and querying large sequences of genomic data due to the large number of unique values, leading to storage and processing challenges, particularly in identifying specific variations within sequences.
Innovation Solution
The Encoded Matrix Index data structure is used, which stores reference values for common sequence data and additional information on variations, allowing for efficient reconstruction of sequences and enabling quick querying by reducing the amount of information needed to represent sequences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If sequence data is stored as large strings in a single column, then storage space is reduced, but querying and identifying specific variations becomes inefficient
Solution Approach 1:
The patent segments sequence data into fixed-length sub-sequences (e.g., 10-base pairs) and creates separate columns for each segment position. This allows the database to navigate directly to specific positions within sequences using integer indices, dramatically improving querying efficiency while maintaining compact storage through the structured columnar format.
2Productivity
If sequences are broken into columns or rows, then querying efficiency improves, but data structure complexity increases and processing becomes less efficient
Solution Approach 1:
The patent applies local quality by creating specialized columns for specific positions within sequences (e.g., column 0 for first 10 base pairs, column 1 for next 10 base pairs). Each column has a defined purpose and structure, allowing efficient querying of specific regions while maintaining an organized, manageable data structure that balances accessibility with complexity control.
3Ease of operation
If more columns are created to represent sequence elements, then data navigation becomes easier, but the number of columns exceeds database system limits
Solution Approach 1:
The patent introduces a third dimension by adding an integer position index that works in conjunction with the column structure. Instead of creating one column per base pair (which would exceed database limits), the system uses columns for fixed-length segments combined with integer indices to navigate to specific positions, effectively adding a dimensional layer for efficient access without overwhelming column counts.
Data Source
AI summary
A method, apparatus, and program product are provided for creating an Encoded Matrix Index for a column in a database table. An element of the column for all rows in the database table is compared to a corresponding reference value in a reference data structure, and in response to at least one value for the element of the column not matching the reference value, indicating a variation in a variation data structure and creating a value data structure. Queries executed using the Encoded Matrix Index include terms associated with a sub-column defined in a column of a database table. The variation data structure is accessed to determine whether any variation exists between rows belonging to a sub-column of the database table. If no variation exists, a value is accessed from the reference data structure; otherwise, a value for each row of the sub-column is accessed from a value data structure.