K-mer Database Partitioning for Memory-Efficient Taxonomic Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current k-mer databases are excessively large, making them difficult to store and load into memory on standard computers, and they lack methods to filter sequences, leading to reduced specificity in taxonomic identifications due to the inclusion of ambiguous information.
Innovation Solution
The method involves subdividing the k-mer database into groups with unique sets of nodes and removing common k-mers, or applying a taxonomic threshold to reduce database size, allowing for more specific and efficient taxonomic classifications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If k-mer databases include all available sequences to maximize discriminatory power, then identification specificity is improved, but database size becomes excessively large and cannot be stored or loaded into memory on standard computers
Solution Approach 1:
The patent divides the large k-mer database into multiple smaller partition files based on taxonomic groups (e.g., kingdoms, phyla). Each partition contains k-mers specific to certain taxonomic lineages, allowing the system to load only relevant partitions into memory for analysis. This segmentation resolves the contradiction by maintaining comprehensive database coverage while enabling practical memory usage through selective loading of partitions.
Solution Approach 2:
The patent extracts and removes ambiguous k-mers that appear in multiple taxonomic groups from the database. By identifying k-mers with low discriminatory power (those shared across multiple organisms or taxa) and excluding them, the system reduces database size while preserving only the high-specificity k-mers needed for accurate identification. This extraction process directly addresses the contradiction by eliminating redundant data that contributes to database bloat without providing proportional identification value.
2Adaptability or versatility
If k-mer databases include ambiguous sequences shared by multiple organisms, then comprehensive coverage is maintained, but false identifications increase and discriminatory power is reduced
Solution Approach 1:
The patent systematically identifies and removes ambiguous k-mers that are shared across multiple taxonomic groups. By filtering out these non-specific k-mers during database construction, the system maintains comprehensive coverage of unique, discriminatory sequences while eliminating the ambiguous ones that cause false identifications. This selective removal preserves the adaptive coverage needed for broad organism detection while significantly improving identification accuracy.
Solution Approach 2:
The patent applies different quality standards to different k-mers based on their discriminatory power. High-specificity k-mers that uniquely identify particular organisms or taxa are retained and emphasized, while ambiguous k-mers with low discriminatory value are removed. This differential treatment of k-mers based on their local quality (specificity) ensures that the database maintains comprehensive coverage where possible while prioritizing accuracy in identification-critical regions.
3Speed
If complete k-mer databases are loaded into memory for fast querying, then query speed is improved, but memory requirements exceed available resources on standard computers
Solution Approach 1:
The patent partitions the k-mer database into multiple smaller files organized by taxonomic groups, enabling the system to load only the necessary partitions into memory during querying. Instead of requiring the entire database to reside in memory simultaneously, the segmented structure allows selective loading of relevant partitions based on the sample being analyzed, thus achieving fast query performance within available memory constraints.
Solution Approach 2:
The patent performs preliminary organization of the k-mer database into taxonomic partitions during database construction, before actual querying occurs. This pre-segmentation allows the querying system to efficiently select and load only the relevant partitions needed for a given analysis, avoiding the need to load the complete database into memory. The preliminary structural preparation enables fast querying with reduced memory footprint.
Data Source
AI summary
Methods are disclosed for reducing the size of a k-mer reference database used for queries and/or taxonomic classifications when available computer storage and/or memory are inadequate. The k-mers of the reference database have been previously classified to a taxonomy, preferably based on genetic distances. In one method, the k-mers are separated into one or more groups followed by removing k-mers common to the groups. In another method, k-mers are removed based on a selected taxonomic threshold level. A third method combines the features of the previous two methods. The methods are adaptable to machine learning.