Token Sequence Alignment via Hash Table Region Examination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for comparing sequences, such as code binaries and text documents, are inefficient in processing speed and library capacity, particularly when dealing with out-of-order matches, which hinders rapid bulk processing and detection of similarities.
Innovation Solution
An apparatus and method utilizing processing circuitry to align token sequences with block permutations by recording n-grams in hash tables and examining regions centered on common indices to identify the largest matching regions, allowing for efficient handling of permuted matches and achieving O(N) space and O(NlogN) time complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional sequence comparison methods are used, then matching between sequences can be achieved, but processing speed and library capacity are insufficient for rapid bulk processing
Solution Approach 1:
The patent segments sequences into fixed-length substrings (k-mers) and uses hashing to segment the search space into manageable buckets. This allows parallel processing of multiple sequence pairs simultaneously, dramatically increasing throughput while maintaining manageable memory requirements for each processing unit.
Solution Approach 2:
The patent pre-computes hash values for all k-mers in the reference library before actual sequence comparison begins. This preliminary hashing action enables O(1) average-time lookups during bulk processing, eliminating the need for computationally expensive string matching operations during the actual comparison phase.
2Reliability
If conventional methods are used to detect out-of-order matches, then similarity detection is possible, but efficiency deteriorates significantly
Solution Approach 1:
The patent transforms the one-dimensional sequence matching problem into a multi-dimensional space by considering k-mer compositions, frequency distributions, and positional patterns independently of original sequence order. This dimensional transformation enables detection of out-of-order matches while maintaining computational efficiency through vector-space operations rather than exhaustive sequence alignment.
3Productivity
If rapid bulk processing is implemented, then processing speed improves, but the ability to handle out-of-order matches deteriorates
Solution Approach 1:
The patent changes the parameter representation from sequential position-based indexing to hash-based bucket indexing, where sequences are grouped by their k-mer composition rather than positional order. This parameter transformation allows the system to rapidly process bulk data while simultaneously handling permuted and out-of-order matches through hash table lookups that are insensitive to sequence ordering.
Data Source
AI summary
A method of determining matching between at least a first sample comprising a sequence of tokens A and a second sample comprising a sequence of tokens B may include, for monotonically decreasing values of n, performing operations including recording a subset SA of n-grams of A in a hash table LA, such that a value of each n-gram determines an index in LA and a location of each respective n-gram in A is recorded as the value in LA, recording a subset SB of n-grams of B in a hash table LB, such that a value of each n-gram determines an index in LB and a location of each respective n-gram in B is recorded as the value in LB, for each location L that is occupied in both LA and LB, examining a region in A centered on LA(L) and a region in B centered on LB(L), and reporting a largest matching region aligning LA(L) with LB(L) that does not include already-matched tokens in A or B and marking the largest matching region as matched.