Hybrid Range Query Data Structure for Faster Substring Searching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Range query data structures, particularly binary tree-based ones, face inefficiencies in processing smaller ranges due to high logarithmic overhead and increased cache misses, leading to degraded performance for large trees and small range queries.
Innovation Solution
A hybrid range query data structure is introduced, where lower levels of the binary tree are replaced with leaf nodes storing values, and the data structure is optimized for depth-first traversal and sorting, reducing recursive calls and improving lookup time by eliminating unnecessary tree levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a full binary tree range query data structure is used, then mapping accuracy is maintained, but lookup time increases and cache misses increase for large trees
Solution Approach 1:
The binary tree is segmented into two parts: an upper binary tree portion for efficient navigation and a lower sorted array portion for direct access. This segmentation allows the system to maintain mapping accuracy through the structured binary tree while improving lookup time by using the sorted array for final value retrieval, avoiding deep tree traversal and reducing cache misses.
Solution Approach 2:
The invention introduces a hybrid structure that combines the hierarchical dimension of the binary tree with the linear dimension of a sorted array. By replacing the lower levels of the binary tree with a sorted array, the system adds a new dimensional approach to value retrieval, allowing direct access patterns that improve cache utilization and reduce lookup time while maintaining the mapping accuracy of the original binary tree structure.
2Adaptability or versatility
If a full binary tree range query data structure is used, then comprehensive mapping coverage is achieved, but memory usage increases due to tree overhead
Solution Approach 1:
The lower levels of the binary tree, which consume significant memory without providing proportional lookup benefits, are extracted and replaced with a compact sorted array. This extraction removes the redundant tree structure overhead while preserving the essential mapping coverage, as the sorted array contains all necessary values for mapping queries.
Solution Approach 2:
The invention changes the structural parameter of the data storage from a hierarchical binary tree to a hybrid structure with a sorted array component. This parameter change reduces memory consumption by eliminating the overhead of numerous tree nodes, particularly at the lower levels, while maintaining the ability to provide comprehensive mapping coverage through the sorted array.
3Measurement precision
If binary tree traversal is used for small ranges, then mapping precision is maintained, but logarithmic overhead dominates the processing time
Solution Approach 1:
Instead of performing complete binary tree traversal for all queries, the invention uses partial traversal by stopping at the upper binary tree portion and switching to linear search in the sorted array portion. This partial action approach maintains mapping precision for the traversed portion while avoiding the excessive logarithmic overhead of completing the full tree traversal, especially beneficial for small range queries.
Data Source
AI summary
A system is disclosed for faster processing of range queries. The system includes at least one processor and a memory storing a hybrid range query data structure in the form of a binary tree. The structure includes a root node, non-leaf nodes, and leaf nodes. The root node is a bitmap that includes one bit for each value in a first set of values. The non-leaf nodes are also bitmaps. The leaf nodes eliminate a number of lower levels of the tree and include two or more second values that map to values in the first set of values. The system may include instructions that cause the system to receive a range of first values, use the range of first values to traverse the data structure to identify second values that map to the first values, and return the identified second values in sorted order.