Two-Level Indexed Data Structure for Latency-Free Database Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database systems face limitations in memory and disk space allocation, leading to inefficiencies in data retrieval and access speeds, particularly in multi-tenant database systems where resources are shared among multiple users, and existing solutions like Lucene database models are not optimized for latency-free searches.
Innovation Solution
Implementing a distributed memory system using a two-level indexed data structure with Most Significant Bits (MSBs) and Least Significant Bits (LSBs) to represent data identifiers, allowing for efficient intersection and union operations through binary interval reduction techniques, which reduces storage requirements and enhances query performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional database systems store entire data strings literally in physical structure, then data retrieval accuracy is maintained, but memory access speed deteriorates and storage space increases
Solution Approach 1:
The patent segments data identifiers into two parts: MSB (Most Significant Bits) and LSB (Least Significant Bits). This segmentation allows the system to store only the essential identifier components rather than entire data strings, reducing storage space while maintaining retrieval capability through the two-level indexed structure.
Solution Approach 2:
The patent creates a compressed representation (copy) of data using identifiers instead of storing the actual data strings. This copying approach stores minimal identifier information in the database while maintaining the ability to retrieve and reconstruct full data when needed, significantly reducing storage requirements.
2Speed
If more memory is allocated to database server, then data access speed improves, but hardware cost and resource consumption increase
Solution Approach 1:
The patent changes the parameter of data representation from storing complete data strings to storing compressed identifiers with MSB/LSB components. This parameter change reduces the amount of memory required to store database indices while maintaining fast access capabilities through efficient identifier comparison operations.
Solution Approach 2:
The patent introduces a two-level indexed data structure that organizes data identifiers in hierarchical dimensions. The first level stores MSB portions and the second level stores LSB portions, creating a dimensional organization that reduces storage requirements while enabling efficient multi-stage retrieval processes.
3Productivity
If distributed memory system with two-level indexed structure is implemented, then storage requirements are reduced and query performance is enhanced, but system complexity increases
Solution Approach 1:
The complex query processing is segmented into two distinct stages: first comparing MSB portions of identifiers, then comparing LSB portions for matching entries. This segmentation of the query process simplifies the overall complexity by breaking down the retrieval operation into manageable, sequential steps.
Solution Approach 2:
The system performs preliminary filtering by comparing MSB portions of data identifiers before proceeding to full identifier comparison. This preliminary action eliminates non-matching entries early in the query process, reducing the workload for subsequent processing steps and improving overall query performance.
Data Source
AI summary
Systems and methods for performing latency-free database searches using two-level indexed data structures having decreasing integer sets as identifiers to represent actual data. The indexed data structures are stored in distributed memory. Data operations such as intersection and union are performed using the indexed data structures. A binary interval reduction technique is used to quickly move through the data sets looking for common elements for the intersection set, or unique elements to add to the union set.