Thin Database Indexing Using Permutation Cycles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional database management techniques are impractical for handling large and complex Big Data sets, as they incur high overhead in storage space and read/write time due to the need for extensive indexing, which becomes prohibitively expensive and inefficient, especially in Big Data applications.
Innovation Solution
The implementation of thin database indexing techniques, which encode data as permutations and use cyclic structures with shortcuts to minimize storage overhead and reduce disk access costs, allowing for efficient retrieval of data without the need for extensive indexing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional database indexing techniques are used, then data retrieval speed is improved, but storage overhead and disk access costs increase significantly
Solution Approach 1:
The patent extracts only the essential indexing information needed for retrieval, discarding redundant index structures. By taking out only the necessary components (row IDs and selective column values) rather than maintaining complete traditional indexes, the system achieves fast retrieval with minimal storage overhead.
Solution Approach 2:
Instead of creating complete indexes for all columns, the patent applies partial indexing only to columns that are frequently queried. This selective approach provides sufficient retrieval speed for common operations while avoiding the storage costs of comprehensive indexing.
2Productivity
If extensive indexing is implemented, then query performance is improved, but read/write time overhead increases
Solution Approach 1:
The patent removes unnecessary indexing operations from the data modification process. By extracting only the minimal index update requirements (updating only affected row IDs in sparse indexes), the system maintains query performance while dramatically reducing the time overhead associated with read/write operations.
Solution Approach 2:
The patent discards redundant index maintenance operations that provide minimal query performance benefit. By eliminating these excessive indexing actions, the system recovers significant read/write time while retaining sufficient query performance through selective indexing of critical columns.
3Reliability
If conventional database management techniques are used, then data integrity is maintained, but scalability to Big Data sets is limited
Solution Approach 1:
The patent segments the indexing strategy into column-specific sparse indexes rather than using monolithic global indexes. This segmentation allows the system to scale to Big Data sets by independently managing indexes for different columns, maintaining data integrity through controlled access while adapting to large distributed data environments.
Solution Approach 2:
The patent changes the fundamental parameter of indexing density from dense (traditional) to sparse (selective). This parameter change enables scalability to Big Data by reducing the complexity and resource requirements of index management, while hash-based indexing mechanisms preserve data integrity through deterministic key-value mappings.
Data Source
AI summary
A method of indexing a database column is disclosed. A permutation function f is determined. A shortcut that connects two non-adjacent elements of a permutation cycle based on f is created. A traversal of the permutation cycle without the shortcut comprises starting at a first element of the permutation cycle by updating a candidate row number to an initial value, wherein the initial value comprises a value stored in the database column. Traversing through one element of the permutation cycle comprises reading a row value in the database column using the candidate row number and then updating the candidate row number with function f of the most recently read row value. The step of traversing through one element of the permutation cycle is repeated until the most recently read row value is equal to the initial value. Creating the shortcut comprises storing a relation between the two non-adjacent elements.