Thin Database Indexing Using Cyclic Structures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional database management techniques are impractical for handling large and complex Big Data sets, as they incur high overhead in storage space and read/write time due to the need for extensive indexing, which becomes prohibitively expensive and inefficient in Big Data applications.
Innovation Solution
The implementation of thin database indexing techniques, which encode data as permutations and use cyclic structures with shortcuts to minimize storage space and disk access, allowing for efficient retrieval of data by traversing cycles and using conversion tables to reduce disk operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional database indexing techniques are used to enable efficient data retrieval, then query speed is improved, but storage space requirements increase significantly
Solution Approach 1:
The patent extracts only the essential indexing information needed for query resolution, storing minimal metadata (such as column minima, maxima, and null counts) rather than complete traditional indexes. This selective extraction enables efficient query processing while dramatically reducing the storage overhead associated with comprehensive indexing structures.
Solution Approach 2:
Instead of organizing data with traditional indexes that prioritize fast retrieval at the cost of storage, the patent inverts the approach by using columnar storage with minimal metadata. The system retrieves data by scanning optimized columnar structures rather than by traversing extensive index trees, thereby achieving acceptable query performance with fraction of the storage requirements.
2Adaptability or versatility
If extensive indexing is implemented to handle complex queries efficiently, then query capability is improved, but read/write overhead increases
Solution Approach 1:
The columnar storage structure inherently provides query optimization capabilities without requiring complex external indexing mechanisms. Each column's metadata (minima, maxima, null counts) enables the system to self-determine query eligibility and skip unnecessary data processing, thereby reducing read/write overhead while maintaining versatile query handling.
Solution Approach 2:
The minimal metadata structure serves multiple functions simultaneously: it enables range query filtering, supports NULL value handling, facilitates data compression, and provides statistics for query optimization. This multi-functionality achieves comprehensive query capability without the proliferation of specialized index structures that would increase overhead.
3Stability of the object's composition
If traditional indexing structures are used to manage large data sets, then data organization is improved, but system complexity increases
Solution Approach 1:
The patent segments data into independent columns rather than organizing by rows with associated indexes. Each column is stored as a separate, self-contained structure with its own metadata, eliminating the need for complex inter-related index structures. This segmentation simplifies the overall system architecture while maintaining effective data organization for analytical queries.
Data Source
AI summary
A method of database indexing is disclosed. Two functions (f and g) from row values to row number values are set. The functions are utilized to determine a row number in a database column containing a target search value, wherein the target search value comprises a search value being sought in the database column. A candidate row number variable is set initially to the function g of the target search value by a processor. Iteratively the following is performed: a current value of the candidate row number variable is used as an address to read a value in a corresponding row in the database column, and the current value of the candidate row number variable is updated to the function f of the most recently read value in a corresponding row in the database column.