Hierarchical Database Indexing for Storage and Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional databases face challenges in balancing indexing for fast search queries with the overhead of managing indexes, particularly when handling both structured and unstructured data, leading to increased latency and storage duplication.
Innovation Solution
The HarperDB employs a fully-indexed hierarchical database structure that uses a hierarchy of attributes and hashed cell values to provide inherent indexing without separate index files, allowing for fast read and write speeds by exploding data records into attributes and values and storing them in a way that inherently indexes all data instances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If indexes are added to a database to enhance searching, then search speed is improved, but write time increases
Solution Approach 1:
The patent merges the index structure with the data storage structure by organizing data in a hierarchical format where each attribute serves dual purposes: storing data values and providing index paths. This eliminates separate index files and reduces write overhead while maintaining fast search capabilities through the inherent hierarchical indexing.
Solution Approach 2:
Each attribute in the hierarchical database structure serves multiple functions simultaneously: it stores data values, provides indexing paths for search operations, and maintains relationships between data elements. This multi-functionality eliminates the need for dedicated index structures, reducing write time while preserving search speed.
2Adaptability or versatility
If multiple indexes are added to a database for broader searching capabilities, then query versatility is improved, but management overhead and storage needs increase
Solution Approach 1:
The hierarchical attribute structure provides universal access paths for querying any attribute at any level of the hierarchy. Each attribute automatically serves as both a data container and an index, enabling versatile queries without requiring separate index management for each query type.
Solution Approach 2:
The patent combines data storage and indexing functions into a single hierarchical structure, eliminating the need for separate index management systems. The hierarchical path structure inherently provides indexing for all attributes, reducing management overhead while maintaining query versatility.
3Speed
If a database is fully-indexed to enable fast access to any field, then search performance is improved, but storage overhead and write complexity increase
Solution Approach 1:
The hierarchical database structure merges data storage with indexing by using the hierarchical path itself as the index. Each attribute and its descendants form an inherent index structure, providing full indexing capabilities without requiring additional storage for separate index files.
Solution Approach 2:
Each element in the hierarchical structure serves dual purposes: storing data and providing index paths. This eliminates the need for duplicate storage of indexed values, reducing storage overhead while maintaining full indexing for fast access to any field.
Data Source
AI summary
This disclosure describes systems, methods, and apparatus for creating or updating a table in a fully-indexed hierarchical database for storing both structured and unstructured data, where the database structure rather than separate index files provides indexing. The database includes a hierarchy of nodes where the lowest nodes represent raw data and hard links to the raw data. Data records in a table, write request, update request, etc. can be exploded into attributes and values for those attributes, and then each value is stored in a separate file at the lowest nodes of the hierarchy.