Hash-Based Secondary Indexing for Graph Data in Non-Relational Stores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Non-relational data stores, such as sparse data maps, lack support for secondary indexing, leading to inefficient querying and data retrieval, as row-key-based queries require scanning all data to find matching rows, which is resource-intensive and ineffective for finding data based on specific criteria.
Innovation Solution
A database management system implements secondary indexing by hashing key-value pairs from sparse data maps to create row-keys for a separate secondary index map, allowing for efficient lookup and storage of references to matching data, using different hash functions for keys and values and incorporating sequential bucket identifiers to distribute references across multiple buckets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If row-key-based queries are used in non-relational data stores, then data can be stored and accessed using simple primary indexing, but querying data based on specific criteria requires scanning all data which is resource-intensive
Solution Approach 1:
The patent segments the indexing function by creating separate secondary index structures for different query criteria. Instead of relying on a single row-key index, the system divides the indexing task into multiple specialized indexes, each optimized for specific query patterns, thereby improving query efficiency without complicating the core storage structure
Solution Approach 2:
The patent introduces secondary indexes as intermediary structures between the primary row-key storage and query operations. These secondary indexes act as mediators that pre-organize data according to specific criteria, allowing efficient query resolution without scanning all data while maintaining the simplicity of the underlying storage structure
2Reliability
If all rows are scanned to find matching data, then complete data coverage is achieved, but resource consumption and query time increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing secondary indexes during data insertion or batch processing. This pre-organization of data according to query criteria allows the system to retrieve relevant data quickly without scanning all rows, thus reducing query execution time while maintaining complete data coverage through the pre-built index structures
3Productivity
If secondary indexes are implemented in non-relational data stores, then query performance improves, but system complexity increases due to additional indexing structures
Solution Approach 1:
The patent implements universality by designing secondary index structures that can serve multiple query patterns simultaneously. The same secondary indexing mechanism handles various query types (range queries, equality queries, prefix queries) across different data fields, improving data retrieval speed without proportionally increasing system complexity through specialized structures for each query type
Data Source
AI summary
A method and apparatus for a graph database instance (GDI) maintaining a secondary index, that indexes data from a sparse data map storing graph application data, within a sparse data map dedicated to the secondary index. The GDI formulates row-keys, for the secondary index map, by hashing the values of key/value pairs stored in rows of a map storing application data. The GDI stores for each formulated row-key, in the row of the secondary index that is indexed by the formulated row-key, references to rows of the map storing application data that match the key/value pair on which formulation of the row-key was based. The row-keys into the secondary index map may incorporate bucket identifiers, which, for each key/value pair, allows the GDI to spread the references to graph elements that match the key/value pair among a set number of “buckets” for the key/value pair within the secondary index map.