Graph Database Encoding for Scalable Triple Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graph database systems face scalability issues, limited range queries, and sub-optimal distributed processing due to inefficient storage and processing of triples, particularly when dealing with large datasets like telephone records or social network data, which require extensive memory and lack efficient mechanisms for range queries and parallel loading.
Innovation Solution
The method involves encoding each tuple-part into a unique part identifier (UPI) with a tag denoting the datatype and using immediate values for numeric data, allowing for efficient storage and sorting, and generating sorted indices to facilitate graph algorithms, enabling scalable graph database creation and pattern analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If triples are stored as strings in tables with individual indexing, then data storage is straightforward, but storage cost and processing overhead increase significantly
Solution Approach 1:
The patent changes the data representation parameter from string-based triples to a compact binary format where each triple is encoded as a fixed-size structure with type identifiers and direct value storage. This parameter transformation reduces storage overhead and eliminates the need for string comparisons while maintaining data integrity.
Solution Approach 2:
The patent extracts the type information from the string data and separates it into distinct type identifier fields. By taking out the type information and storing it separately as integers, the system eliminates the need for string-to-number mappings and reduces processing overhead associated with type checking.
2Adaptability or versatility
If range queries are implemented on numeric values, then query capability improves, but existing triple-store systems lack this functionality
Solution Approach 1:
The patent changes the storage parameter for numeric values from string representation to direct binary encoding. This allows numeric values to maintain their ordinal properties directly in storage, enabling range queries to be performed by simple binary comparisons rather than requiring complex string parsing and conversion operations.
3Ease of operation
If string-to-number mappings are used for part identifiers, then data identification is simplified, but memory requirements become excessive for large datasets
Solution Approach 1:
The patent extracts the type information from string data and stores it as separate integer type identifiers. This separation eliminates the need for string-to-number mapping tables, significantly reducing memory requirements while maintaining efficient data identification through the compact binary format.
Solution Approach 2:
Instead of using expensive string-to-number mapping tables that require extensive memory, the patent uses a disposable-like approach where type information is embedded directly in the binary triple structure as small integer fields, eliminating the need for large lookup tables.
4Productivity
If distributed processing is implemented, then processing capability scales, but existing systems lack efficient distributed processing mechanisms
Solution Approach 1:
The patent segments the triple storage into distributed components where each node stores a portion of the total triples in the same binary format. This segmentation enables parallel loading and processing across multiple machines while maintaining consistency through the unified binary representation that can be efficiently serialized and transmitted.
Data Source
AI summary
Embodiments of a method for creating a graph database which is arranged to store or process data in the form of graph tuples comprising n-parts, are described. In an embodiment, each tuple-part is encoded into a unique part identifier (hereinafter called a UPI), each UPI comprises a tag at a fixed position within the UPI. The tag indicates the datatype of the encoded tuple-part. The content data for the tuple-part is encoded in a code that is configured to reflect the ranking or order of the content data, corresponding to each datatype, relative to other tuples in a set of tuples. For content data that comprises a character-string, the code comprises a hashcode; and for content data that comprises or includes a numeric value, the code comprises an immediate value that directly stores the numeric value without encoding.