Near-Storage Graph Import Using Sorted Edge and Vertex Lists
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Handling and processing sparse data in graph databases is challenging due to their irregular nature and sparse representations, leading to inefficiencies in data import and high resource requirements, which can exceed the capabilities of existing graph databases.
Innovation Solution
A method and system utilizing near-storage accelerator modules, including parsers and merge sorters, to convert graph input data into edge and vertex lists, employing B-tree vertex maps for efficient ID translation and sorting, thereby generating a graph database with improved performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If graph input data is imported using conventional methods, then the graph database can be generated, but the resource requirements are high and the import performance is poor
Solution Approach 1:
The patent segments the data import process into distinct phases: parsing phase (converting input data to edge and vertex lists), sorting phase (organizing vertices and edges sequentially), and database generation phase. This segmentation allows each phase to be optimized independently and processed efficiently using near-storage computing, thereby improving overall import performance while reducing resource requirements.
Solution Approach 2:
The patent introduces intermediate data structures (edge lists, vertex lists, and their sorted versions) as mediators between the raw input data and the final graph database. These intermediate structures enable efficient processing by organizing data in a manner that reduces computational complexity during database generation, thus lowering resource requirements while maintaining high productivity.
2Quantity of substance
If sparse data is handled in relational databases, then data can be stored, but large amounts of storage are occupied by null values
Solution Approach 1:
The patent extracts sparse data elements from traditional relational table structures and represents them using graph database entities (nodes and edges). By taking out the essential connectivity information and representing it through explicit edge and vertex lists, the system eliminates the need to store numerous null values while preserving the quantitative relationships in the data.
Solution Approach 2:
Instead of forcing sparse data into dense relational table structures (traditional approach), the patent inverts the approach by using graph database structures that naturally accommodate sparsity. The edge and vertex list representations allow sparse relationships to be stored efficiently without requiring dense matrix structures, thereby improving storage efficiency.
3Adaptability or versatility
If graph data is processed with irregular structures, then the data can be represented, but processing becomes challenging and inefficient
Solution Approach 1:
The patent applies preliminary action by pre-sorting the vertex list and edge list before database generation. Vertices are sorted to create a sequential list, and edges are sorted based on vertex IDs. This preliminary organization transforms irregular graph data into structured sequences that can be processed efficiently, thereby maintaining adaptability for various graph types while significantly improving processing efficiency.
Solution Approach 2:
The patent changes the parameter organization of graph data by transforming unordered edge and vertex collections into sorted sequences with specific ordering parameters. Edges are reordered based on vertex ID sequences, and vertices are arranged in deterministic order. This parameter reorganization maintains the flexibility to represent diverse graph structures while enabling efficient processing through predictable data access patterns.
Data Source
AI summary
Methods and memory devices are provided in which at least one parser of a memory device converts graph input data into an edge list and a vertex list for a graph database. A merge sorter of the memory device sorts the vertex list to generate a sequential list of vertices. The edge list is converted into a translated list of edges using identifiers (IDs) of the sequential list of vertices. The merge sorter sorts the translated list of edges to generate a sequential list of edges. The graph database is generated using the sequential list of edges.