Relationship Graph Storage Using Sparse Rows, Columns, and Attributes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graph data management methods fail to efficiently handle large-scale graph data, leading to suboptimal graph analysis performance due to inefficient storage and traversal of connection relationships and attribute information.
Innovation Solution
Implementing compressed sparse row and column formats to store node identifiers, along with column storage for vertex and edge attributes, to enhance memory access efficiency and edge traversal speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional graph data storage methods are used, then storage simplicity is maintained, but graph analysis performance deteriorates due to inefficient traversal and memory access
Solution Approach 1:
The patent segments graph data into three distinct storage structures: connection relationship information (adjacency lists), node attribute information, and edge attribute information. Each segment is stored independently with optimized access patterns, allowing efficient retrieval of specific graph components without loading entire datasets into memory.
Solution Approach 2:
The patent introduces a new dimension of organization by storing attribute information in columnar format rather than traditional row-based storage. This dimensional change enables efficient aggregation and filtering operations across large datasets by allowing vectorized processing of attribute columns independent of graph structure traversal.
2Speed
If all graph data is loaded into memory for fast access, then access speed improves, but memory consumption increases excessively
Solution Approach 1:
The patent divides graph data into structure-only components (stored in memory as adjacency lists) and attribute components (stored in columnar format with selective loading). This segmentation allows the system to keep essential graph topology in fast memory while storing detailed attributes in a more compact, selectively accessible format.
Solution Approach 2:
The patent implements partial loading of attribute data based on query requirements. Instead of loading all attribute information into memory, the system loads only the specific attribute columns needed for the current analysis task, reducing memory consumption while maintaining access speed for required data.
3Reliability
If connection relationship information is stored in dense format, then storage completeness is ensured, but storage space consumption increases for sparse graphs
Solution Approach 1:
The patent extracts and stores only the non-zero elements of the adjacency matrix in connection relationship information, using adjacency lists that record only existing edges. This extraction approach maintains complete graph structure information while eliminating storage of redundant zero-value connections in sparse graphs.
Solution Approach 2:
The patent changes the representation parameter of connection relationships from a dense two-dimensional array to a sparse format using parallel arrays (one for neighbor indices, one for edge attributes). This parameter change adapts the storage efficiency to the actual density of the graph while preserving complete structural information.
4Ease of operation
If attribute information is stored in row-based format, then data organization simplicity is maintained, but memory access efficiency deteriorates for analytical queries
Solution Approach 1:
The patent transforms attribute storage from row-based (one record per graph entity) to column-based (one structure per attribute type). This dimensional transformation enables efficient vectorized access and processing of attribute data, allowing analytical queries to operate on entire attribute columns simultaneously rather than accessing individual rows sequentially.
Data Source
AI summary
A computer implemented method for graph data storage includes acquiring connection relationship information between any two nodes in a relationship network graph including a directed connecting edge between nodes. Based on the connection relationship information, a first mapping relationship between an identifier of each node and a node identifier of an outgoing edge-connected node of the node in a compressed sparse row format is stored. A second mapping relationship between the identifier of each node and a node identifier of an incoming edge-connected node of the node in a compressed sparse column format is stored. A set of attribute information in the relationship network graph is acquired, where the set of attribute information comprises several node attributes, several edge attributes, and/or several pieces of temporary information. Using column storage, storing each attribute value of a same attribute in the set of attribute information in continuous space.