Graph Data Storage via Entity Node Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large knowledge graphs with entity and non-entity nodes face storage and computing bottlenecks due to uneven hot spot distribution, leading to inefficiencies in data processing and resource utilization in centralized and distributed storage scenarios.
Innovation Solution
A method for storing graph data by grouping entity nodes into sub-tables and distributing them across multiple computing units, along with corresponding sub-tables for node pairs, to balance data distribution and reduce hot spots, thereby enabling efficient distributed computing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If graph data is stored in a centralized manner, then data storage is simple, but hot spot aggregation occurs leading to computing bottlenecks
Solution Approach 1:
The patent segments graph data into two separate tables: a first table storing entity node information and a second table storing node pair information. This segmentation allows independent optimization of each table's storage and access patterns, eliminating the hot spot aggregation that occurs in centralized graph storage while maintaining data integrity and enabling parallel processing across multiple computing units.
2Productivity
If graph data is distributed across multiple computing units, then hot spot aggregation is reduced, but data distribution balance is poor
Solution Approach 1:
The patent applies local quality by storing entity nodes and their associated node pairs in the same computing unit partition. This ensures that data access patterns remain localized to specific partitions, maintaining data distribution balance while enabling efficient distributed computing. Each computing unit handles its local data independently, avoiding the need for frequent cross-partition data access.
3Speed
If entity nodes and node pairs are stored together, then data access is efficient, but storage flexibility is reduced
Solution Approach 1:
The patent segments graph data into two independent tables: entity node information and node pair information. This segmentation provides storage flexibility by allowing each table to be optimized and accessed independently based on specific query requirements, while still enabling efficient data access through the established table structures and access methods.
Data Source
AI summary
The present specification relates to the field of data processing, and in particular, to a storage method for graph data and a distributed computing method for graph data. The graph data includes nodes and edges, the nodes include entity nodes and non-entity nodes, and the storage method for the graph data includes: obtaining a first table and a second table based on the graph data; obtaining first sub-tables and second sub-tables respectively corresponding to a plurality of entity node groups; and distributing the first sub-tables and the second sub-tables corresponding to the entity node groups to a plurality of computing units for distributed storage. In the distributed computing method for the graph data, the graph data is stored in a plurality of computing units in a distributed way according to the storage method for the graph data, and the method can be executed by one of the computing units.