A distributed knowledge graph fragmentation method and device
By collaborating on edge set partitioning and node clustering through master and worker devices, combined with streaming sharding strategies, data redundancy and performance issues in distributed knowledge graph sharding are resolved, enabling efficient graph computing and deep learning applications.
Patent Information
- Application Number
- CN202310445019.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-23
- Publication Date
- 2025-10-14
- Estimated Expiration
- 2043-04-23
AI Technical Summary
Existing distributed knowledge graph sharding methods have problems with data redundancy and cross-machine communication affecting performance in large-scale graph data processing. In particular, traditional graph sharding algorithms such as METIS, DistrubitedNE, and CLUGP have shortcomings in memory requirements and efficiency.
A distributed knowledge graph sharding method is adopted. The master device divides the entire set of edges into multiple edge sets, and the working device performs node clustering and sharding proposal evaluation. Finally, the sharding result with the smallest cost is selected. Combined with the streaming sharding strategy, memory usage and data redundancy are reduced.
It achieves efficient segmentation of large-scale graph data, reduces memory usage, improves the operating efficiency of graph computing and deep learning, and adapts to the needs of different application scenarios.
