A distributed knowledge graph fragmentation method and device

By collaborating on edge set partitioning and node clustering through master and worker devices, combined with streaming sharding strategies, data redundancy and performance issues in distributed knowledge graph sharding are resolved, enabling efficient graph computing and deep learning applications.

CN116226409BActive Publication Date: 2025-10-14ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
2 Cites 0 Cited by

Patent Information

Application Number
CN202310445019.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-04-23
Publication Date
2025-10-14
Estimated Expiration
2043-04-23

AI Technical Summary

Technical Problem

Existing distributed knowledge graph sharding methods have problems with data redundancy and cross-machine communication affecting performance in large-scale graph data processing. In particular, traditional graph sharding algorithms such as METIS, DistrubitedNE, and CLUGP have shortcomings in memory requirements and efficiency.

Method used

A distributed knowledge graph sharding method is adopted. The master device divides the entire set of edges into multiple edge sets, and the working device performs node clustering and sharding proposal evaluation. Finally, the sharding result with the smallest cost is selected. Combined with the streaming sharding strategy, memory usage and data redundancy are reduced.

Benefits of technology

It achieves efficient segmentation of large-scale graph data, reduces memory usage, improves the operating efficiency of graph computing and deep learning, and adapts to the needs of different application scenarios.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

Embodiments of the present specification relate to a distributed knowledge graph fragmentation method and device, the method is applied to a distributed system, the distributed system includes a master device and N working devices, the method comprises: any working device clusters nodes in an edge full set assigned to edges in the device subset into a plurality of node clusters, forms a clustering result and sends it to the master device; the master device combines the received N clustering results and sends them to the N working devices; any working device assigns a plurality of node clusters in the received clustering result to a plurality of fragments according to a preset cost evaluation function, forms a fragmentation proposal and sends it to the master device; the master device receives N fragmentation proposals, and determines the first fragmentation result with the minimum cost value as the first target fragmentation result for fragmenting the target knowledge graph.
Need to check novelty before this filing date? Find Prior Art