Distributed Graph Database Backup via Intermediate Shard Staging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing methods for distributed graph databases are not flexible and efficient enough for data backup and restoration, leading to inefficiencies and high communication overheads.
Innovation Solution
A data processing method and apparatus that involves determining to-be-backed-up graph data shards in a first cluster, exporting them to an intermediate storage device, and storing them in a second cluster based on graph topology structure information, decoupling the export and import processes by using a stored complete data replica, thereby providing a more flexible, practical, and efficient solution for data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is directly transferred between clusters without intermediate storage, then communication overhead is reduced, but flexibility and efficiency of backup operations deteriorate
Solution Approach 1:
The patent introduces an intermediate storage device as a mediator between the first cluster (source) and second cluster (destination). This intermediate storage decouples the export and import processes, allowing data to be staged before final transfer. The intermediate storage enables flexible backup operations by allowing data to be held in a buffer state, enabling partial backups, selective restoration, and coordination between multiple operations without direct cluster-to-cluster communication overhead.
2Ease of manufacture
If data backup is performed using traditional methods, then implementation is simpler, but efficiency and practicality deteriorate
Solution Approach 1:
The patent segments the data backup process into distinct phases: data export from the first cluster to intermediate storage, data staging and management in intermediate storage, and data import to the second cluster. This segmentation allows each phase to be optimized independently, improving overall backup efficiency while maintaining manageable complexity through modular operation.
Solution Approach 2:
The patent performs preliminary actions by exporting data to intermediate storage before final destination is determined or before import operations begin. This allows for data validation, transformation, or selection to occur before the final import, improving efficiency by avoiding unnecessary transfers and enabling better resource planning.
3Reliability
If graph data shards are distributed across multiple storage nodes, then data reliability is improved, but locating and managing specific shards becomes more complex
Solution Approach 1:
The patent employs feedback mechanisms through graph topology structure information that tracks the location and status of graph data shards across storage nodes. This feedback system enables the system to automatically locate specific shards, manage their distribution, and coordinate backup operations across multiple nodes, reducing the perceived complexity while maintaining high reliability through distributed storage.
Data Source
AI summary
Embodiments of this specification describe distributed graph database data processing. To-be-backed-up graph data is determined, where graph data of a distributed graph database is stored in a form of a shard in a storage node in a first cluster. At least one target storage node based on a correspondence between a graph data shard and a storage node, in the first cluster, in which the graph data shard is located. Several graph data shards in the target storage node are exported to an intermediate storage device. The graph data exported to the intermediate storage device is stored in at least one storage node in a second cluster based on a correspondence between the graph data shard and a storage node, in the second cluster, in which the graph data shard is to be stored.