Distributed Storage Edge Node XOR Differential Backup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, especially in the IoT era, existing remote copy technologies require a large-capacity storage apparatus in the backup data center to duplicate data from edge nodes, leading to high storage costs and resource inefficiencies.
Innovation Solution
A distributed storage system that generates XOR update differential data at edge nodes and transmits it to a core system, where erasure codes are updated, reducing the storage capacity needed for backup while maintaining data protection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If asynchronous remote copy technology is used to back up data from edge nodes, then data protection is achieved, but the storage capacity required in the backup-side data center becomes enormous
Solution Approach 1:
The patent extracts only the essential information needed for backup by generating XOR update differential data that represents changes between snapshots, rather than transmitting complete data sets. This differential data is then used to update erasure codes at the core system, significantly reducing the storage capacity required while maintaining data protection capabilities
Solution Approach 2:
The patent transforms the backup approach by changing from storing complete data copies to storing erasure codes that can be regenerated from differential data. This parameter change in the backup methodology reduces the storage capacity requirement from proportional to total data size to proportional to the number of edge nodes and snapshot differences
2Reliability
If complete data copies are stored in the backup data center, then data protection is ensured, but storage costs increase significantly
Solution Approach 1:
The patent uses a simplified copying mechanism where instead of copying complete data sets, only XOR update differential data representing snapshot differences is transmitted and used to update erasure codes. This selective copying approach maintains data protection while reducing storage costs by storing only essential backup information
3Quantity of substance
If XOR update differential data and erasure codes are used, then storage capacity is reduced, but system complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing erasure codes at the core system based on XOR update differential data from multiple edge nodes. This preliminary preparation of backup information in an optimized format reduces the need for complex real-time processing during backup operations, thereby reducing overall system complexity
Data Source
AI summary
A distributed storage system includes an edge system including edge nodes, and a core system, which is coupled to the edge system via a network, and is configured to hold backup data of the edge system. Each of the edge nodes is configured to provide a volume to a host, generate XOR update differential data representing a differential between a first generation snapshot of the volume and an old generation snapshot of the volume, the old generation being older than the first generation, and transmit the generated XOR update differential data to the core system. The core system is configured to hold, as the backup data, erasure codes generated based on pieces of XOR update differential data transmitted from the edge nodes, and update the erasure codes based on the pieces of XOR update differential data received from the edge nodes.