Probabilistic Data Replication via Dynamic Aggregator Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data replication methods face challenges in efficiently adapting to dynamically changing network conditions, leading to potential data loss and latency issues during disaster recovery scenarios, as they rely on fixed routes that may not be optimal due to network congestion or site availability changes.
Innovation Solution
The method involves dividing a data image into chunks, sending them to multiple aggregator sites, and using a probabilistic selection mechanism based on efficacy values to determine the most efficient paths for data replication, allowing for dynamic adaptation to network conditions and site changes, with a manifest ensuring correct reassembly at the destination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If fixed routes are used for data replication, then system simplicity is maintained, but adaptability to changing network conditions deteriorates
Solution Approach 1:
The patent implements dynamic route selection by computing multiple possible paths (primary and secondary routes) between source and destination sites. The system dynamically switches between routes based on real-time network conditions, site availability, and performance metrics, transforming the static fixed-route approach into a dynamic adaptive system that responds to changing environmental conditions.
Solution Approach 2:
The system continuously monitors network conditions, data transfer performance, and site availability, using this feedback to dynamically adjust route selection and replication strategies. Performance metrics from actual data transfers inform subsequent routing decisions, creating a closed-loop control system that adapts to changing conditions based on observed outcomes.
2Reliability
If data is replicated to multiple destinations simultaneously, then data availability is improved, but data loss risk increases due to network congestion
Solution Approach 1:
The patent segments the data image into multiple chunks and distributes them across different aggregation sites and routes. By dividing the data into smaller units that can be independently transmitted through multiple paths, the system achieves redundancy and fault tolerance while managing network load effectively, preventing data loss even when some paths fail or experience congestion.
Solution Approach 2:
The patent introduces aggregation sites as intermediary nodes between the source and final destination. These intermediaries receive, store, and forward data chunks, providing buffer capacity and alternative transmission paths. This intermediary layer isolates the source from direct congestion risks at the destination while maintaining data availability through multiple indirect routes.
3Productivity
If probabilistic selection is used for data chunks, then network efficiency is improved, but data integrity complexity increases
Solution Approach 1:
The patent performs preliminary actions by pre-computing multiple routes, pre-segmenting data into chunks with embedded integrity information, and pre-establishing aggregation sites before actual data transfer. This preparation reduces the complexity of real-time integrity management during active replication, as the probabilistic selection operates on pre-validated paths and data units with built-in verification mechanisms.
Data Source
AI summary
Disclosed embodiments provide techniques for multi-destination probabilistic data replication. Data transfer occurs over multiple time intervals. A data image to be transferred is divided into chunks. A manifest is created that lists each chunk and specifies an order, such that the data image can be reconstructed at its destination. The manifest is sent to the destination. The chunks may be sent to the destination, or to an aggregator site that then forwards the chunks to the destination. The chunks are reassembled at the destination based on information in the manifest. A probabilistic function is used to select an aggregator site based on an efficacy. The efficacy is based on a reward function that is computed for destinations for each time interval. A data transfer policy is periodically updated with a new efficacy value which is used for adjustment of the probabilistic function.