Data Replication Priority Management via Dynamic Queue
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data replication methods incur high costs and latency due to frequent replication of data with high write frequency and pending replication tasks, particularly in asynchronous replication systems, which can be costly and inefficient.
Innovation Solution
A data replication priority manager calculates priorities based on the write frequency, size, and age of replication tasks, using a priority queue to manage and process tasks efficiently, delaying lower-priority tasks and combining updates to reduce bandwidth and processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data replication is performed frequently for data with high write frequency, then data consistency is improved, but bandwidth consumption and processing costs increase
Solution Approach 1:
The patent changes the parameter of replication priority by introducing a priority value calculation that considers write frequency, data size, and age. This allows the system to dynamically adjust which data gets replicated based on multiple parameters rather than uniformly replicating all data, thereby reducing unnecessary bandwidth consumption while maintaining consistency for important data.
Solution Approach 2:
Instead of replicating all data updates immediately, the patent applies partial action by selectively replicating only high-priority data. The priority queue ensures that not all replication requests are processed equally, but only those with sufficient priority values, reducing overall bandwidth consumption while maintaining essential data consistency.
2Reliability
If data replication is performed frequently for data with high write frequency, then data consistency is improved, but processing time and costs increase
Solution Approach 1:
The patent introduces a priority value parameter that combines write frequency, data size, and age to determine replication urgency. This parameter change allows the system to process only critical replications immediately while deferring less critical ones, reducing overall processing time and costs while maintaining consistency for high-priority data.
Solution Approach 2:
The system performs partial replication by processing only high-priority tasks from the priority queue. Lower priority replication tasks are deferred or combined with other operations, reducing the total processing time and computational costs while still ensuring consistency for the most important data.
3Reliability
If replication tasks are processed immediately, then data consistency is improved, but latency for read operations on remote nodes increases
Solution Approach 1:
The patent changes the timing parameter of replication by introducing priority-based scheduling. Instead of immediate processing of all replication tasks, the system calculates priority values and processes tasks in priority order. This allows read operations to complete before lower-priority replications, reducing read latency while maintaining consistency through prioritized replication of critical data.
4Productivity
If all replication requests are processed equally, then simplicity is maintained, but efficiency decreases due to unnecessary replication of low-priority data
Solution Approach 1:
The patent introduces priority value as a new parameter to differentiate replication requests. By calculating priority based on write frequency, data size, and age, the system efficiently identifies which replications are necessary. This parameter addition significantly improves replication efficiency by eliminating unnecessary operations, while the complexity introduced is minimal and manageable through straightforward calculation and queue management.
Data Source
AI summary
A computer receives, at a target node, a replication target update, wherein the replication target update comprises a write frequency of a replication target and a size of the replication target. The computer creates a replication task to perform replication based upon the replication target update. The computer calculates a priority value of the replication task wherein the priority value is based upon the write frequency of the replication target and the size of the replication target. The computer inserts the replication task into a priority queue. The computer processes the replication task.