Secondary Node Load Shedding for Data Synchronization Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In primary/secondary data storage systems, the secondary node often lags behind the primary node due to unequal compute resource allocation, leading to potential data loss if it needs to take over in case of a failure, as it may have a backlog of unprocessed requests and be out of sync, especially during periods of increased incoming requests.
Innovation Solution
Implement load shedding at the secondary node by rejecting requests that would cause excessive latency, allowing the request originator to resubmit them later when resources are available, and executing any remaining requests to synchronize with the primary node, thus reducing the backlog and ensuring timely data synchronization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the secondary node processes all incoming requests to maintain data synchronization, then data consistency is improved, but execution latency increases and the node becomes overloaded
Solution Approach 1:
The patent applies partial action by selectively processing only critical requests at the secondary node while rejecting non-critical ones. The system determines which requests to process based on their importance to data consistency, rather than processing all incoming requests. This resolves the contradiction by maintaining essential data synchronization while avoiding overload and excessive latency.
Solution Approach 2:
The patent implements preliminary action by pre-calculating and storing execution latencies for different request types at the secondary node. Before receiving actual requests, the system prepares latency estimates that enable quick decision-making about which requests to process or reject. This preliminary preparation reduces real-time processing latency while maintaining data consistency for critical operations.
2Productivity
If the secondary node allocates more compute resources to process requests, then data synchronization speed is improved, but system complexity and resource requirements increase
Solution Approach 1:
The patent changes the parameter of resource allocation from a static, uniform approach to a dynamic, request-specific approach. By adjusting compute resource allocation based on request criticality and current system state, the system achieves faster synchronization for important data while avoiding the complexity of uniformly high resource allocation across all operations.
Solution Approach 2:
The secondary node performs self-service by autonomously determining which requests to process and which to reject based on pre-established criteria and latency thresholds. This self decision-making capability eliminates the need for complex external resource management systems, achieving efficient synchronization with reduced overall system complexity.
3Loss of time
If the secondary node rejects requests to reduce latency, then response time is improved, but data consistency may be compromised
Solution Approach 1:
The patent applies local quality by treating different requests differently based on their specific characteristics and criticality. Critical requests that affect data consistency are processed with high priority and low latency, while non-critical requests are rejected or deferred. This differentiated approach maintains data consistency for essential operations while achieving fast response times overall.
Solution Approach 2:
The system uses feedback mechanisms to monitor data consistency status and adjust request processing decisions accordingly. When the secondary node detects that data consistency is at risk, it modifies its request rejection criteria to ensure critical synchronization operations are processed. This feedback loop maintains reliability while preserving fast response times for non-critical operations.
Data Source
AI summary
Methods, apparatus, systems and articles of manufacture for load shedding are disclosed. An example apparatus includes memory, and at least one processor to execute machine readable instructions to cause the at least one processor to at least access a request to execute a process using a second computing environment, the second computing environment being a replica of a first computing environment, determine an expected execution latency associated with execution of the process using the second computing environment, and reject the request in response to the expected execution latency exceeding a threshold.