Batch Process Failover Using Cursor-Based Data Center Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data center redundancy systems cause burdens on both primary and backup data centers due to the need for redundant processing when a primary data center experiences an interruption, as batch processes are aborted and restarted from the beginning, leading to inefficiencies and resource waste.
Innovation Solution
A system that auto-establishes data transfers between data centers using an imitation artifact to track the current operation of processes with a record counter, allowing seamless continuation of batch processes from a validated cursor position in a backup site, reducing redundant processing and resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If batch processes are mirrored to backup data centers for resiliency, then data center reliability is improved, but redundant processing occurs causing greater burdens on computing resources
Solution Approach 1:
The system performs preliminary actions by continuously tracking the execution state of batch processes in the primary data center and pre-positioning the backup data center in a standby mode with updated process state information. When a failure occurs, the backup data center can immediately continue the process from the tracked cursor position without needing to restart from the beginning, thus avoiding redundant processing while maintaining resiliency.
Solution Approach 2:
The system implements feedback mechanisms where the primary data center continuously reports its process execution state (cursor position) to the backup data center. This feedback loop enables the backup data center to synchronize its state with the primary data center, allowing it to resume processing exactly where the primary data center left off during failover, thereby eliminating redundant processing of already-completed tasks.
2Reliability
If batch processes are restarted from the beginning in backup data centers, then process completion is ensured, but processing time and resource waste increase
Solution Approach 1:
The system performs preliminary tracking of the batch process execution state in the primary data center and maintains this state information in the backup data center. When failover is needed, the backup data center uses this pre-acquired state information (cursor position) to resume the process immediately, avoiding the time-consuming operation of restarting from the beginning while ensuring all processes are eventually completed.
Solution Approach 2:
The system establishes a feedback mechanism where the primary data center continuously updates the backup data center with its current process execution state. This ensures that when the backup data center takes over, it has accurate information about which processes have been completed and where the current process stands, allowing for immediate resumption without time loss while maintaining assurance of complete process execution.
3Measurement precision
If manual intervention is used to establish data transfers between data centers, then transfer accuracy can be controlled, but operational complexity and intervention time increase
Solution Approach 1:
The system implements self-service automation where the data transfer establishment process occurs automatically without manual intervention. The system autonomously identifies failing data centers, selects appropriate backup data centers, tracks process states, and establishes data transfers based on pre-defined policies and criteria, thereby maintaining transfer accuracy while eliminating manual operational complexity and reducing intervention time.
Solution Approach 2:
The system uses feedback mechanisms to automatically monitor data center health status, process execution states, and transfer progress. Based on this real-time feedback, the system dynamically adjusts and establishes data transfers autonomously, ensuring accuracy through continuous validation while simplifying operations by removing the need for manual intervention in the transfer establishment process.
Data Source
AI summary
Systems, computer program products, and methods are described herein for auto-establishing data transfers between data centers for batch processing. The present disclosure is configured to identify a process associated with a primary site; determine a backup site associated with the primary site; apply the process to the backup site; generate, by an imitation artifact component, a record counter of the process as the process runs in the primary site, wherein the record counter comprises a cursor position indicating a current operation of the process; and mimic, based on the record counter, the process in the backup site as the process runs in the primary site.