Batch Application Recovery via Syncpoint Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In complex data processing systems like global banking networks, batch applications cannot run concurrently with transaction processing systems without causing temporary unavailability, leading to significant service disruptions and revenue loss, as batch applications need to lock records, which conflicts with the 24/7 requirement of transaction processing systems.
Innovation Solution
Implementing a method that issues syncpoints and checkpoints during batch application execution, allowing for restart from the last checkpoint and deferring updates until the most recent syncpoint is reached, thereby breaking down batch applications into smaller units of work and enabling concurrent updates with transaction processing systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If batch applications lock records for processing, then data consistency is ensured, but transaction processing systems become unavailable during batch execution
Solution Approach 1:
The batch application execution is segmented into multiple units of work, each bounded by syncpoints. This allows the batch processing to be divided into smaller, manageable transactions that can be interleaved with online transaction processing, enabling both batch data consistency and continuous system availability.
Solution Approach 2:
Syncpoints are issued at predetermined intervals during batch application execution to pre-establish save points before potential failures occur. This preliminary action enables rapid recovery by allowing the system to restart from the most recent syncpoint rather than the beginning of batch processing, minimizing downtime while maintaining data consistency.
2Object-affected harmful factors
If batch applications are executed offline, then record locking conflicts are avoided, but service disruptions occur
Solution Approach 1:
The system maintains continuous availability by allowing batch applications to execute online alongside transaction processing systems. The batch application processes records in units of work with syncpoints, ensuring that useful batch processing action continues without interruption while minimizing conflicts through proper transaction boundary management.
Solution Approach 2:
The batch application dynamically adjusts its execution by issuing syncpoints at strategic points during processing. This dynamic approach allows the system to manage record locking flexibly, releasing locks at syncpoint boundaries while maintaining continuous operation, thereby reducing service disruption time while avoiding locking conflicts.
3Stability of the object's composition
If batch applications process all updates in a single transaction, then data integrity is maintained, but system outages are required
Solution Approach 1:
The single large batch transaction is segmented into multiple smaller units of work, each protected by its own syncpoint. This segmentation maintains data integrity within each unit while allowing failed units to be restarted independently, enabling continuous system operation without requiring complete batch reexecution or system outages.
Solution Approach 2:
Syncpoints are established at predetermined intervals throughout batch processing to create preliminary save points. This preliminary structuring of transactions ensures data integrity at each boundary while enabling efficient recovery, allowing the system to maintain both data integrity and high availability by restarting only from the last successful syncpoint rather than requiring full batch reprocessing.
Data Source
AI summary
A method of operating a data processing system comprises executing a batch application, the executing comprising reading one or more inputs from one or more data files, performing updates on one or more records according to the or each input read from a data file, and issuing a syncpoint when said updates are completed. During the execution of the batch application, syncpoints are periodically issued and checkpoints are less frequently issued. Following detection of a failure of the batch application, the batch application is restarted with the last issued checkpoint, and the batch application is executed by reading one or more inputs from one or more data files, but not performing updates on said records, until the last issued syncpoint is reached.