FIFO Structure for Transaction-Aware Data Protection in Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data protection technologies in networked storage systems provide coarse-grained snapshots, making it difficult to efficiently restore data to specific transaction boundaries, which can delay recovery and breach recovery time objectives (RTO) and recovery point objectives (RPO).
Innovation Solution
A first-in-first-out (FIFO) structure is implemented to store pointers to modified storage locations and consistent file system structures near transaction boundaries, allowing for quick access and restoration of data containers, including databases, by retaining previous data and file system states, enabling efficient restoration to specific transaction points without extensive log replay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional snapshot technology is used to protect stored data, then data protection is provided at fixed time intervals, but the recovery point objective is coarse grained and restoration to specific transaction boundaries is not possible
Solution Approach 1:
The patent segments the continuous data protection requirement into discrete transaction boundary checkpoints. Instead of taking snapshots at fixed time intervals, the system identifies and marks specific transaction boundaries where data state changes occur. This segmentation allows precise restoration to any transaction boundary without having to replay logs from earlier snapshots, thereby reducing recovery time while maintaining data protection.
Solution Approach 2:
The system performs preliminary actions by pre-identifying and marking transaction boundaries as they occur during normal database operations. Transaction boundaries are detected and recorded in advance, so when restoration is needed, the system can immediately jump to the appropriate pre-marked boundary without having to replay log files. This preliminary marking of restoration points eliminates the time penalty associated with conventional snapshot-based recovery.
2Reliability
If snapshots are taken at fixed time intervals, then data protection is maintained, but extensive log replay is required to restore to specific transactions, breaching recovery time objective
Solution Approach 1:
The patent divides the log replay process into discrete segments corresponding to transaction boundaries. By identifying specific transaction boundaries where data state changes occur, the system can segment the restoration process to only replay logs from the most recent transaction boundary up to the target transaction, rather than replaying from the beginning of the log file. This segmentation dramatically increases restoration speed while maintaining data protection integrity.
Solution Approach 2:
The system implements self-service by automatically detecting and marking transaction boundaries during normal database operations without requiring manual intervention. The database management system itself generates the restoration checkpoints by identifying when transactions commit, eliminating the need for external snapshot management tools. This self-marking process enables rapid restoration while maintaining continuous data protection.
3Loss of time
If conventional snapshot technology is used, then data can be restored to snapshot time, but not to specific transactions between snapshots, resulting in loss of data precision
Solution Approach 1:
The system performs preliminary detection and marking of transaction boundaries as they occur during normal database operations. By preemptively identifying when transactions commit and marking these boundaries in the log file, the system ensures that precise transaction-level restoration points are available in advance. This preliminary action eliminates the need to approximate restoration times and enables exact restoration to any transaction boundary.
Solution Approach 2:
The system implements feedback mechanisms where the database management system continuously monitors transaction commits and provides feedback about transaction boundary locations. This feedback loop allows the system to dynamically update the list of available restoration points based on actual transaction activity, ensuring that the most precise and relevant transaction boundaries are always available for restoration. The feedback mechanism maintains measurement precision by constantly syncing the restoration point database with actual transaction states.
Data Source
AI summary
Data protection methods and systems for a storage environment are provided. A first-in-first out (FIFO) structure stores a logical representation of a first storage location that retains previous data for a data container, after new data for the data container is stored at a second storage location. The FIFO structure also stores a logical representation of a file system tree structure that is stored in persistent storage, after a consistent point operation. In response to an event, the file system tree structure is selected, based on the file system tree structure being closest to a transaction. A snapshot is generated using the file system tree structure. Thereafter, the data container is restored from the snapshot or from a copy of the snapshot.