Selective Data Restoration Using Change Indexes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data restoration methods require full restores of entire datasets, which are time-intensive and resource-heavy, making it inefficient to roll back or recover specific portions of data, especially in large-scale databases.
Innovation Solution
The system uses change indexes generated from streaming data to selectively restore data by filtering time window batches, allowing for partial data restoration without processing the entire dataset, using stream filters to identify and revert changes within specified time windows or key ranges.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full restore is performed to restore corrupt data, then data restoration is achieved, but processing time becomes excessively long
Solution Approach 1:
The patent divides the dataset into multiple partitions or segments. Instead of restoring the entire dataset, the system identifies and restores only the specific partition containing corrupt data. This segmentation allows selective restoration, dramatically reducing processing time while maintaining data restoration reliability for the affected portion.
Solution Approach 2:
The system extracts and identifies only the corrupt items or specific partitions affected by corruption using change indexes and metadata. By taking out only the necessary data portions that require restoration rather than processing the entire dataset, the system achieves efficient targeted recovery.
2Reliability
If full restore is performed to ensure complete data recovery, then all data is restored, but resource consumption increases significantly
Solution Approach 1:
The dataset is segmented into manageable partitions with associated metadata and change indexes. The system queries these indexes to identify only the partitions requiring restoration, thereby reducing resource consumption proportionally to the fraction of data actually restored rather than processing 100% of the dataset.
Solution Approach 2:
The system performs partial restoration by restoring only the minimum necessary portion of data required to fix the corruption. By applying partial action rather than excessive full-restoration action, resource consumption is optimized while still achieving complete recovery of the affected data.
3Measurement precision
If full dataset is processed to identify corrupt items, then all corrupt data can be found, but processing efficiency decreases
Solution Approach 1:
Change indexes and metadata are maintained preliminarily as data changes occur. These pre-computed indexes enable direct querying of corrupt or modified items without needing to scan the entire dataset at restoration time. This preliminary action preserves identification accuracy while dramatically improving processing efficiency.
Solution Approach 2:
Change indexes serve as intermediary structures between the raw data and the restoration process. These indexes mediate by providing a efficient lookup mechanism to identify corrupt items without direct examination of the entire dataset, thus maintaining precision while enhancing productivity.
Data Source
AI summary
A selective data restoration technique is disclosed. A request to recover data in a dataset is received along with an identifier of a particular portion of the data to be recovered. The request may specify a time that indicates a desired recovery state (e.g., recover the values as they were at the time). Change data is determined for the specified time. For example, data stream filters generate recovery data by applying filters (e.g., keys) to a time window batch of the stream data. The stream data may correspond to changes to a data set, and the recovery data may correspond to only a portion of the dataset (e.g., a portion of a table). The system performs a recovery action, based on the recovery data, using the recovery data to automatically restore data in a data store, or providing the recovery date to the requesting entity, for example.