Historical Table Recovery Using Metadata-Linked Partition Files
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data recovery techniques for historical table data in database systems are resource-intensive, slow, and inefficient, involving multiple manual steps that are error-prone and difficult to manage during large-scale recovery, especially in the event of system failures or user mistakes.
Innovation Solution
The disclosed data recovery techniques utilize a DR manager to automate the recovery process by associating recovered data files of a table with a newly created table without copying data, reducing manual steps and involving database support engineers, thus enabling efficient and error-proof failsafe recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional data recovery techniques are used, then data can be recovered, but the process is resource-intensive and slow
Solution Approach 1:
The patent applies the copying principle by creating a new table structure that references deleted partition files through metadata rather than copying the actual data. The recovered table is formed by copying metadata information (table schema, partition metadata) while the actual data remains in the original partition files, eliminating the need for resource-intensive data copying during recovery.
Solution Approach 2:
The patent introduces metadata as an intermediary mechanism between the deleted table and the recovered table. The metadata acts as a mediator that stores references to partition files and table structure information, allowing the recovery process to operate on metadata level rather than data level, significantly reducing resource consumption.
2Reliability
If manual recovery steps are performed, then data can be recovered, but the process is error-prone and difficult to manage
Solution Approach 1:
The patent implements self-service by enabling users to initiate data recovery through simple SQL commands (DROP TABLE, TRUNCATE TABLE, DELETE FROM) without requiring complex manual intervention. The system automatically handles the recovery process by detecting deleted tables, retrieving metadata, and reconstructing tables, eliminating the need for manual recovery procedures and reducing operational complexity.
Solution Approach 2:
The patent applies preliminary action by maintaining metadata information about tables and partition files continuously during normal operation. This metadata is preserved even after tables are deleted, enabling automatic recovery without requiring manual investigation or complex recovery procedures, thus improving reliability while simplifying operation.
3Loss of time
If data copying is performed during recovery, then data can be recovered, but the process is time-consuming
Solution Approach 1:
The patent applies copying at the metadata level rather than data level. The recovery process copies table schema information, partition metadata, and references to partition files, but not the actual data rows. This selective copying approach dramatically reduces recovery time while maintaining full data accessibility through the recovered table's reference to original partition files.
Solution Approach 2:
The patent extracts the recovery process from the data layer to the metadata layer. By taking out the actual data copying operation and replacing it with metadata reference copying, the system achieves rapid recovery through metadata manipulation while the original data remains intact and accessible through the recovered table's metadata references.
Data Source
AI summary
Provided herein are systems and methods for configuring data recovery. A method includes decoding, by at least one hardware processor, a request to recover historical table data. The request is received from an account of a data provider. The historical table data includes a plurality of partition files. Each of the plurality of partition files includes a deleted file designation. Based on the request, a recovery process of the plurality of partition files is performed to obtain recovered partition files. A schema associated with the historical table data is retrieved. Metadata corresponding to the retrieved schema is generated. The metadata is associated with the recovered partition files to recover the historical table data.