Database Backup Optimization via Time-Tagged File Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In database systems, the process of incremental backup is inefficient due to the need to read and combine all data storage files, leading to wasted network resources and prolonged backup times, especially after data has been combined into a single file.
Innovation Solution
A method where data storage files are saved with time metadata, allowing for the selection and combination of files based on their write times relative to the most recent backup operation, enabling separate combination of files before and after the backup time, thereby reducing the time and resources required for backup operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If all data storage files are combined into one data storage file to improve data reading performance, then data reading performance is improved, but backup time and network resource consumption increase significantly
Solution Approach 1:
The patent segments data storage files by creating separate combined files for data written before and after the backup time point. Instead of combining all data storage files into a single file, the system divides them into multiple segments based on their write time relative to the backup operation, allowing incremental backup to efficiently locate and process only the necessary segments.
Solution Approach 2:
The patent performs preliminary classification of data storage files by their write time during the combination operation. By pre-organizing files into pre-backup and post-backup segments with associated time metadata, the system prepares the data structure in advance to enable fast identification of files requiring incremental backup, avoiding the need to scan all files during the backup process.
2Speed
If all data storage files are combined into one data storage file to improve data reading performance, then data reading performance is improved, but network resource consumption increases
Solution Approach 1:
The patent segments data storage files by creating separate combined files for data written before and after the backup time point. Instead of combining all data storage files into a single file, the system divides them into multiple segments based on their write time relative to the backup operation, allowing incremental backup to efficiently locate and process only the necessary segments.
Solution Approach 2:
The patent extracts and utilizes time metadata from data storage files to identify and separate files that need to be backed up. By extracting the write time information and using it as a selection criterion, the system can take out only the relevant post-backup files for incremental backup operations, avoiding unnecessary network transmission of already-backed-up data.
3Speed
If data storage files are combined after incremental backup to improve reading performance, then data reading performance is improved, but the backup process becomes inefficient
Solution Approach 1:
The patent performs preliminary classification of data storage files by their write time during the combination operation. By pre-organizing files into pre-backup and post-backup segments with associated time metadata, the system prepares the data structure in advance to enable fast identification of files requiring incremental backup, avoiding the need to scan all files during the backup process.
Solution Approach 2:
The patent implements a dynamic combination strategy where the combination operation adapts to the backup status. The system dynamically determines which files to combine based on the backup time point, creating different combined file structures depending on whether the operation is a full backup or incremental backup, thereby optimizing both reading performance and backup efficiency.
Data Source
AI summary
The present application provides a data processing method and apparatus. The method includes: when a region in a table of a database system meets a predefined condition, saving data in the region as a data storage file to a storage device, where the data storage file includes time metadata, and the time metadata comprises a region write time of the data in the region; when a condition for triggering a combination operation is met, obtaining a time for performing a backup operation most recently; selecting at least two first data storage files, where a region write time recorded in time metadata included in the at least two first data storage files is before the obtained time; and combining the selected at least two first data storage files, therefore resolving a problem in the prior art that a time consumed for performing a backup operation is long.