Database Archiving via Record Group Compression and Parallel Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As data production and consumption increase, databases face capacity limitations, necessitating efficient archiving methods to manage storage capacity, and existing techniques lack effective solutions for parallel data retrieval from compressed archives.
Innovation Solution
A method and device that classify database records based on criteria like time and field value, compress, and archive them, while also enabling parallel retrieval by determining the number of database retrieval processes based on computer performance and group compression data, allowing for efficient storage and retrieval of archived data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If database capacity is expanded to store more data, then data storage capacity is improved, but cost and management complexity increase
Solution Approach 1:
The patent segments the database into original data and archived data based on access frequency and time. Frequently accessed data remains in the main database, while less frequently accessed data is moved to archive tables. This segmentation reduces the size of the main database, improving management efficiency without sacrificing total storage capacity.
Solution Approach 2:
The patent extracts data that is less frequently accessed from the main database and stores it in separate archive tables. This extraction reduces the burden on the main database system, lowering management complexity and improving performance for frequently accessed data while maintaining the ability to retrieve archived data when needed.
2Quantity of substance
If data is compressed and archived to save storage capacity, then storage efficiency is improved, but retrieval speed deteriorates
Solution Approach 1:
The patent segments archived data into groups based on access patterns and stores metadata about these groups in a compression table. This segmentation allows the system to quickly identify and retrieve only the specific archived data that is needed, rather than decompressing and searching through all archived data, thus maintaining fast retrieval speeds while achieving compression benefits.
Solution Approach 2:
The patent performs preliminary organization of archived data into structured groups with associated metadata stored in the compression table before retrieval operations. This preliminary action includes creating indexes and storing selection information that enables rapid location of archived data, eliminating the need for full decompression and search during retrieval operations.
3Productivity
If parallel retrieval processes are increased to improve retrieval speed, then data retrieval efficiency is improved, but system resource consumption increases
Solution Approach 1:
The patent implements a dynamic parallel retrieval mechanism that adjusts the number of parallel processes based on the size of the dataset, available system resources, and retrieval requirements. The system dynamically determines the optimal degree of parallelization, allowing it to maximize retrieval efficiency when resources are abundant while automatically reducing resource consumption when system load is high or datasets are small.
Data Source
AI summary
Provided is a database archiving method. According to the exemplary embodiment of the present invention, a database archiving method includes: selecting at least one record group including a plurality of records from an original table from which data is archived, based on selection information on at least one of a time and a field value; storing group compression data compressed to be created for every record group and the selection information corresponding to the group compression data in a compression table, with respect to each of at least one selected record group; and deleting a plurality of records included in at least one selected record group from the original table.