Database Compression via Frequency-Based Table Indexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Flash disks are more expensive but offer faster data retrieval compared to ATA and SATA disks, necessitating a method to reduce the number of flash disks required while maintaining performance.
Innovation Solution
A system that stores a database on a storage array, determines the most frequently used values, and compresses data using a table-based approach, employing block, block+table, and table-level compression techniques to reduce data size.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is compressed using traditional compression algorithms, then storage space is reduced, but data retrieval speed decreases
Solution Approach 1:
The patent segments data into fixed-size blocks and creates separate compression indexes for each block. This allows the system to compress data efficiently while maintaining quick access to specific blocks through their individual indexes, resolving the contradiction between compression ratio and retrieval speed.
Solution Approach 2:
The patent introduces compression indexes as intermediary structures that map compressed data blocks to their original content. These indexes enable rapid data retrieval without requiring full decompression, thus maintaining fast access speeds while achieving effective compression.
2Speed
If more flash disks are used to avoid compression, then data retrieval speed is maintained, but storage cost increases
Solution Approach 1:
The patent changes the parameter of data representation by encoding frequently occurring values as integers and using differential encoding. This transforms the data into a more compact form that requires fewer storage units while maintaining access performance through the block-based indexing system.
3Ease of manufacture
If data is compressed without considering data structure, then compression is simple to implement, but compression ratio is poor
Solution Approach 1:
The patent applies different compression strategies to different parts of the data based on local characteristics. Frequently occurring values are encoded as integers, while other values use differential encoding. This localized approach to compression achieves high compression ratios without requiring complex global optimization.
Solution Approach 2:
The patent performs preliminary analysis to identify frequently occurring values and builds compression indexes before actual compression. This preliminary action enables the compression process to efficiently encode data using the pre-computed frequency information, achieving high compression ratios with relatively simple implementation.
Data Source
AI summary
In one aspect, a method includes storing a database on a storage array, determining a plurality of most used values in the database, forming a table using the plurality of most used values and compressing data in the database according to the table. In another aspect, an apparatus includes electronic hardware circuitry configured to store a database on a storage array, determine a plurality of most used values in the database, form a table using the plurality of most used values and compress data in the database according to the table. In a further aspect, an article includes a computer-readable medium that stores computer-executable instructions. The instructions cause a machine to store a database on a storage array, determine a plurality of most used values in the database, form a table using the plurality of most used values and compress data in the database according to the table.