OLTP Database Block Compression with Lazy Re-Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database management systems (DBMS) for OLTP systems cannot efficiently compress data due to the real-time processing requirements, which makes compression difficult for OLTP transactions that access different tables and blocks, leading to inefficient use of storage space.
Innovation Solution
Implement a lazy compression method where database blocks are initially filled with uncompressed data until a predetermined condition is met, allowing for periodic compression and re-compression based on specific conditions, thereby maintaining real-time transaction processing performance without significant degradation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is compressed in OLTP database blocks, then storage space efficiency is improved, but transaction processing speed deteriorates due to real-time processing requirements
Solution Approach 1:
The system performs compression in advance during periods of low transaction activity or in the background, so that data is already compressed and ready for quick retrieval during OLTP operations. This preliminary compression action separates the compression overhead from the critical transaction processing path.
Solution Approach 2:
Compression is performed periodically rather than continuously, allowing the system to maintain uncompressed data during high-transaction periods for fast access, and switch to compression during low-activity periods. This periodic switching optimizes both storage efficiency and transaction speed by adapting to workload conditions.
2Quantity of substance
If compression is performed on every OLTP transaction, then storage space is optimized, but system complexity increases due to managing compression state for each transaction
Solution Approach 1:
Multiple OLTP transactions are merged into a single compression operation. Instead of compressing data for each individual transaction, the system accumulates changes from multiple transactions and performs a unified compression, reducing the number of compression operations and simplifying state management.
Solution Approach 2:
The compression system serves itself by automatically detecting when compression is beneficial and executing compression operations without requiring complex external coordination for each transaction. The system self-manages the compression state and timing based on intrinsic workload patterns.
3Speed
If data is kept uncompressed in database blocks, then transaction processing speed is maintained, but storage space efficiency deteriorates
Solution Approach 1:
Different portions of the database are maintained in different compression states based on their access patterns. Frequently accessed data blocks remain uncompressed for fast retrieval, while less frequently accessed blocks are compressed to save space. This local differentiation optimizes both speed and storage efficiency for different data regions.
Solution Approach 2:
The compression state of database blocks is dynamic rather than static. Blocks can transition between compressed and uncompressed states based on changing workload patterns and access frequencies. This dynamic adaptation allows the system to optimize storage efficiency without permanently sacrificing transaction speed for hot data.
Data Source
AI summary
A computer is programmed to compress data of a database in response to database modification language (DML) statements generated by on-line transaction processing (OLTP) systems. In several embodiments, data that is initially added to a database block is left uncompressed until a predetermined condition is satisfied, which happens infrequently (relative to OLTP transactions on the block). When satisfied, the computer automatically compresses all uncompressed data in the block, which increases the amount of unused space in the block. New data is thereafter added uncompressed to the partially compressed block, until satisfaction of a predetermined condition whereby the partially compressed block is again compressed, i.e. re-compressed. Adding of new data to a partially compressed block and its compression are repeated unless another predetermined condition is met, in response to which the block is not further re-compressed, thereby to recognize a limit on the benefit from compression.