Bitmap Region Partitioning for Reduced I/O Persistence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Bitmap data management systems require a large number of I/O operations for persistence, especially when dealing with large disk spaces or fine granularities, leading to increased system burden due to the need for direct flushing of dirty data during checkpoint events.
Innovation Solution
Divide the Bitmap region into partitions and set an update region, where dirty data from partitions with less data is merged and recorded in the update region through optimized I/O operations, reducing the number of I/O operations required for persistence by aggregating updates with relatively less dirty data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If direct flushing of Bitmap is performed during checkpoint events, then data persistence is ensured, but the number of I/O operations increases significantly
Solution Approach 1:
The Bitmap region is divided into multiple partitions, allowing selective flushing of only those partitions containing dirty data rather than flushing the entire Bitmap region. This segmentation reduces the scope of I/O operations while maintaining data persistence reliability.
Solution Approach 2:
Multiple partitions containing dirty data are merged into a single update region before flushing to disk. This consolidation allows multiple dirty data sets to be written in fewer I/O operations, improving productivity while ensuring all dirty data is persisted.
2Manufacturing precision
If Bitmap manages large disk space with fine granularity, then data management precision is improved, but the number of I/O operations required for persistence increases
Solution Approach 1:
The Bitmap region is segmented into multiple partitions, each managing a portion of the disk space. This allows fine-grained control over which specific partitions need to be flushed, maintaining data management precision while reducing the overall number of I/O operations by only flushing partitions with dirty data.
Solution Approach 2:
Instead of flushing the entire Bitmap region or using coarse-grained flushing, the system performs partial flushing by identifying and flushing only the specific partitions that contain dirty data. This partial action maintains precision while improving persistence efficiency.
Data Source
AI summary
A method and an apparatus for processing Bitmap data are provided by the embodiments of the present disclosure. The method for processing Bitmap data includes: dividing a Bitmap region in a disk into a plurality of partitions in advance and setting an update region in the disk; obtaining a respective amount of dirty data corresponding to each of the plurality of partitions in memory in response to a condition for writing back to the disk being satisfied; finding multiple second partitions with an amount of dirty data satisfying to be merged into the update region from the plurality of partitions according to the respective amount of dirty data corresponding to each of the plurality of partitions; and recording dirty data corresponding to the multiple second partitions in the memory into the update region in the disk through one or more I/O operations after merging.