Per-core Bitmaps for Multi-core Storage Log Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-core storage systems, conventional methods using a single global bitmap for allocating transaction log space lead to lock contention and overhead, limiting the ability of multiple cores to take advantage of concurrent processing due to memory and cache contention.
Innovation Solution
Implementing a set of per-core bitmaps for each core of a multi-core processor to allocate transaction log entries from a pre-allocated portion of transaction log space, reducing the need for a single lock and minimizing lock contention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single global bitmap is used for allocating transaction log space, then allocation management is simplified, but lock contention and overhead increase, limiting concurrent processing capability
Solution Approach 1:
The patent divides the single global bitmap into multiple per-core bitmaps, with each core having its own bitmap for tracking transaction log space allocation. This segmentation eliminates the need for a single global lock, allowing each core to independently manage its allocated space without contention with other cores, thereby improving concurrent processing capability while maintaining manageable complexity through modular organization
2Device complexity
If a single global bitmap is used for allocating transaction log space, then memory structure is simplified, but memory and cache contention increase, reducing I/O performance
Solution Approach 1:
The memory structure is segmented into multiple per-core bitmaps distributed across different cores, reducing cache contention by allowing each core to access its own bitmap in local cache without interfering with other cores. This segmentation maintains relatively simple memory structures while dramatically improving I/O performance through reduced contention
Solution Approach 2:
Each core is assigned its own dedicated bitmap in local memory/cache, providing each core with fast local access to allocation information without requiring access to shared global memory structures. This local quality improvement enhances I/O performance by minimizing memory access latency and cache misses
3Productivity
If per-core bitmaps are implemented for each core, then lock contention is reduced and concurrent processing improves, but device complexity increases
Solution Approach 1:
The system is segmented into independent per-core bitmap structures, where each core manages its own allocation tracking. This segmentation reduces lock contention and improves concurrent processing while the modular nature of per-core structures keeps the increase in system complexity manageable and organized
4Speed
If per-core bitmaps are implemented for each core, then memory and cache contention is reduced, but implementation complexity increases
Solution Approach 1:
The implementation is segmented into independent per-core bitmap management units, where each core has its own simplified bitmap structure. This segmentation reduces memory and cache contention, improving I/O performance while the repetitive, modular nature of per-core implementation keeps implementation complexity manageable through code reuse and standardized patterns
Data Source
AI summary
A method is used in managing log transactions in storage systems. A first portion of a transaction log space is allocated from a persistent journal associated with a file system to a first core of a multi-core storage processor and a second portion of the transaction log space is allocated from the persistent journal to a second core of the multi-core storage processor. A set of per-core bitmaps is used for allocating a set of transaction log entries from the persistent journal. Each per-core bitmap is associated with a respective core of the multi-core storage processor. A per-core bitmap of the set of per-core bitmaps associated with a core of the multi-core storage processor is used for allocating a transaction log entry for a write request performed by the core.