Hybrid Heap Memory Management for Multi-Threaded Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database memory management systems face challenges in efficiently allocating and deallocating memory, particularly in handling memory fragmentation and ensuring performance under multi-threaded environments, where synchronization costs are high and memory access is not optimized.
Innovation Solution
A hybrid memory management approach combining thread-local storage (TLS) and core-striped memory management, where memory allocation requests are categorized by size classes and assigned to appropriate sub-allocators, utilizing TLS for small blocks and core-striped management for larger blocks, reducing synchronization needs and optimizing memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional memory allocation is used in multi-threaded environments, then memory can be allocated, but synchronization overhead increases and performance decreases
Solution Approach 1:
The memory manager is segmented into multiple independent sub-allocators, each responsible for a specific size class of memory blocks. Each sub-allocator maintains its own free list and metadata, allowing parallel access without synchronization. This segmentation eliminates contention between threads for memory allocation while ensuring correctness within each size class.
Solution Approach 2:
Different parts of the memory management system are optimized for different purposes: small blocks use thread-local storage for fastest access, medium blocks use per-core allocators, and large blocks use a global allocator. This local quality approach ensures that each memory allocation operation uses the most appropriate strategy for its size and access pattern, maximizing performance while maintaining correctness.
2Device complexity
If memory is allocated from a single pooled allocator, then allocation is simple, but memory fragmentation increases and allocation efficiency decreases
Solution Approach 1:
The single pooled allocator is segmented into multiple size-class-specific sub-allocators. Each sub-allocator manages a specific range of block sizes and maintains its own free list, preventing fragmentation across different size classes. This segmentation improves allocation efficiency by ensuring that freed blocks of a particular size remain available for future allocations of the same size.
Solution Approach 2:
The allocator structure changes based on the size parameter of the memory block being allocated. Different data structures and allocation strategies are used for different size ranges: simple bump pointers for small blocks, segmented free lists for medium blocks, and coalescing allocators for large blocks. This parameter-based differentiation optimizes allocation efficiency for each size class while managing complexity.
3Reliability
If synchronization is used for memory allocation in multi-threaded environments, then memory safety is ensured, but synchronization overhead increases and performance decreases
Solution Approach 1:
The memory management system is segmented into thread-local and core-local allocators that do not require cross-thread synchronization. Each thread or core has its own allocator instance with dedicated memory pools, eliminating the need for mutexes or atomic operations during allocation. Memory safety is maintained through the isolated nature of these allocators and controlled handoff points to the global allocator.
Solution Approach 2:
Each thread or core serves itself by maintaining local copies of allocator state and free lists. Instead of relying on centralized synchronization, each allocator instance independently manages its own memory pools and makes allocation decisions without consulting other threads. This self-service approach eliminates synchronization overhead while ensuring memory safety through isolated management.
Data Source
AI summary
A database memory manager determines a size class for each of a plurality of memory allocation requests. The memory manager then, based on the determined size classes, assigns which of a plurality of sub-allocators forming part of a plurality of memory pools should handle each memory allocation request. The sub-allocators assignments are based on predefined size ranges for each size class. The corresponding assigned sub-allocators then identify locations within the memory for each memory allocation request. The corresponding assigned sub-allocators next handle the memory allocation requests to allocate memory at the identified locations such that one of the sub-allocators utilizes both thread local storage and core-striped memory management.