Persistent Directory for Variable-Size Entry Free-Space Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In-memory databases face inefficiencies when handling variable-size data entries, leading to slower performance and increased overhead due to the lack of spatial and temporal locality in free space management, especially when evicting and reloading pages, which can result in scattered free space and concurrency issues.
Innovation Solution
Implementing a system that consolidates free space within containers by evicting and reloading pages to group contiguous free space, reusing space from deleted entries, and dynamically deciding allocation options to minimize overhead and maximize contiguous free space, while tracking and storing relevant free-space information to optimize memory usage and access times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If variable-size data entries are used in in-memory databases, then memory flexibility and adaptability are improved, but free space management efficiency deteriorates due to scattered gaps and loss of spatial locality
Solution Approach 1:
The patent segments free space management by introducing a persistent directory structure that divides the container into manageable sections with tracking information. This segmentation allows efficient tracking of free space gaps without requiring full-page scans, thus maintaining productivity while supporting variable-size entries.
Solution Approach 2:
The patent performs preliminary action by pre-tracking and recording free space information in the persistent directory before pages are evicted. This advance preparation ensures that when pages are reloaded, free space can be quickly identified and reused without costly searches, maintaining both flexibility and efficiency.
2Quantity of substance
If pages are evicted and reloaded in in-memory databases, then memory utilization is improved, but spatial and temporal locality deteriorates leading to slower access times
Solution Approach 1:
The persistent directory pre-records free space locations and container metadata before pages are evicted to persistent storage. When pages are reloaded, this preliminary information allows the system to quickly restore spatial locality by placing entries in their original or optimized locations without extensive searching, thus reducing access time while maintaining high memory utilization.
Solution Approach 2:
The system uses feedback from the persistent directory about free space locations and container states to make informed decisions about page reloading and entry placement. This feedback mechanism ensures that pages are reloaded in a way that optimizes both memory utilization and access patterns, minimizing loss of spatial and temporal locality.
3Device complexity
If conventional free space management techniques are used, then implementation simplicity is maintained, but performance deteriorates due to significant overhead in accessing and processing scattered free space
Solution Approach 1:
The patent segments free space tracking into a dedicated persistent directory structure with specific metadata fields. This segmentation provides a clear, organized approach that is relatively simple to implement while dramatically improving processing speed by eliminating the need to scan entire pages for free space identification.
Solution Approach 2:
The persistent directory acts as an intermediary between the variable-size entry container and the free space management process. This intermediary layer simplifies implementation by providing a standardized interface for tracking free space while improving productivity through efficient querying and management of free space gaps without requiring complex scanning algorithms.
4Device complexity
If no free space tracking techniques are employed, then system complexity is reduced, but memory usage increases due to inevitable fragmentation and inability to reuse freed space
Solution Approach 1:
The persistent directory performs preliminary tracking of free space information before pages are evicted. This advance tracking enables efficient reuse of freed space upon page reloading, significantly reducing memory usage through fragmentation mitigation while maintaining relatively simple system complexity through a straightforward directory structure.
Solution Approach 2:
The system provides self-service for free space management through the persistent directory, which automatically tracks and manages free space gaps without requiring complex external intervention. This self-service approach efficiently reduces memory usage through automatic space reuse while keeping system complexity manageable through automated rather than manual management.
Data Source
AI summary
Disclosed herein are embodiments for free-space handling in pages and in-memory containers allowing variable-size data entries. An example system may determine expanses of unallocated space, within a page loaded in memory and configured to allocate a first data entry of a first size within the page that may also allocate at least one subsequent data entry of a second size different from the first size; save, into memory, transient free-space information corresponding to the page, including first position information for a first number of expanses of contiguous unallocated space within the page; evict the page from memory, writing allocated data entries contained therein to persistent storage; store, into persistent storage, persistent free-space information corresponding to the page, including second position information for a second number of expanses of unallocated space within the page, the second number being less than the first number; and discard the transient free-space information.