Online Database Defragmentation Using IX-Locks and Record Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database systems face challenges with memory fragmentation, leading to wasted memory and degraded data access performance, particularly due to the inefficiencies of online defragmentation methods that require complex locking mechanisms and cannot ensure continuous operation.
Innovation Solution
The implementation of online defragmentation using intent exclusive locks (IX-locks) and record maps to iteratively move data from sparse to non-sparse pages, allowing the database system to remain in production use while performing defragmentation, thereby minimizing downtime.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If offline defragmentation is used, then memory defragmentation effectiveness is improved, but system availability deteriorates due to service downtime
Solution Approach 1:
The patent applies preliminary action by marking pages for defragmentation in advance while the database system remains online. The defragmentation process is prepared and planned before actual data movement occurs, allowing the system to maintain availability while still achieving effective defragmentation through staged execution.
Solution Approach 2:
The patent implements dynamics by transitioning from static offline defragmentation to dynamic online defragmentation. The system adapts its defragmentation operations to occur concurrently with normal database transactions, using dynamic locking mechanisms and iterative page movement that responds to system state changes in real-time.
2Reliability
If online defragmentation with table locks is used, then system availability is maintained, but transaction throughput deteriorates due to blocked updates
Solution Approach 1:
The patent applies segmentation by dividing the defragmentation process into individual page-level operations rather than locking entire tables. Each page can be processed independently with fine-grained locking, allowing other transactions to access different pages simultaneously, thus maintaining throughput while enabling defragmentation.
Solution Approach 2:
The patent uses partial action by applying locks only to specific pages being defragmented rather than entire tables. The locking scope is minimized to only the necessary pages, allowing the majority of the database to remain accessible for transactions, thereby maintaining throughput while achieving defragmentation goals.
3Productivity
If online defragmentation without proper locking is used, then transaction throughput is maintained, but data consistency deteriorates due to concurrent modifications
Solution Approach 1:
The patent introduces an intermediary mechanism in the form of a record map that mediates between the defragmentation process and concurrent transactions. The record map tracks page movement operations and coordinates with locking mechanisms to ensure data consistency is maintained while allowing transactions to proceed with minimal interruption.
Solution Approach 2:
The patent implements feedback through the record map that continuously monitors and records the state of pages being defragmented. This feedback mechanism allows the system to track which pages have been moved, which are in progress, and which are still accessible, enabling consistent data access patterns while maintaining throughput.
Data Source
AI summary
Methods, systems, and computer-readable storage media for online defragmentation of memory in database systems by applying an IX-lock to each table having data stored in a marked page in a set of marked pages, generating a record map including key-value pairs, each being associated with a record location in a marked page, a value of each key-value pair initially set to a first value, iteratively executing the online defragmentation to delete data from marked pages and add the data to non-sparse pages, at least one iteration including applying a try-lock to a record in a marked page, and at iterations of the online defragmentation, updating the record map to change the value of at least one key-value pair from the first value to the second value, the second value representing that data of a marked page has been deleted from the marked page and added to a non-sparse page.