Versioned Database Partitioning for Shared-Scenario Memory Reclamation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database administrators face challenges in reducing the memory footprint of versioned databases due to shared data between scenarios, as deleting a single scenario does not free up memory since other scenarios still require the shared data.
Innovation Solution
A computer-implemented method and system utilize a version visibility data structure (VVDS) to determine the best combination of scenarios to delete, evaluating memory reclamation for each combination, and executing the deletion to optimize memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a single scenario is deleted, then the scenario pointer is removed, but memory is not freed due to data sharing with other scenarios
Solution Approach 1:
The patent segments the deletion process into two distinct operations: (1) deleting scenario pointers (easy operation), and (2) deleting shared data (memory reclamation). This segmentation explains why simple pointer deletion doesn't free memory while the full process does.
Solution Approach 2:
The system performs preliminary identification of shared data before deletion by analyzing the version graph and scenario structure. It determines which data is shared across multiple scenarios and plans to delete that data only after all scenario pointers are removed, ensuring proper memory reclamation.
2Quantity of substance
If all possible scenario combinations are evaluated to find optimal memory reclamation, then complete memory optimization is achieved, but the number of combinations becomes computationally enormous
Solution Approach 1:
The patent extracts the essential information needed for memory optimization from the complex version graph structure. By identifying only the critical paths and shared data elements, it reduces the problem from evaluating all scenario combinations to analyzing a simplified representation of data sharing relationships.
Solution Approach 2:
The system creates a simplified model (version visibility data structure) that copies only the necessary relationships from the full version graph. This copying approach allows analysis of memory reclamation without processing the complete complex scenario graph.
3Quantity of substance
If shared data is deleted to free memory, then memory footprint is reduced, but data loss occurs for scenarios that still need the data
Solution Approach 1:
The system continuously monitors which scenarios reference which data through the version graph. Before deleting shared data, it verifies that all scenarios have been properly deleted or that no active scenarios depend on the data, providing feedback that prevents premature or incorrect data deletion.
Solution Approach 2:
The system performs preliminary analysis of the version graph to identify all scenarios that reference specific data before deletion. It ensures that data is only deleted after all referencing scenarios are removed, preventing data loss while maximizing memory reclamation.
Data Source
AI summary
Systems and methods for partitioning data in a versioned database, comprising: initializing, a list of empty partitions; initializing a new empty partition as a current partition; selecting a lead scenario for placement in the current partition, the lead scenario having a size less than a maximum partition size; placing the lead scenario into the current partition; marking the lead scenario as processed; obtaining a list of candidate scenarios to place in the current partition; determining a best candidate scenario from the list of candidate scenarios; adding the best candidate scenario to the current partition; marking the best candidate scenario as processed; iterating a new list of candidate scenarios to place in the current partition until there are no more scenario candidates to place in the current partition; adding the current partition to a partition list; and iterating through all the scenarios in the versioned database.