Hierarchical Data Structure for Column-Oriented Database Memory Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Column-oriented databases face a tradeoff between eviction and recycling of intermediate query results, leading to performance degradation due to memory exhaustion, as existing eviction policies either clear valuable memory space or result in the loss of reusable instruction data.
Innovation Solution
Implementing a hierarchical data-structure, such as a directed acyclic graph or tree, to represent query progression, allowing nodes to be maintained even after eviction, enabling the retention of recycling capabilities while reducing memory costs by evicting higher-level intermediates and maintaining lower-level nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If intermediate query results are stored in memory for recycling, then query performance is improved through reuse of intermediates, but memory consumption increases leading to memory exhaustion
Solution Approach 1:
The patent segments the memory management of intermediates by introducing a hierarchical data structure that divides intermediates into different levels based on their reuse potential and memory consumption. This segmentation allows the system to manage memory more efficiently by treating different intermediates differently, thereby resolving the contradiction between storing intermediates for performance and limiting memory consumption.
Solution Approach 2:
The patent changes the parameter of memory management by implementing eviction policies that dynamically adjust which intermediates are retained or evicted from memory based on their hierarchical level and reuse characteristics. This parameter change allows the system to optimize the balance between keeping useful intermediates in memory and preventing memory exhaustion.
2Quantity of substance
If eviction policies are applied to clear memory space, then memory consumption is reduced, but reusable instruction data is lost
Solution Approach 1:
The patent applies local quality by differentiating between different types of intermediates in the hierarchical structure. Not all intermediates are treated equally - those at lower levels with high reuse potential are preserved while those at higher levels are more readily evicted. This localized differentiation resolves the contradiction by preserving valuable reusable data while still freeing memory space.
Solution Approach 2:
The patent implements a selective discarding and recovering mechanism where intermediates are discarded from memory based on their hierarchical level but can be recovered by reconstructing them from their parent intermediates in the hierarchical structure. This allows memory space to be freed while maintaining the ability to recover and reuse instruction data when needed.
3Productivity
If a hierarchical data-structure is implemented to represent query progression, then recycling capability is preserved, but device complexity increases
Solution Approach 1:
The patent introduces a hierarchical dimension to the data structure, organizing intermediates in levels based on their position in the query execution plan. This dimensional organization preserves recycling capability by maintaining relationships between parent and child intermediates, while the hierarchical nature actually simplifies management compared to flat structures by providing clear eviction priorities and reconstruction paths.
Data Source
AI summary
The presently disclosed subject matter includes a system and method of managing computer memory of a column-oriented database, which enables to reduce the tradeoff between eviction and recycling of intermediate query results and thus retain much of the recycling capability at a considerably lower computer memory cost. The computer memory comprises one or more intermediates, each intermediate being generated by execution of a respective instruction, the instruction being part of a given query. An hierarchical data-structure is generated and stored in the computer memory, wherein different nodes in the hierarchical data-structure represent different respective intermediates stored in the computer memory. Responsive to a delete command, the at least one intermediate is deleted from the computer memory, while a respective node representing the at least one intermediate is maintained in the hierarchical data-structure.