Metadata Converter Hierarchical Memory Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face challenges in efficiently updating memory management techniques, as they lack flexibility in data loading and unloading strategies, leading to suboptimal resource utilization and throughput, especially when transitioning from column-based to more granular load preferences like partition and table-level settings.
Innovation Solution
A metadata converter and memory management system (MCS) that migrates column metadata to object metadata, enabling column, partition, and table-level load preferences, using a hierarchical structure to prioritize loading techniques and manage data aging, thereby improving resource utilization and system performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If column-based metadata structure is used, then system simplicity is maintained, but resource utilization and query processing efficiency deteriorate
Solution Approach 1:
The patent segments the metadata structure from a single column-based level into multiple hierarchical levels (column-level, partition-level, and table-level). This segmentation allows different granularity control for data loading, enabling the system to optimize query processing by selecting appropriate loading strategies at each level while maintaining manageable complexity through systematic organization.
Solution Approach 2:
The patent adds dimensional hierarchy to the metadata structure by introducing partition-level and table-level abstractions above the column-level. This dimensional change transforms a one-dimensional column-based structure into a multi-dimensional hierarchical structure, enabling finer-grained control over data loading decisions and improving query processing efficiency through more precise data selection.
2Productivity
If new memory management techniques are implemented, then resource utilization improves, but system update complexity increases
Solution Approach 1:
The patent creates a universal metadata structure that can accommodate multiple memory management techniques and loading strategies (column-load, partition-load, table-load) within a single unified framework. This multi-functional design allows the system to implement advanced resource utilization techniques without requiring separate systems for each technique, thereby reducing overall update complexity while improving resource utilization.
3Productivity
If finer-grained load preferences are implemented, then data loading efficiency improves, but metadata complexity increases
Solution Approach 1:
The patent implements a dynamic metadata structure where the granularity of load preferences can be flexibly adjusted across different levels (column, partition, table). This dynamic design allows the system to apply finer-grained control where needed while maintaining coarser control elsewhere, optimizing data loading efficiency without uniformly increasing metadata complexity across the entire system.
Data Source
AI summary
System, method, and various embodiments for providing a metadata converter and memory management system are described herein. An embodiment operates by determining that first metadata corresponding to a table of a database comprising load preferences for a column level for a plurality of columns of the table, wherein the load preferences include either column load or page load. It is determined that the database is enabled with both load preferences for a table level and load preferences partition level, in addition to load preferences for the column level. Values for the load preferences are automatically assigned for both the table level and the partition level in second metadata, and wherein the second metadata preserves the load preferences for the column level of the first metadata. A query against the table based on load preferences from the second metadata.