Regrouping Tabular Database Rows to Reduce Column Extent Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Tabular databases face inefficiencies in managing snapshots due to the formation of holes in column extents, leading to increased metadata size and storage requirements, as column extents become smaller over time due to row deletions and cell overwriting, necessitating a method to regroup rows and optimize column extent sizes.
Innovation Solution
A method is introduced to regroup rows by assigning scores based on fullness across multiple snapshots, relocating rows to create larger column extents, and reordering them to minimize the number of column extents, thereby reducing metadata size and storage requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If rows are deleted and cells are overwritten in a tabular database, then data updates are achieved, but column extents become smaller and holes are formed, increasing metadata size and storage requirements
Solution Approach 1:
The system performs preliminary actions by creating snapshots before data modifications occur. These snapshots preserve the state of column extents at specific points in time, allowing the system to later regroup rows based on historical fullness information without affecting current data integrity. This preliminary preservation enables optimization of column extent structure while maintaining reliable data records.
Solution Approach 2:
The system dynamically regroups rows based on their fullness scores across multiple snapshots. Instead of static column extent management, the system continuously adapts row ordering to maximize column extent sizes. This dynamic reorganization minimizes holes and extends column extents, reducing metadata requirements while preserving data integrity through the snapshot history.
2Quantity of substance
If column extents are made larger to reduce metadata size, then storage efficiency improves, but row reorganization complexity increases
Solution Approach 1:
The system uses copies of historical data states through snapshots to inform row reorganization decisions. Instead of analyzing complex real-time data patterns, the system copies fullness information from multiple snapshot states and uses these copies to score and rank rows. This simplifies the reorganization process by providing clear, pre-computed fullness metrics that guide row ordering without requiring complex real-time analysis.
Solution Approach 2:
The system changes the parameter of row ordering based on calculated fullness scores. By transforming the complex problem of column extent optimization into a parameter-based sorting problem, the system simplifies row reorganization. Rows are reordered according to their fullness scores across snapshots, converting a complex structural optimization problem into a straightforward parameter-driven sorting operation that reduces metadata size efficiently.
3Loss of information
If multiple snapshots are maintained to preserve database states, then data history is preserved, but storage requirements increase
Solution Approach 1:
The system extracts only the necessary information from snapshots for row fullness scoring, rather than maintaining complete copy histories. By taking out only the fullness metadata and score calculations from the snapshot data, the system preserves essential data history for optimization purposes while minimizing the storage footprint of maintained snapshot information.
Solution Approach 2:
The system discards redundant snapshot data after extracting useful fullness information, and recovers optimization benefits through row reorganization. By discarding unnecessary snapshot copies and retaining only the essential fullness scores, the system reduces storage requirements while recovering the benefits of historical data through the reorganized column extent structure.
Data Source
AI summary
There may be provided a method for managing column extents of a tabular database, the method may include (a) generating a multi-snapshot row score to each row of a group of rows of the tabular database multiple rows; wherein the multi-snapshot score is indicative of fullness of the row in the multiple snapshots; wherein the group of rows comprises column extents associated with column extents metadata; and (b) improving a column extent metadata parameter by reordering the rows of the group of rows according to the multi-snapshot row scores to provide a re-ordered group of rows that comprises re-ordered group extents associated with re-ordered column extents associated with re-ordered column extents metadata.