Database Repeated Values Update via Append-Only Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional relational databases face challenges in efficiently storing and updating repeated values, particularly due to the need for shifting data structures and handling variable-size character columns, which leads to computational inefficiencies and limitations in supporting flexible schema and extended data types.
Innovation Solution
The implementation of a columnar in-page representation with append-only techniques for updating repeated values, using a repeated map and operational position map to manage updates independently at the record level, and periodically merging these updates with base data to minimize restructuring and optimize storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional relational databases store repeated values in variable-size character columns, then the database can support flexible data formats, but the computational cost increases due to data shifting operations
Solution Approach 1:
The patent segments the data storage structure into fixed-size character columns, where each column stores a specific number of characters. This segmentation eliminates the need for data shifting operations when updating repeated values, as each column has a predetermined size. The segmentation principle resolves the contradiction by maintaining flexibility through multiple columns while improving computational efficiency by avoiding costly shift operations.
Solution Approach 2:
The patent introduces an intermediary mechanism in the form of a repeated values manager that coordinates updates across multiple fixed-size columns. This intermediary layer handles the complexity of managing repeated values without requiring direct manipulation of the underlying data structures, thereby reducing computational overhead while maintaining adaptability.
2Adaptability or versatility
If the database uses variable schema to support extended data types, then the database becomes more versatile, but the complexity of managing data structures increases
Solution Approach 1:
The patent divides the storage structure into multiple fixed-size character columns, each capable of holding portions of repeated values. This segmentation allows the database to support extended data types and variable schemas while maintaining simple, uniform column structures. The complexity of managing variable schemas is reduced by breaking down complex data into manageable fixed-size segments.
Solution Approach 2:
Instead of using variable-size columns to achieve flexibility, the patent inverts the approach by using fixed-size columns in combination. This inversion simplifies the underlying data structure management while achieving the same flexibility goal, thereby reducing device complexity while maintaining adaptability.
3Reliability
If the database performs frequent updates to repeated values, then the data remains current, but the computational cost and time increase due to restructuring operations
Solution Approach 1:
By segmenting repeated values into multiple fixed-size columns, the patent enables updates to be performed without restructuring operations. Each column maintains a fixed size, allowing direct updates to repeated values without the need to shift data or reorganize the storage structure, thereby reducing update time while maintaining data currency.
Solution Approach 2:
The patent performs preliminary actions by pre-defining the size and structure of character columns before data insertion. This preliminary structuring eliminates the need for dynamic restructuring during updates, allowing frequent updates to be performed efficiently while maintaining current data.
4Productivity
If the database uses append-only techniques with periodic merging, then the update cost is reduced, but the complexity of managing multiple data pages increases
Solution Approach 1:
The patent segments data into base pages and append-only pages, with each page containing fixed-size character columns. This segmentation allows updates to be performed efficiently on append-only pages without affecting the entire data structure. The periodic merging of append-only pages with base pages is simplified by the uniform fixed-size structure, reducing page management complexity while improving update efficiency.
Solution Approach 2:
The patent implements periodic merging of append-only pages with base pages at predetermined intervals. This periodic action allows the database to accumulate updates efficiently without continuous restructuring, reducing update operation complexity while maintaining data consistency. The fixed-size column structure simplifies the merging process by providing a uniform target structure.
Data Source
AI summary
Data within a database is partitioned into one or more sets of read-only data pages and one or more sets of append-only data pages, wherein each set of read-only data pages is associated to one of the one or more sets of append-only data pages. A repeated map is maintained for at least one of the one or more sets of append-only data pages. An operational position map is maintained for at least one of the one or more sets of append-only data pages. A request is received to update one or more records in a given set of read-only pages, wherein at least one of the one or more records comprises repeated values. The repeated map and operational position map of the append-only pages associated with the given set of read-only pages are updated in response to the request. The one or more sets of append-only pages are automatically merged with their respective set of read-only pages on a periodic basis to create one or more new sets of read-only pages.