Database Engine Update Mechanism for Columnar Storage

The patent segments the update process into two independent phases: write phase where updates are appended to the tertiary structure without re-compression, and read phase where updates are sorted and merged into the secondary structure. This segmentation allows compression to be applied only once during initial data loading, while updates are handled separately without triggering re-compression, thus resolving the contradiction between maintaining compression benefits for reads and avoiding re-compression overhead for updates.

The patent performs preliminary sorting of updated values during the read phase before merging them into the secondary structure. By pre-sorting the update data in memory before the merge operation, the system avoids the need for re-compression during updates, as the sorted data can be efficiently integrated into the compressed secondary structure without decompressing or re-compressing the entire data block.

The patent applies local quality by maintaining different data structures for different operational needs: the primary structure stores compressed data for efficient reads, the secondary structure stores sorted updated values, and the tertiary structure stores unsorted update data. Each structure has optimized properties for its specific purpose, allowing updates to be applied locally to the tertiary and secondary structures without affecting the compression of the primary structure, thus maintaining data consistency while avoiding re-compression overhead.

The patent creates a copy of the update data in the tertiary structure as a separate data structure from the primary compressed structure. Instead of modifying the primary compressed structure directly (which would require re-compression), the system copies update data to the tertiary structure, then merges it with the secondary structure during reads. This copying approach maintains the integrity and compression of the primary structure while enabling efficient updates.