Using column namespaces to avoid fragmentation of disk memory and row-level persistent version store for multi-version concurrency control for tables stored in columnar file format
By employing column namespaces and a row-level Persistent Version Store, the challenges of disk memory fragmentation and concurrency control for columnar file formats are addressed, resulting in improved storage efficiency and utilization.
Patent Information
- Application Number
- PCT/IB2023/057385
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-07-20
- Publication Date
- 2025-01-23
AI Technical Summary
Existing technologies face challenges in managing disk memory fragmentation and optimizing storage utilization for tables stored in columnar file formats, particularly in maintaining efficient concurrency control.
The use of column namespaces to uniquely identify columns or groups of columns, combined with a row-level Persistent Version Store (PVS) for Multi-Version Concurrency Control (MVCC), helps manage disk memory by maintaining a list of free blocks for each column namespace and allocating new pages only when necessary.
This approach effectively avoids disk memory fragmentation, enhances disk utilization, and ensures efficient concurrency control by maintaining previous rows or old data tagged with the table namespace.
Description
[0001] Using Column Namespaces to Avoid Fragmentation of Disk Memory and Row-Level Persistent Version Store for Multi- Version Concurrency Control for Tables Stored in Columnar File Format
[0002] Here in this invention, for tables stored in columnar file format, we use column namespaces to unique identify columns or group of columns and maintain list of free blocks for the pages assigned to the unique column namespace. The above said list helps to avoid fragmentation of disk memory resulting in better disk utilisation. Basically the storage management layer of the database first assigns free blocks of memory from the already assigned pages for new “column data” and, allocates and assigns new set of pages (like extents in open source MySQL for a tablespace) to the column namespace only when the free blocks for that column namespace is exhausted or in cases of variable length columns requiring more memory. Further for Multi- Version Concurrency Control (MVCC) we maintain a row-level Persistent Version Store (PVS) where previous rows or old data is tagged with the table namespace to differentiate between table data stored in row-oriented format and column-oriented format.
Claims
Following are the claims for this invention1. Here, for tables stored in columnar file format, we use column namespaces to unique identify columns or group of columns and maintain list of free blocks for the pages assigned to the unique column namespace. The above said list helps to avoid fragmentation of disk memory resulting in better disk utilisation.
2. Basically the storage management layer of the database first assigns free blocks of memory from the already assigned pages for new “column data” and, allocates and assigns new set of pages (like extents in open source MySQL for a tablespace) to the column namespace only when the free blocks for that column namespace is exhausted or in cases of variable length columns requiring more memory.
3. Further for Multi- Version Concurrency Control (MVCC) we maintain a row-level Persistent Version Store (PVS) where previous rows or old data is tagged with the table namespace to differentiate between table data stored in row-oriented format and column-oriented format.