Parallel Insert Sub-ranges for Column Store Concurrency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Column store databases face inefficiencies in handling concurrent insert operations due to serialization requirements, leading to potential data corruption and increased overhead, as only one application can insert rows at a time without proper serialization.
Innovation Solution
The implementation of parallel insert sub-ranges divides the Tuple Sequence Number (TSN) range into sub-ranges, allowing multiple transactions to insert data concurrently while maintaining serialization within each sub-range, thereby enabling parallel processing of insert operations across different sub-ranges.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If serialization is used to prevent data corruption during insert operations, then data integrity is maintained, but concurrency and insert efficiency deteriorate
Solution Approach 1:
The patent divides the column store into multiple sub-ranges (first sub-range, second sub-range, etc.) that can be independently accessed. By segmenting the data storage space, multiple applications can insert data into different sub-ranges simultaneously without conflicting, thus maintaining data integrity while enabling parallel insert operations and improving overall insert efficiency.
2Reliability
If locking mechanisms are implemented to ensure safe concurrent access, then data corruption is prevented, but system overhead and complexity increase
Solution Approach 1:
By dividing the column store into distinct sub-ranges and allowing each application to access specific sub-ranges without requiring global locks, the patent eliminates the need for complex locking mechanisms. This segmentation approach prevents data corruption through spatial isolation rather than temporal coordination, significantly reducing system overhead and complexity.
3Productivity
If staging tables are used for insert operations, then concurrent insert capability is improved, but the number of tables and query complexity increase
Solution Approach 1:
The patent segments the single column store into multiple sub-ranges that can be concurrently accessed, eliminating the need for separate staging tables. This approach maintains concurrent insert capability while avoiding the complexity of managing multiple tables and the additional overhead of copying data between staging and target tables.
Data Source
AI summary
Provided are techniques for using parallel inserts to insert into a column store. A list of sub-ranges for a columnar table is created. A change statement for one or more rows to be inserted into the columnar table is received. For each of the received change statements, a sub-range from the list of sub-ranges is identified and data is serially inserted into the identified sub-range of the columnar table, while allowing parallel processing of each change statement for different sub-ranges of the columnar table.