Partition-Local Value Identifier Mapping for Faster GroupBy Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face inefficiencies in processing queries across partitioned tables due to mismatched value identifiers (VIDs) across fragments, leading to increased computational resources and time, especially when grouping data values.
Innovation Solution
A query processing system generates partition value identifiers (PartVIDs) that map fragment-local VIDs to partition-local identifiers, maintaining a mapping across fragments to facilitate efficient data grouping without reading actual data values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If fragment-local value identifiers are used in partitioned tables, then data storage flexibility is improved, but query processing efficiency deteriorates due to mismatched identifiers across fragments
Solution Approach 1:
The patent introduces partition value identifiers (PartVIDs) as an intermediary layer between fragment-local VIDs and query processing operations. The mapping table stores PartVIDs that correspond to fragment-local VIDs, allowing queries to group data by PartVID without directly handling mismatched fragment-local identifiers. This intermediary mapping mechanism resolves the contradiction by enabling efficient query processing while preserving the flexibility of fragment-local storage.
2Measurement precision
If actual data values are read for grouping operations, then grouping accuracy is improved, but computational resources and processing time increase
Solution Approach 1:
The patent extracts only the necessary identifier information (PartVIDs) from the partitioned table for grouping operations, rather than reading actual data values. The mapping table provides direct access to PartVIDs that represent the grouping keys, allowing the system to perform accurate grouping based on identifiers alone. This extraction approach maintains grouping accuracy while significantly reducing computational resource consumption by avoiding full data value reads.
Data Source
AI summary
A method for executing a query may include generating a partition value identifier for a partitioned table. The partitioned table may include a main fragment including a main dictionary storing a first value and a main value identifier corresponding to the first value and a delta fragment including a delta dictionary storing a second value and a delta value identifier corresponding to the second value. The partition value identifier may be set based at least in part on the first value and the second value. The generated partition value identifier and a corresponding one of the main value identifier and the delta value identified may be maintained as part of a mapping. A query to group data stored in the partitioned table may be received. The query may be executed by at least using the mapping.