Data Perturbation of Non-Unique Values for OLAP Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
OLAP applications face slow data aggregation processes, leading to prolonged query processing times for users accessing data warehouses, as existing methods are inefficient in handling non-unique values in data tables.
Innovation Solution
The method involves perturbing non-unique values in data tables to render them unique, allowing for faster aggregation by utilizing commands like SQL DISTINCT instead of costly CROSS JOIN operations, and pre-calculating results to reduce query processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If data aggregation is performed using traditional methods with non-unique values, then data accuracy is maintained, but query processing time increases significantly
Solution Approach 1:
The patent applies preliminary action by perturbing non-unique values in the data table before aggregation operations are performed. This pre-processing step modifies the data structure to enable faster subsequent queries, trading a small amount of initial processing for significant gains in query performance. The perturbation creates unique identifiers that allow direct aggregation without complex deduplication operations during query execution.
2Productivity
If non-unique values are used as identifiers, then data integrity is preserved, but aggregation operations become computationally expensive
Solution Approach 1:
The patent applies parameter changes by modifying the identifier values in the data table from non-unique to unique through perturbation. This changes the fundamental parameter of value uniqueness, which directly impacts aggregation complexity. By ensuring each identifier is unique, the system eliminates the need for expensive GROUP BY and DISTINCT operations, reducing computational complexity from O(n log n) to O(n).
Data Source
AI summary
In embodiments of the present invention, improved capabilities are described for perturbing non-unique values may comprise finding the non-unique values in a data table, perturbing the non-unique values to render unique values, and using the non-unique values as an identifier for a data item.