Database Aggregate Calculation via Processing Unit Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems lack the processing power and memory capacity to efficiently compute aggregates for all possible column pair combinations, leading to excessive data swapping and slow performance in large-scale data analysis scenarios.
Innovation Solution
A method is introduced that divides processing units into multiple groups, replicates data within each group, and employs a two-stage aggregation algorithm to reduce the number of passes required for calculating aggregates, allowing for efficient computation of aggregates across multiple column combinations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a SQL query is issued to compute aggregates for all possible column combinations, then the completeness of data analysis is improved, but the processing time and memory requirements increase excessively
Solution Approach 1:
The patent divides the set of processing units into multiple groups, where each group computes aggregates for a specific subset of column combinations. This segmentation allows the system to distribute the computational workload across multiple groups, reducing the processing time for each individual group while maintaining completeness through the collective results of all groups.
Solution Approach 2:
The patent introduces a new dimension by organizing processing units into groups and assigning different subsets of column combinations to each group. This dimensional organization transforms the problem from a single large computation to multiple smaller computations across different dimensions, improving overall processing efficiency.
2Measurement precision
If data is replicated to all processing units for aggregate computation, then the computation accuracy is improved, but the memory consumption increases excessively
Solution Approach 1:
The patent implements local quality by having each processing unit replicate data only to other units within its own group, rather than replicating to all processing units. Each group maintains its own replicated data for accurate computation of its assigned column combinations, while avoiding the memory overhead of system-wide replication.
Solution Approach 2:
By segmenting processing units into groups and assigning specific column combination subsets to each group, the patent reduces the scope of data replication. Each group only replicates data needed for its specific computation task, significantly reducing overall memory consumption while maintaining computation accuracy for the assigned subsets.
3Power
If the number of processing units is increased, then the computational power is improved, but the system complexity increases
Solution Approach 1:
The patent segments the processing units into multiple groups, which simplifies the system architecture by introducing a hierarchical organization. This segmentation allows the system to scale computational power by adding more groups rather than increasing the complexity of individual processing units, making the system more manageable and easier to deploy.
Data Source
AI summary
In a database system having processing units, a request is received to calculate aggregates of multiple combinations of a given set of columns. The processing units are divided into multiple groups, where each of the multiple groups includes a corresponding subset of the processing units. Within each of the groups, data of each processing unit in the group is replicated to other one or more processing units in the group. Within each of the groups, aggregates of at least some of the multiple combinations of the given set of columns are calculated by the processing units in the group using the replicated data.