Virtual Aggregation Framework for Hierarchical Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Performing data aggregation on hierarchical data in relational databases is challenging due to resource intensity and the difficulty in describing or querying such data using Structured Query Language (SQL), particularly with large volumes of data.
Innovation Solution
A framework that receives a data aggregation request, identifies previously generated aggregation hierarchies through a configuration table, and maps attributes to create a target aggregation hierarchy by applying aggregation methods to nodes and their children, reusing existing calculations to generate reports efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data aggregation is performed on hierarchical data using traditional SQL methods, then the aggregation can be completed, but the computational resource usage becomes excessive and performance degrades with large volumes of data
Solution Approach 1:
The system pre-generates and stores aggregation hierarchies in a configuration table before they are needed. When a data aggregation request is received, the system queries the configuration table to identify previously generated aggregation hierarchies that correspond to the request, and reuses the pre-computed attribute values directly without performing new aggregation calculations. This preliminary computation and storage approach eliminates redundant computational resource usage while maintaining high aggregation performance.
2Ease of operation
If hierarchical data is queried using traditional SQL methods, then the data can be retrieved, but the difficulty of describing and querying hierarchical structures increases complexity
Solution Approach 1:
The system introduces a configuration table as an intermediary layer between the hierarchical data source and the query interface. This configuration table stores pre-computed aggregation hierarchies with mapped attributes in a flattened structure. When queries are received, the system translates them into configuration table queries rather than direct hierarchical data queries, simplifying the query structure while maintaining ease of operation for users working with hierarchical data.
3Measurement precision
If aggregation calculations are performed on all hierarchical data, then complete aggregation results are obtained, but the time required increases significantly with large data volumes
Solution Approach 1:
The system applies local quality by storing and reusing aggregation results at specific hierarchical levels and scopes in the configuration table. When a data aggregation request is received, the system queries the configuration table to identify previously generated aggregation hierarchies that correspond to the request, and reuses the pre-computed attribute values directly. This localized reuse of pre-computed results at appropriate hierarchical levels maintains aggregation completeness while dramatically reducing computation time for large data volumes.
Data Source
AI summary
A framework for performing aggregation methods on hierarchical data objects is described herein. A data aggregation request is received, the request including a tree of nodes to aggregate, an aggregation method, and a filter criteria. A configuration table is used to identify previously generated aggregation hierarchies that correspond to the received request, and a mapping of attributes of the received nodes to attributes of the previously generated aggregation hierarchies. For each node in the tree of nodes, when a corresponding node in the previously generated aggregation hierarchy is identified, attribute values from the corresponding node are used to create a node in a target aggregation hierarchy. When a corresponding node does not exist, but corresponding child nodes do exist in the previously generated aggregation hierarchy, the aggregation method is applied to the corresponding child nodes and the result is used to create a node in the target aggregation hierarchy.