In-Memory DBMS Hierarchical Graph Factor Table Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generating aggregated reports for complex, hierarchically organized data with millions of items is computationally demanding in existing systems, especially when data topology changes frequently, leading to slow calculation speeds and inefficiencies in processing large datasets.
Innovation Solution
An in-memory database management system (DBMS) generates a hierarchical graph and transforms it into a factor table, allowing for efficient database-JOIN operations to aggregate data, separating topological information from transactional data and processing only relevant subsets, thereby reducing memory usage and increasing performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data aggregation is performed in the application layer for complex hierarchical data, then complex procedures can be implemented, but calculation speed is slow
Solution Approach 1:
The patent separates topological information (hierarchical structure) from transactional data, organizing them into distinct data structures. The hierarchical graph stores topological relationships while the data cube stores transactional data, allowing independent optimization of each structure and enabling efficient processing of complex aggregation procedures at database speed.
Solution Approach 2:
The patent introduces a hierarchical graph as an intermediary structure that bridges the relationship between transactional data and hierarchical organization. This graph serves as a mediator that enables the database system to understand and process hierarchical relationships without requiring complex application-layer procedures.
2Measurement precision
If all topological information is processed for data aggregation, then complete hierarchical aggregation is achieved, but processing load increases
Solution Approach 1:
The patent extracts only the relevant subset of topological information needed for a specific aggregation request. Instead of processing the entire hierarchical graph, the system identifies and processes only the portion of the hierarchy relevant to the query, significantly reducing processing load while maintaining aggregation completeness for the requested scope.
Solution Approach 2:
The patent implements partial processing by selecting a fraction of the topological information based on the specific aggregation request. The system processes only the necessary subset of hierarchical data required for the current query rather than performing exhaustive processing of all topological information, optimizing the balance between completeness and efficiency.
3Speed
If hierarchical graph processing is performed in memory, then real-time aggregation is achieved, but memory consumption increases
Solution Approach 1:
The patent segments the data storage into two distinct components: the hierarchical graph stored in memory for fast topological queries, and the transactional data cube stored in memory for efficient data retrieval. This segmentation allows the system to maintain only the essential topological structure in memory rather than duplicating all data, reducing overall memory consumption while enabling real-time processing.
Solution Approach 2:
The patent transforms the hierarchical structure into a graph representation with nodes and edges, adding a topological dimension to the data organization. This dimensional transformation enables efficient traversal and aggregation operations in memory by leveraging the graph structure's inherent properties for fast pathfinding and relationship resolution.
4Productivity
If database routines are used for data aggregation, then processing efficiency is improved, but complexity of hierarchical data cannot be handled
Solution Approach 1:
The hierarchical graph acts as an intermediary that translates complex hierarchical relationships into a form that standard database routines can process efficiently. The graph structure provides a universal representation that works with conventional SQL JOIN operations and database aggregation functions, eliminating the need for custom application-layer procedures while maintaining the ability to handle complex hierarchies.
Solution Approach 2:
The patent creates a universal data structure (hierarchical graph) that can handle various types of hierarchical relationships and aggregation scenarios through a single unified approach. This universal structure works with standard database routines for different aggregation needs, making the system versatile without requiring specialized processing logic for each hierarchical complexity scenario.
Data Source
AI summary
The present disclosure relates to a data aggregation method implemented by an in-memory DBMS (157). The DBMS comprises an in-memory database (156) with a transactional data cube (208), the cube comprising a plurality of data records (292). The method comprises: -generating (402) a hierarchical graph (258) comprising a plurality of interconnected nodes (N1-N14) and leafs (L1-L5), wherein at least one of the leafs is a direct child of two or more of the nodes, wherein each of the data records of the transactional data cube comprises an ID of one of the leafs; -receiving (404) a request (256) via a telecommunication network from a client device (140), the request comprising a node-ID (254); -in response to receiving the request, selecting (406) a sub-graph (272) from the graph, the root node (N7) of the sub-graph being identified by the node ID (254) of the request; -automatically generating (408) a factor table (166) as a derivative of the selected sub-graph, the factor table comprising one or more lines for each of the leafs of the sub-graph, wherein each of the lines of the factor table comprises a first and a second ID and a factor indicating to which degree a particular leaf or node having assigned the first ID is a direct or indirect child of a particular node of the sub-graph, said particular node having assigned the second-ID; -generating an aggregated report by joining the factor table and the transactional data cube in a database JOIN operation at least via the first ID, wherein only a factor-dependent fraction of the transactional data is aggregated in accordance with the topology of the sub-graph; and -returning (414) the aggregated report comprising the aggregated data to the client device.