Virtual Cube Real-Time Data Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional multidimensional database systems face performance issues due to the need to load all source data, which becomes unwieldy with the increasing volume of big data, leading to reduced performance and increased storage footprint.
Innovation Solution
Implementing real-time data aggregation in a virtual cube, where the multidimensional database server queries and aggregates data from external sources on-demand, reducing the need to store all data locally and allowing only targeted data to be accessed and processed, thereby minimizing storage footprint and enhancing performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all source data is loaded into the multidimensional database, then data access and analysis can be performed, but storage footprint increases and performance decreases with big data volumes
Solution Approach 1:
The patent extracts only the necessary data from external sources on-demand rather than loading all source data. The virtual cube queries external data sources dynamically, pulling only the specific data needed for current analysis, thereby eliminating the need to store entire data sources locally while maintaining data accessibility.
Solution Approach 2:
The system transitions from a static data loading model to a dynamic on-demand querying model. The virtual cube adapts its data retrieval based on real-time query requirements, dynamically fetching only the necessary subsets of data from external sources rather than maintaining fixed pre-loaded datasets.
2Productivity
If all source data is loaded into the multidimensional database, then complete data analysis is enabled, but system performance deteriorates with increasing data volume
Solution Approach 1:
The virtual cube extracts and processes only the specific data subsets required for each analysis query rather than operating on complete loaded datasets. This extraction approach maintains comprehensive data analysis capability while significantly reducing the volume of data that must be processed in memory, thereby preserving system performance.
Solution Approach 2:
The system performs partial data retrieval - fetching only the necessary portion of data needed for each specific analysis rather than loading or processing complete datasets. This partial action approach enables complete analytical capability through iterative queries while avoiding the performance penalty of handling full data volumes simultaneously.
3Reliability
If data is accessed in real-time from external sources, then up-to-date analysis is achieved, but data access complexity increases
Solution Approach 1:
The virtual cube acts as an intermediary layer between end-users and external data sources. It abstracts the complexity of real-time data access by providing a standardized querying interface, while internally managing the complex interactions with multiple external sources, thereby enabling real-time data freshness without exposing access complexity to users.
Solution Approach 2:
The virtual cube provides a universal interface for accessing diverse external data sources through standardized multidimensional querying. This multi-functional capability allows the same interface to access different data sources with varying access mechanisms, hiding the underlying complexity while maintaining real-time data access across multiple sources.
Data Source
AI summary
In accordance with an embodiment, described are systems and methods for real time data aggregation in a virtual cube in a multidimensional database. A method can provide, at a computer that includes one or more microprocessors, a multidimensional database server executing on the computer, wherein the multidimensional database server supports at least one multidimensional cube, a data source associated with the multidimensional cube, and at least one source of real-time data accessible by the data source. The method can query, by the multidimensional cube, the data source, the query comprising an identification of the at least one source of real-time data. The method can provide, by the data source, on-demand access to the multidimensional cube to the at least one source of real-time data. The method can use targeted data from the at least one source of real-time data within the multidimensional cube without being stored therein.