Dynamic Data Aggregation via Metadata-Driven Access Plans
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Standard approaches to updating dimensions in a data warehouse do not automate and scale well, requiring manual updates across all four tiers of the architecture, especially when new multidimensional data models are established or organizations merge, leading to time-consuming modifications in the information access layer.
Innovation Solution
A computer-implemented method and system that dynamically aggregates data from a plurality of distributed data sources without a centralized multidimensional data warehouse, generating a data access plan based on metadata aggregation requirements to fetch data on-demand, keeping business intelligence applications updated in near real-time through a subscription program.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a centralized multidimensional data warehouse is used to store and manage enterprise data, then data analysis and reporting can be performed, but the system requires manual updates across all four tiers of architecture when new data models are established or organizations merge, leading to time-consuming modifications
Solution Approach 1:
The system enables self-service through automated metadata-driven data aggregation. The server automatically generates data access plans and fetches data from distributed sources based on metadata definitions, eliminating the need for manual updates across architecture tiers when new data models are introduced or organizations merge.
Solution Approach 2:
The patent implements dynamic data aggregation where the data access plan is generated and executed on-demand based on subscriber requests. The system dynamically adapts to new data sources and metadata definitions without requiring static pre-configuration, allowing rapid response to organizational changes and new data models.
2Stability of the object's composition
If manual updates are performed across all four tiers of data warehouse architecture, then data consistency is maintained, but the process does not automate and scale well, requiring significant effort especially when new multidimensional data models are established
Solution Approach 1:
The patent introduces metadata as an intermediary layer that mediates between data sources and business intelligence applications. The metadata definitions serve as the single source of truth for data aggregation requirements, automatically translating into data access plans without manual intervention across architecture tiers.
Solution Approach 2:
The system segments the data warehouse architecture into independent distributed data sources that communicate through standardized metadata interfaces. This segmentation allows each data source to be updated independently while maintaining overall data consistency through the metadata-driven aggregation mechanism.
3Adaptability or versatility
If a centralized data warehouse is used, then data can be accessed for reporting, but the system lacks scalability and requires significant modifications in the information access layer when dimensional data attributes change
Solution Approach 1:
The patent creates a universal metadata-driven data aggregation system that can handle multiple data sources, different data models, and various organizational structures through a single standardized interface. The same metadata definitions and data access plan generation mechanism work across diverse data sources without requiring system modifications.
Data Source
AI summary
Dynamically aggregating data is provided. A server device receives a subscriber request for a report based on a subset of metadata contained in a data dimensions catalog. The server device analyzes data aggregation requirements from a plurality of data sources for the report based on the subset of metadata defined in the subscriber request. The server device generates a data access plan for movement of data from the plurality of data sources based on the data aggregation requirements for the report. Then, the server device executes the data access plan to fetch the data from the plurality of data sources based on the data aggregation requirements for the report.