Automated Hierarchy Detection in Cloud Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data warehouse and OLAP systems face challenges in automatically detecting and modeling dimensions and hierarchies from diverse data sources, especially without formal definitions, and in incorporating real-time data for enhanced analytics.
Innovation Solution
A cloud-based analytics platform automates the discovery and modeling of dimensions and hierarchies by using data sampling, probing, dimension modeling, hierarchy modeling, and incorporating real-time data streams to create n-dimensional cubes, leveraging existing dimensions and hierarchies, and referencing additional data sources for enhanced data analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated hierarchy detection is implemented in cloud-based analytics, then productivity and ease of operation are improved, but device complexity increases
Solution Approach 1:
The system performs self-service by automatically detecting hierarchies and dimensions from data sources without requiring manual configuration. The automated hierarchy detection process examines data sources, identifies potential hierarchies through probing and sampling, and configures multidimensional cubes automatically, eliminating the need for data analysts to manually define these structures.
Solution Approach 2:
The system performs preliminary actions by pre-configuring standard hierarchies and dimensions that can be automatically applied to common data sources. This includes pre-defined hierarchy templates and dimension models that are prepared in advance and can be automatically matched to incoming data, reducing the complexity of real-time analysis.
2Adaptability or versatility
If diverse data sources are integrated with real-time streaming, then adaptability and analytics capability are improved, but difficulty of detecting and measuring increases
Solution Approach 1:
The system achieves universality by implementing a unified automated hierarchy detection process that works across multiple data source types including relational databases, non-relational databases, and real-time streaming sources. The same probing and sampling mechanisms are applied universally to detect hierarchies regardless of the underlying data source format or structure.
Solution Approach 2:
The system applies dynamics by adapting its detection strategy based on the characteristics of each data source. The probing and sampling processes dynamically adjust their approach when dealing with real-time streaming data versus static database sources, modifying detection parameters and methods to suit the specific data source being analyzed.
Data Source
AI summary
A platform for data analytics may be provided in a hosted environment on a multi-tenant system. The platform provider may also provide transactional processing services. Data obtained from processing the transactional services may be stored in an n-dimensional cube with which analytics may be performed. A dimension and hierarchy model may be identified based on correlations between hierarchy dimensions and levels in a dataset, or in schema and queries related to the dataset. Correlations may be further based on data received from a data stream. Priority for calculating a hierarchy may be based on data received from a data stream.