Automatic Cube Schema Inference from Tabular Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manually extracting a cube schema from tabular data for multidimensional databases is time-consuming and daunting, especially when dealing with hundreds of columns, due to the need to identify relationships and dependencies among columns.
Innovation Solution
A system and method that automatically infer a cube schema by calculating information gain ratios between columns, constructing a cross correlation matrix, and creating a hierarchy directed graph to represent relationships, which can be transformed into a cube schema for loading tabular data into a multidimensional database environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual schema extraction is used, then relationship identification accuracy is improved, but time consumption and operational complexity increase significantly
Solution Approach 1:
The system performs self-service by automatically inferring cube schemas from tabular data without requiring manual user intervention. The cube schema inference component autonomously calculates information gain ratios, constructs cross correlation matrices, determines column relationships, and generates hierarchy directed graphs, enabling the system to extract schemas itself rather than relying on manual user analysis.
Solution Approach 2:
The patent replaces the manual mechanical process of schema extraction with an automated computational system. Instead of users manually analyzing column relationships, the system uses information gain ratio calculations, cross correlation matrix construction, and automated relationship determination algorithms to substitute human analytical work with machine-based statistical analysis.
2Measurement precision
If manual schema extraction is used, then relationship identification accuracy is improved, but operational ease deteriorates
Solution Approach 1:
The system performs self-service by automatically inferring cube schemas from tabular data without requiring manual user intervention. The cube schema inference component autonomously calculates information gain ratios, constructs cross correlation matrices, determines column relationships, and generates hierarchy directed graphs, enabling the system to extract schemas itself rather than relying on manual user analysis.
Solution Approach 2:
The patent replaces the manual mechanical process of schema extraction with an automated computational system. Instead of users manually analyzing column relationships, the system uses information gain ratio calculations, cross correlation matrix construction, and automated relationship determination algorithms to substitute human analytical work with machine-based statistical analysis.
3Productivity
If automated inference is used, then productivity is improved, but measurement precision of relationships may deteriorate
Solution Approach 1:
The patent replaces the manual mechanical process of schema extraction with an automated computational system. Instead of users manually analyzing column relationships, the system uses information gain ratio calculations, cross correlation matrix construction, and automated relationship determination algorithms to substitute human analytical work with machine-based statistical analysis.
Solution Approach 2:
The system uses feedback mechanisms by calculating information gain ratios between column pairs and using these calculations to construct cross correlation matrices. The relationship determination process uses feedback from the matrix construction to refine and confirm column relationships, ensuring accurate schema inference through iterative statistical analysis.
4Ease of operation
If automated inference is used, then device complexity increases, but ease of operation is improved
Solution Approach 1:
The automated inference system is segmented into distinct functional components: the cube schema inference component that performs information gain ratio calculations, the cross correlation matrix construction module, the relationship determination engine, and the hierarchy directed graph generator. This segmentation allows each component to handle a specific aspect of schema extraction, making the overall complex process manageable and automated.
Data Source
AI summary
In accordance with an embodiment, described herein is a system and method for automatic inference of a cube schema from a tabular data for use in a multidimensional database environment. A cube schema inference component can successively perform column-wise splits to calculate information gain ratios between each pair of a plurality of columns in the tabular data. A cross correlation matrix can be constructed from the information gain ratios between each pair of columns. The system can determine relationships among the plurality of columns based on the cross correlation matrix; create a hierarchy directed graph to represent the relationships; and further transform the hierarchy directed graph into a cube schema that can be used to create a cube for loading the tabular data, or to map the tabular data into an existing cube in the multidimensional database environment.