Metadata-Driven Data Warehouse Automation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data warehouses are costly and difficult to adapt, often requiring multiple specialists for months to create and are not easily adaptable to changes, with existing ETL tools not satisfactorily automating best practices for data integration and reporting.

Innovation Solution

A packaged data warehouse solution system using a metadata model, user interface, and engine to generate and manage a data warehouse from multiple data source systems, incorporating a metadata model for business logic and information needs, and a user interface for manipulating metadata to automate data transformation and reporting.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If a data warehouse is created using traditional data-driven approach with multiple specialists, then the data warehouse can be built, but it requires months of work and high cost

Engineering Contradiction:

Improvedata warehouse creation speedVSAvoidtime required for data warehouse creation

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-defining metadata models, data information models, and ETL process templates before actual data warehouse creation. These pre-configured frameworks contain standard data structures, transformation rules, and best practices that can be directly applied to new projects, eliminating the need to build everything from scratch and reducing implementation time from months to days.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent uses copying by providing reusable metadata models and ETL process templates that can be replicated across different data warehouse projects. These templates encapsulate proven patterns and structures that can be copied and adapted to new requirements, significantly accelerating the data warehouse creation process while maintaining quality standards.

Inventive Principle:

Principle #26Copying

2Adaptability or versatility

If a data warehouse is built using custom ETL processes, then the data warehouse can be created, but it is difficult to adapt to changes

Engineering Contradiction:

Improveadaptability to changesVSAvoidcomplexity of ETL processes

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent applies dynamics by making the ETL processes configurable and adaptable through metadata-driven definitions. The system allows dynamic modification of data extraction, transformation, and loading parameters without requiring fundamental process redesign. Users can adjust metadata models and ETL configurations to accommodate changing business requirements, making the system flexible and adaptable to future changes.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent uses segmentation by breaking down the ETL process into distinct, independently configurable components represented as metadata objects. Each ETL process is divided into extract, transform, and load phases with separate configurable parameters, allowing individual components to be modified without affecting the entire system. This modular approach simplifies adaptation to changes while maintaining overall system integrity.

Inventive Principle:

Principle #1Segmentation

3Extent of automation

If existing ETL tools are used, then data extraction and transformation can be performed, but they do not satisfactorily automate best practices

Engineering Contradiction:

Improveautomation of ETL processesVSAvoidease of implementing best practices

Core Design Contradiction:

Extent of automationVSEase of manufacture

Solution Approach 1:

The patent applies self-service by enabling automated generation of ETL processes from predefined metadata models that encode best practices. The system automatically creates extraction, transformation, and loading logic based on metadata definitions, eliminating the need for manual configuration and ensuring that industry best practices are consistently applied. This automation reduces human error and ensures compliance with established standards.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent uses parameter changes by transforming static ETL configurations into dynamic, metadata-driven processes. The system allows parameters such as data sources, transformation rules, and target structures to be defined and modified through metadata models, enabling flexible automation that adapts to different scenarios while maintaining consistent application of best practices across all ETL operations.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS7720804B2Method of generating and maintaining a data warehouse

Publication Date: 2010.05.18 X CORP

AI summary

A data warehouse solution system comprises a metadata model, a user interface and an engine. The metadata model has an information needs model including metadata regarding information needs for building reports by users, and a data information model including metadata describing data that is available for building reports. The user interface has a customer user interface for presenting the information needs model to the users for report generation, and a modeling user interface for presenting the data information model to the users for manipulating data warehouse objects. The engine has a report management service unit for providing report management service using the information needs model, and a data management service unit for providing data management service including generation of a data warehouse using the data information model.