Automated Data Model Transformation via Directive Templates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The process of transforming data models from one data structure to another, such as creating a data warehouse, is complex and prone to errors, requiring significant time and effort, especially when optimizing for different purposes like transactional processing and reporting, and often involves manual writing of schema and ETL code.
Innovation Solution
The system generates code to create a target data model by receiving directive information that maps a source data model to a target data model using predefined patterns, allowing for automated generation of schema and ETL code, reducing the need for manual intervention and minimizing errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If manual code writing is used for data model transformation, then flexibility and customization are improved, but time consumption and error rate increase
Solution Approach 1:
The patent uses template copying where pre-defined code templates store common data model transformation patterns. The system copies and adapts these templates automatically during transformation, eliminating manual code writing while preserving customization through template selection and parameter configuration.
Solution Approach 2:
The patent performs preliminary action by pre-defining transformation patterns, mappings, and code templates before actual data model transformation occurs. This preparation work is done once and reused multiple times, significantly reducing repeated manual effort while maintaining flexibility.
2Adaptability or versatility
If manual code writing is used for data model transformation, then adaptability is improved, but error rate increases
Solution Approach 1:
By copying proven, tested code templates rather than writing new code manually, the system inherits the reliability of previously validated transformations. This reduces errors while maintaining adaptability through the ability to select and configure appropriate templates.
Solution Approach 2:
The system incorporates feedback mechanisms that validate transformations against defined patterns and constraints, automatically detecting and correcting errors. This feedback loop ensures reliability while allowing flexible customization through configurable parameters.
3Productivity
If automated code generation is used, then productivity is improved, but complexity of the system increases
Solution Approach 1:
The patent segments the transformation system into modular components: pattern definitions, template libraries, configuration interfaces, and execution engines. This segmentation manages complexity by organizing functions into independent, reusable modules while maintaining high productivity through automated generation.
Solution Approach 2:
The system uses universal templates and patterns that can handle multiple transformation scenarios through configuration rather than requiring separate custom code for each case. This universality reduces system complexity while maintaining versatility and high productivity.
4Productivity
If automated code generation is used, then productivity is improved, but manual intervention requirements change
Solution Approach 1:
The patent introduces an intermediary configuration layer between automated generation and user needs. Users interact with high-level configuration parameters and template selections rather than raw code, making automated generation easier to operate while maintaining productivity benefits.
Data Source
AI summary
Described herein are systems and methods of transforming data models, for example, creating a data warehouse. A directives model may be loaded based upon a parsed directives file. An entity model may be loaded, and tables, attributes, and foreign keys of a data warehouse model may be created based upon the directives model and the entity model. Mappings may be created between tables, columns, and foreign keys of the data warehouse model and entities, attributes, and relationships, respectively, of the entity model. Code to define a data warehouse may be generated based upon the tables, attributes, and foreign keys of the data warehouse model. Code to transfer data from the source data source can be generated based upon the created mappings. A lineage report can be generated that provides information identifying a corresponding source for each table and column in the data warehouse.