Multi-Tiered Metadata Model for Automatic Governance Code Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data governance methods become inefficient and error-prone as data volumes and complexities increase, leading to inconsistencies and errors due to the need for manual definition of controls for each dataset, making it unsustainable and costly.
Innovation Solution
A development environment that uses a multi-tiered metadata model to define controls at a logical level, enabling automatic propagation to multiple datasets, incorporating metadata characteristics to ensure accurate and efficient data governance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual processes are used to define controls for each dataset, then data governance can be implemented, but efficiency decreases and errors increase as data volumes and complexities increase
Solution Approach 1:
The patent segments data governance into two distinct layers: metadata layer (logical level) and data layer (physical level). Controls are defined once at the metadata layer and automatically propagated to multiple datasets at the data layer, eliminating the need to manually define controls for each dataset while maintaining governance accuracy.
Solution Approach 2:
The patent introduces metadata as an intermediary layer between control definitions and actual data. This metadata model acts as a mediator that stores control definitions and automatically applies them to multiple datasets, reducing manual intervention and improving efficiency while maintaining reliability.
2Reliability
If controls are defined manually for each dataset, then specific data requirements can be addressed, but the process becomes unsustainable and costly as data complexity increases
Solution Approach 1:
The patent enables preliminary definition of controls at the metadata layer, which then automatically propagate to all relevant datasets. This preliminary action eliminates the need for repeated control definitions when new datasets are added, significantly reducing time loss while maintaining accurate control application.
Solution Approach 2:
The metadata model provides a universal control definition mechanism that can be applied across multiple datasets simultaneously. A single control definition at the metadata layer serves multiple datasets, making the governance process sustainable and cost-effective even as data complexity increases.
3Ease of operation
If manual data governance processes are used, then implementation is feasible for smaller environments, but inconsistencies and errors compromise governance integrity as data volumes increase
Solution Approach 1:
The system enables self-service data governance by automatically propagating control definitions from the metadata layer to multiple datasets without manual intervention. This automated self-service mechanism maintains governance integrity across large data volumes while keeping the system easy to operate through centralized metadata management.
Data Source
AI summary
A method for using a development environment to automatically generate code from a multi-tiered metadata model includes: receiving a specification to process a dataset, and, in response, accessing dataset characteristics and identifying controls received from a development environment to be applied to a field of the dataset in accordance with a metadata model by: accessing a first instance of a data structure that corresponds to the dataset; based on a reference in the first instance, accessing a second instance of a data structure associated with the field; based on a reference in the second instance, accessing a third instance of a data structure associated with metadata describing the field, and based on a reference in the third instance, accessing a fourth instance of a data structure storing a control defined based on the metadata. Based on the dataset characteristics, code is generated to apply the identified control to the field.