Intent-Based Enterprise Data Governance with Modular Data Products
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data management systems face challenges in delivering timely insights due to data governance limitations, making it difficult to unlock value from data and manage complex data across multiple countries.
Innovation Solution
An intent-based data management system using a data mesh architecture that enforces policies and rules as code through a decentralized domain-driven dataset approach, ensuring self-contained, self-aware, and self-secure data products by encapsulating modules for data standardization, sovereignty, and anonymization, and applying declarative definitions of rules and policies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a centralized data management system is used to enforce data governance policies, then policy compliance is improved, but system complexity and difficulty in managing multi-country data increases
Solution Approach 1:
The patent segments data governance into independent data products, each with its own governance policies and metadata. This decomposition allows policies to be enforced at the data product level rather than requiring a monolithic centralized system, reducing overall system complexity while maintaining compliance through modular, self-contained governance units.
Solution Approach 2:
Data products are designed to be self-aware and self-secure, automatically enforcing their own governance policies through embedded metadata and intent-based rules. This self-service capability eliminates the need for complex centralized oversight mechanisms, as each data product independently ensures its own compliance with relevant policies.
2Reliability
If traditional data management approaches are used, then data governance control is maintained, but the speed of delivering timely insights decreases
Solution Approach 1:
Governance policies and rules are defined in advance as intent metadata associated with each data product. This preliminary configuration of governance parameters allows data products to automatically enforce compliance without real-time intervention, enabling rapid data access and insight delivery while maintaining pre-established governance controls.
Solution Approach 2:
The system dynamically evaluates requests against intent metadata and policy metadata at runtime, allowing flexible and adaptive governance enforcement. This dynamic approach replaces static, rigid control mechanisms with responsive evaluation that maintains governance integrity while enabling timely insights through automated real-time decisioning.
3Manufacturing precision
If detailed policy enforcement is applied to all data requests, then data governance accuracy is improved, but processing time and system overhead increases
Solution Approach 1:
Each data product has its own localized intent metadata containing governance parameters specific to that data product's requirements. This local quality approach allows precise governance enforcement tailored to each data product without requiring uniform detailed inspection of all data requests, reducing processing overhead while maintaining accuracy through context-specific rules.
Solution Approach 2:
The system applies governance evaluation selectively based on the specific request and data product context, rather than uniformly enforcing all possible policies on every request. This partial action approach focuses computational resources on relevant policy checks, maintaining high governance accuracy while minimizing unnecessary processing time and system overhead.
Data Source
AI summary
Each module of a plurality of modules receives intent metadata associated with a data product instance of a data mesh. Each module corresponds to a unique data governance category of a plurality of data governance categories and a request associated with data was intercepted at the data product instance. Each module receives policy metadata associated with the data governance category corresponding to the module. Each modules determines, based on the intent metadata and the policy metadata, whether the request is valid for the data governance category corresponding to the module. Each modules transmits an indication of whether the request is valid for the module.