Zone-Based Data Governance with Predictive Dataset Assignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data governance systems lack an efficient and automated process for managing data assets, leading to unnecessary resource expenditure and inefficiencies in data management, security, and compliance.
Innovation Solution
Implementing a zone-based database management system that categorizes data into transient, raw, trusted, refined, and analytical zones, with a graphical user interface for visualization and policy-driven data movement, utilizing machine learning for predictive zone assignment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional data governance systems are used, then data management processes are established, but resource expenditure is excessive and efficiency is low
Solution Approach 1:
The patent segments data into five distinct zones (transient, raw, trusted, refined, and analytical) based on data quality and processing stage. This segmentation enables targeted management strategies for each zone, improving efficiency by applying appropriate governance measures only where needed rather than uniformly across all data, thus reducing overall resource expenditure while maintaining high productivity.
2Extent of automation
If manual data governance processes are used, then data policies can be implemented, but automation and scalability are limited
Solution Approach 1:
The patent implements self-service mechanisms where the system automatically classifies incoming data into appropriate zones based on predefined criteria and policies. Data movement between zones is automated through policy-driven rules, reducing manual intervention. This self-service approach increases automation extent while managing system complexity through standardized, rule-based processes rather than complex custom workflows.
3Reliability
If data is stored in a single location, then storage is simple, but data security and access control are compromised
Solution Approach 1:
The patent divides the storage architecture into five separate zones (transient, raw, trusted, refined, and analytical), each with distinct security requirements and access controls. This segmentation enhances data security by isolating sensitive data in appropriate zones with restricted access while allowing broader access to less sensitive data. The complexity is managed through standardized zone definitions and policy-driven access control mechanisms.
4Loss of information
If comprehensive data classification is performed, then data can be located efficiently, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary classification by automatically assigning incoming data to appropriate zones based on predefined criteria and policies at the point of ingestion. This preliminary action establishes the initial classification early in the data lifecycle, enabling efficient subsequent retrieval and reducing the need for repeated comprehensive classification operations, thus minimizing processing time while maintaining data locability.
Data Source
AI summary
Systems and methods are disclosed for moving data based on data governance policies, wherein a plurality of datasets from a plurality of sources are received and stored into a data catalog. Predefined zones are generated, each having predefined policies. At least one common characteristic is determined for a first dataset and a second dataset. The system receives a request from an authorized user to move the first and second datasets into a particular zone and moves the datasets accordingly. The system then displays, via a graphical user interface, a representation depicting the first and second datasets. The predefined zones include a transient zone, a raw zone, a trusted zone, and a refined zone. The plurality of datasets move through the zones through a data pipeline that performs a data quality check to ensure that the data is moved through the zones according to the predefined polices.