Automated Data Access Policy Generation via Lineage Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data access control systems in data lakes face complexity in setting access control policies, particularly when data is not organized hierarchically, leading to increased man-hours required for management across an organization.
Innovation Solution
A data access control system that includes a data lake with a metadata management unit and a policy administration unit, which collects and stores data lineage information and metadata to generate and apply access control policies based on the treatment content of data, reducing the need for manual policy setting through automated policy recommendation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If hierarchical access control policies are applied to data lakes, then access control management becomes structured and manageable, but the system fails when data does not follow hierarchical organization patterns
Solution Approach 1:
The patent introduces metadata and lineage information as intermediary elements that bridge the gap between hierarchical access control requirements and non-hierarchical data storage. These intermediaries enable policy application without requiring actual hierarchical data organization.
Solution Approach 2:
The patent shifts from enforcing hierarchy in data storage structure to implementing hierarchy in the policy management dimension. Access control policies operate at the metadata and lineage information level rather than requiring physical data hierarchy.
2Reliability
If manual access control policy setting is performed for each data file in a large-scale data lake, then precise access control can be achieved, but the man-hours and operational complexity increase significantly
Solution Approach 1:
The patent performs preliminary actions by collecting metadata and lineage information in advance, and pre-defining access control policies at the data lake level. This preparation enables automatic policy application to individual data files without manual intervention.
Solution Approach 2:
The system enables self-service access control where the data lake automatically applies access control policies to data files based on pre-configured metadata and lineage information, eliminating the need for manual policy setting for each file.
3Device complexity
If access control policies are defined based on organizational hierarchy, then policy structure becomes clear and manageable, but the system cannot accommodate data sharing requirements beyond default policy permissions
Solution Approach 1:
The patent introduces dynamic policy adjustment capabilities where access control policies can be modified based on lineage information and metadata, allowing flexible data sharing beyond static organizational hierarchy constraints while maintaining structured policy management.
Data Source
AI summary
A predetermined access control policy is generated with reference to a lineage table and a metadata table to be stored in a policy table, and an access control policy which should be applied or recommended to treated data is provided with reference to the policy table.