Role-Driven Metadata Layers in Data Mesh
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data mesh frameworks, it is challenging to support the different views of a dataset needed by various types of users, and the current approach requires copying data across domains, increasing resource utilization.
Innovation Solution
A multi-plane metadata approach is introduced to enable visibility into data stored in a data mesh for different user roles, simplifying the exposure of datasets to users without requiring data copying. This involves generating user role-specific metadata layers based on metadata regarding a dataset with multiple data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If data is copied across domains to support different user views, then user-specific data access is improved, but resource utilization deteriorates
Solution Approach 1:
The patent segments metadata into different layers (first metadata layer for general dataset information, second metadata layer for user role-specific information) that can be independently generated and served to different users without duplicating the underlying data, thereby providing user-specific access while avoiding resource waste from data copying
Solution Approach 2:
The patent introduces metadata layers as an intermediary between the underlying data and users. These metadata layers provide user-specific views and access control information without requiring actual data copying, serving as a mediator that enables efficient user-specific data access while preserving resource utilization
2Device complexity
If centralized data administration is used, then data management is simplified, but adaptability to different user needs deteriorates
Solution Approach 1:
The patent segments the metadata into multiple layers where the first layer contains general dataset metadata and the second layer contains user role-specific metadata. This segmentation allows centralized management of the base metadata while enabling flexible, role-specific adaptations without increasing overall system complexity
Solution Approach 2:
The patent adds a dimensional layer to metadata organization by creating hierarchical metadata layers. This additional dimension allows the system to maintain simple centralized management at the base level while providing adaptability to different user needs at higher metadata layers, effectively resolving the contradiction between simplicity and versatility
Data Source
AI summary
In one embodiment, a device of a data mesh generates a first metadata layer for the data mesh that comprises metadata regarding a dataset having a plurality of data sources. The device identifies user role-specific metadata associated with a particular user role and based at least in part on the metadata regarding the dataset. The device generates a second metadata layer for the data mesh that comprises the user role-specific metadata associated with the particular user role. The device provides the user role-specific metadata for presentation to a user associated with the particular user role.