Centralized Data Asset Registry for Discovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face difficulties in discovering and utilizing data assets due to varying schemas, structures, volumes, and locations, with no centralized system for searching and tracking changes to these assets, leading to inefficiencies in data management and integration.
Innovation Solution
A system that identifies data assets from disparate sources, generates metadata including schema, location, and descriptive information, and provides a centralized search interface to index and graph dependencies, allowing users to explore and track changes to relevant data assets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data assets are stored at different locations with different schemas and structures, then data diversity and versatility are improved, but difficulty in discovering and integrating data assets increases
Solution Approach 1:
The patent introduces a centralized data asset registry as an intermediary system that catalogs metadata about data assets from disparate locations. This registry acts as a mediator between users and the distributed data assets, providing a unified search interface and metadata standardization layer that simplifies discovery and integration without requiring changes to the underlying diverse data storage systems.
Solution Approach 2:
The patent segments data asset information into standardized metadata fields (schema, structure, volume, frequency, location) that can be independently captured and indexed. This segmentation allows the system to handle diverse data assets by breaking them down into common descriptive attributes, making them comparable and searchable through the centralized registry.
2Ease of operation
If a centralized search system is implemented to locate data assets, then ease of discovery is improved, but system complexity increases
Solution Approach 1:
The centralized data asset registry serves multiple functions: it catalogs data assets, stores metadata, provides search capabilities, tracks dependencies, and monitors changes. By consolidating these functions into a single system, the patent avoids the complexity of multiple separate tools and provides comprehensive data asset management through one unified interface.
3Reliability
If dependency tracking is implemented to monitor changes in data assets, then reliability of dependent systems is improved, but complexity of metadata management increases
Solution Approach 1:
The patent establishes dependency relationships between data assets in advance by capturing metadata that describes how assets relate to each other. This preliminary documentation of dependencies allows the system to proactively track changes and notify affected systems before issues arise, improving reliability without requiring complex real-time monitoring of every data interaction.
Data Source
AI summary
Data assets, such as streams, databases, spreadsheets, or other data sources or types, are identified and representations of the data asset are stored. The representation of a data asset includes a schema used by the data asset, a location of the data asset, and keywords or other descriptive information. The representations of each data asset are indexed, and a search interface is provided that allows users to search for relevant data assets. In addition, dependencies, or other relationship information, among the various data assets is maintained and is used to generate a graph that shows the interrelatedness of the data assets. The graph can be explored by users to select data assets, and used to alert users when a change has been made to a data asset that may affect a data asset that they have used.