Metadata Enrichment Scoring for Adaptive Label Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for metadata enrichment of data assets struggle to assess the quality and reliability of the association between tagged assets and tags, leading to potential high costs due to misguided decisions based on unreliable output.
Innovation Solution
A method for metadata enrichment that determines informativeness scores for input data assets, adaptively executing or skipping enrichment steps based on these scores, and combining labels to optimize metadata enrichment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all enrichment steps are executed for metadata enrichment, then completeness of metadata is improved, but processing time and computational resources increase
Solution Approach 1:
The enrichment process dynamically adapts by computing informativeness scores for metadata values and using these scores to selectively execute enrichment steps. The system transitions from a static all-or-nothing approach to a dynamic conditional execution model where steps are adapted or skipped based on real-time quality assessment
Solution Approach 2:
The system changes the parameter of enrichment step execution from binary (execute or not) to conditional (execute, adapt, or skip) based on computed informativeness scores. This parameter change enables selective execution of enrichment steps based on the quality characteristics of input metadata
2Productivity
If enrichment steps are executed without quality assessment, then processing speed is improved, but output reliability deteriorates
Solution Approach 1:
The system performs preliminary assessment by computing informativeness scores for metadata values before executing enrichment steps. This preliminary quality assessment enables informed decisions about which steps to execute, adapt, or skip, ensuring reliable output without unnecessary processing
Solution Approach 2:
The system uses computed informativeness scores as feedback to control the execution of enrichment steps. The feedback mechanism allows the system to adapt its processing behavior based on the quality characteristics of input metadata, balancing speed and reliability
3Productivity
If enrichment steps are selectively skipped based on informativeness scores, then processing efficiency is improved, but metadata completeness may deteriorate
Solution Approach 1:
The system applies local quality assessment by computing informativeness scores for specific metadata values and applying different execution strategies (execute, adapt, or skip) to different enrichment steps based on local characteristics. This localized approach ensures that only necessary steps are executed while maintaining overall metadata quality
4Productivity
If adaptive enrichment is implemented with informativeness scores, then resource utilization is improved, but system complexity increases
Solution Approach 1:
The system performs self-assessment by computing informativeness scores for its own metadata values and using this self-knowledge to control its enrichment process. This self-service capability enables automatic adaptation without external intervention, improving resource utilization while managing complexity through autonomous decision-making
Data Source
AI summary
The present disclosure relates to a method of metadata enrichment using an enrichment comprising multiple steps. The method comprises: determining for an input data asset a metadata value descriptive of the input data asset. Characteristics of the metadata value of the input data asset may be determined. At least one informativeness score of the metadata value of the input data asset may be computed using the determined characteristics. An execution of the enrichment step may be skipped in case an input characteristic of the enrichment step is not part of the determined characteristics. In case the input characteristic of the enrichment step is part of the determined characteristics, the enrichment step may be adapted and executed or the enrichment step may be executed without adaptation. Labels resulting from the executed enrichment steps may be combined for providing one or more labels of the data asset.