Automated Label Linking via LSH Forest and ML

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Manual linking of technical metadata to domain definitions in cloud computing environments is a time-consuming and labor-intensive process, especially when dealing with large volumes of data, making it difficult for data producers to find and manage trusted data effectively.

Innovation Solution

A method using a machine-learning model and Locality Sensitive Hashing (LSH) forest to automatically recommend similar labels by generating feature sets for pairs of technical and domain terms, allowing for automated association and improved search capabilities within a cloud-computing data catalog.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If manual linking of domain definitions to technical metadata is performed, then data governance and data discovery are improved, but time consumption and labor intensity increase significantly

Engineering Contradiction:
Improvedata governanceVSAvoidtime consumption
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent replaces the manual mechanical process of linking domain definitions to technical metadata with an automated machine learning system. The system uses trained models to automatically perform the linking that previously required human data stewards to manually search and associate terms, thereby eliminating the time-consuming manual effort while maintaining or improving data governance quality.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system enables self-service automation where the data catalog automatically performs the linking operation without requiring human intervention. The machine learning models autonomously analyze technical metadata, generate candidate domain definitions, and establish associations, allowing the system to serve itself rather than relying on continuous manual input from data practitioners.

Inventive Principle:
Principle #25Self-service

2Ease of operation

If manual search and linking is performed to improve data discovery, then data understanding is enhanced, but scalability deteriorates when metadata volume grows

Engineering Contradiction:
Improvedata discoveryVSAvoidscalability
Core Design Contradiction:
Ease of operationVSProductivity

Solution Approach 1:

The patent substitutes manual search and linking operations with automated machine learning-based systems. Instead of data practitioners manually searching through growing metadata volumes, the system automatically discovers relationships between technical metadata and domain definitions using trained models, maintaining ease of data discovery regardless of metadata scale.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system changes the operational parameters from manual human search behavior to automated algorithmic processing. By transforming the data discovery process from human-cognitive operations to machine-executed algorithms, the system achieves scalability that manual processes cannot match, as computational operations can handle large volumes efficiently without proportional increases in time or effort.

Inventive Principle:
Principle #35Parameter changes

3Productivity

If automated machine learning linking is implemented, then time efficiency and scalability are improved, but system complexity increases

Engineering Contradiction:
Improvetime efficiencyVSAvoidsystem complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by training machine learning models in advance on curated datasets before deployment. The models are pre-trained to understand the relationships between domain definitions and technical metadata, so that during actual operation, the system can quickly perform linking without requiring complex real-time processing. This upfront preparation reduces the operational complexity of the automated system.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system introduces trained machine learning models as intermediaries between the raw metadata and the linking operation. Rather than implementing complex rule-based systems or requiring direct human intervention, the patent uses pre-trained models as intermediary components that automatically perform the matching, simplifying the overall system architecture while maintaining high productivity.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS11797549B2Techniques for linking data to provide improved searching capabilities
Publication Date: 2023.10.24 ORACLE INT CORP
  • US11797549B2 patent drawing
  • US11797549B2 patent drawing
  • US11797549B2 patent drawing

AI summary

A machine-learning model may be previously trained with a supervised learning algorithm to identify whether a pair of labels provided as input are similar. A locality sensitive hashing forest (LSH) may be generated for the set of candidate labels. When a user later identifies an input label (e.g., by search query, by interface selection, etc.) the input label may be used to query the LSH forest to identify a subset of the candidate labels. This subset may be used to generate respective pairs comprising the input label, one of the subset candidate labels, and a corresponding feature set generated for the pair. This data may be provided to the model to identify a degree to which the pair of labels are similar. The user may be provided one or more recommendations including similar terms identified from the model's output.