Schema Annotation via Precomputed Entity Similarity Scoring

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data analytics solutions fail to effectively link and annotate data elements across heterogeneous sources, leading to inefficiencies in data retrieval and analysis due to imprecise and labor-intensive methods, which are not scalable or context-dependent.

Innovation Solution

A method and system for determining schema element types by pooling potential annotations from multiple sources, scoring their relevancy using instance information, and annotating elements based on structural properties to provide dynamic and context-dependent linking, incorporating user profiles and external metadata for precise and efficient data correlation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If batch offline entity matching is performed to link heterogeneous data sources, then all possible links between entities are generated, but substantial computation time, storage space, and manual verification effort are required

Engineering Contradiction:

Improvecompleteness of entity linkingVSAvoidcomputation time and verification effort

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system pre-computes and stores similarity scores between data instances from different sources before actual querying. This preliminary action creates a ready-to-use similarity index that eliminates the need for real-time batch matching, significantly reducing computation time during operational queries while maintaining comprehensive entity linking capability

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The entity matching process is divided into independent similarity computations for individual data instance pairs. Instead of processing all entities simultaneously in a monolithic batch operation, the system segments the task into discrete similarity calculations that can be pre-computed, stored, and quickly retrieved during querying, reducing both computation time and storage requirements

Inventive Principle:

Principle #1Segmentation

2Speed

If keyword search is used to find information across repositories, then search results are returned quickly, but the search fails to provide precise connections between entities and returns irrelevant results

Engineering Contradiction:

Improvesearch response timeVSAvoidprecision of entity connection

Core Design Contradiction:

SpeedVSMeasurement precision

Solution Approach 1:

The system introduces similarity scores as an intermediary mechanism between keyword search and entity connection. Instead of directly linking keywords to entities, the system uses pre-computed similarity scores to mediate the connection, providing both fast retrieval and precise entity matching by ranking results based on actual similarity rather than just keyword occurrence

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system changes the search parameter from simple keyword matching to similarity-based ranking. By transforming the search criterion from exact keyword matches to configurable similarity thresholds, the system maintains fast search response times while dramatically improving the precision of entity connections returned

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If manual verification and cleaning of entity links is performed, then link precision is improved, but substantial effort and time are required

Engineering Contradiction:

Improveprecision of entity linksVSAvoidthroughput of data processing

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system implements self-service entity linking by automatically computing similarity scores and ranking potential matches without requiring manual verification. The automated similarity-based ranking system serves itself by providing sufficiently precise links directly, eliminating the need for human intervention while maintaining high precision and enabling scalable processing of large data volumes

Inventive Principle:

Principle #25Self-service

4Loss of information

If comprehensive entity linking is performed across all heterogeneous sources, then data context is improved, but computational burden and storage requirements increase substantially

Engineering Contradiction:

Improvedata context and connectionsVSAvoidstorage space for links

Core Design Contradiction:

Loss of informationVSQuantity of substance

Solution Approach 1:

The system extracts only the essential similarity information needed for entity linking and stores it in a compact format. By taking out only the relevant similarity scores and metadata rather than storing complete link graphs for all possible entity pairs, the system maintains comprehensive data context while significantly reducing storage requirements

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS9959326B2Annotating schema elements based on associating data instances with knowledge base entities

Publication Date: 2018.05.01 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Methods and systems for determining schema element types are shown that include pooling potential annotations for an element of an unlabeled schema from a plurality of heterogeneous sources, scoring the pool of potential annotations according to relevancy using information using instance information from the plurality of heterogeneous sources to produce a relevancy score, and annotating the element of the unlabeled schema using the most relevant potential annotations.