Concept Expansion Using Web Tables for Tail Entities
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing knowledge bases focus on popular concepts, leaving less popular concepts, known as tail concepts, with limited entity coverage due to insufficient mentions in text, and existing techniques struggle to effectively apply to tail concepts as they often overlap with other concepts.
Innovation Solution
The system uses concept expansion frameworks that leverage web tables to expand entity coverage for tail concepts by modeling table-entity relationships, utilizing rich content signals from captions, headings, and anchor text, and employing probabilistic models to rank entities and tables, thereby reducing semantic drift without a reference ontology.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If existing techniques are used for concept expansion, then popular concepts can be covered, but tail concepts suffer from limited entity coverage due to insufficient mentions in text
Solution Approach 1:
The patent uses web tables as an intermediary data source to bridge the gap for tail concepts. Tables serve as a mediator that connects concepts with entities even when text mentions are insufficient, allowing entity discovery through table structures and relationships rather than relying solely on text frequency
Solution Approach 2:
The patent transitions from one-dimensional text-based concept expansion to multi-dimensional expansion by incorporating web tables with their structured rows and columns. This adds a new dimension (table structure) for discovering entities associated with concepts, particularly benefiting tail concepts that lack sufficient text mentions
2Quantity of substance
If traditional label propagation methods are used, then concept coverage can be expanded, but semantic drift occurs without proper table-entity relationship modeling
Solution Approach 1:
The patent applies local quality by treating different table-entity relationships with different weights and confidence levels. Not all tables or entities are treated equally - the system evaluates the quality and relevance of each table-entity connection locally, allowing more reliable relationships to have greater influence on concept expansion while reducing semantic drift
3Device complexity
If concept expansion is performed without leveraging web tables, then processing is simpler, but recall for tail concepts remains low
Solution Approach 1:
The patent performs preliminary action by pre-processing and indexing web table data before concept expansion. Tables are parsed, entities are extracted, and relationships are established in advance, creating a ready-to-use knowledge structure that can be efficiently queried during concept expansion without adding significant complexity to the main processing pipeline
Data Source
AI summary
Concept expansion using tables, such as web tables, can return entities belonging to a concept based on an input of the concept and at least one seed entity that belongs to the concept. A concept expansion frontend can receive the concept and seed entity and provide them to a concept expansion framework. The concept expansion framework can expand the coverage of entities for concepts, including tail concepts, using tables by leveraging rich content signals corresponding to concept names. Such content signals can include content matching the concept that appear in captions, early headings, page titles, surrounding text, anchor text, and queries for which the page has been clicked. The concept expansion framework can use the structured entities in tables to infer exclusive tables. Such inference differs from previous label propagation methods and involves modeling a table-entity relationship. The table-entity relationship reduces semantic drift without using a reference ontology.