Structured Data Search Table Understanding Module
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Structured data search systems struggle to index tables lacking explicitly defined subject columns and column headers, making it difficult for users to find relevant structured data on the internet, as they are unable to understand and associate tables with semantic concepts.
Innovation Solution
The method involves identifying subject columns and detecting column headers using other tables and a knowledge base, which enables the system to annotate and index tables without explicit definitions, facilitating better search results by leveraging co-occurrence frequencies and knowledge base attributes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the SDSS uses traditional indexing methods that require explicit subject columns and column headers, then indexing efficiency is improved, but the system becomes incapable of indexing tables lacking explicit definitions
Solution Approach 1:
The patent introduces an intermediary component (table understanding module) that mediates between the raw table data and the indexing system. This module detects subject columns and column headers even when not explicitly defined, using heuristics and contextual analysis. The intermediary translates undecorated tables into a format suitable for traditional indexing methods, thereby resolving the contradiction between maintaining efficient indexing and handling tables without explicit definitions.
2Device complexity
If the SDSS indexes only tables with explicit subject columns and column headers, then the system complexity is reduced, but the ability to return relevant tables in search results deteriorates
Solution Approach 1:
The patent applies preliminary action by performing subject column and column header detection during the table ingestion phase, before the indexing process. The table understanding module pre-processes tables to identify and label subject columns and headers using contextual analysis and heuristics. This preliminary structuring allows subsequent indexing operations to proceed with standard methods, maintaining system simplicity while improving the reliability of search results by ensuring more tables can be properly indexed and retrieved.
3Quantity of substance
If the SDSS implements advanced table understanding to detect subject columns and headers in tables without explicit definitions, then the coverage of indexable tables is improved, but the processing time and computational resources increase
Solution Approach 1:
The patent implements local quality by applying sophisticated table understanding methods selectively rather than uniformly to all tables. The system identifies tables that lack explicit subject columns or headers and applies detection algorithms only to those cases. For tables with clear explicit definitions, the system uses faster, simpler processing paths. This localized application of advanced processing improves overall coverage of indexable tables while minimizing the impact on processing time for the majority of tables that don't require complex analysis.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and computer program products for understanding tables for search. Identifying a subject column for a table, detecting a column header using other tables, and detecting a column header using a knowledge base. Methods can be utilized in a structured data search system (SDSS) that indexes structured information, such as, tables in a relational database or html tables extracted from web pages. The SDSS allows users to search over the structured information (tables) using different mechanisms including keyword search and data finding data.