Web Table Stitching via Hidden Attribute Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Fragmented tables across web pages complicate data extraction and integration, leading to inconsistent and confusing data when trying to join related tables, as the context of the tables is often unknown, making it difficult to determine if they should be joined to produce a consistent table.
Innovation Solution
The method involves identifying stitchable tables based on semantic equivalence of headers and extracting hidden attributes from web pages to create a union table, using techniques such as segmentation, alignment, and iterative pairwise alignment to align segment sequences and filter meaningful attributes, which are then added to the union table to disambiguate and enrich the data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If tables are extracted from web pages, then structured data is obtained, but the tables may be fragmented across multiple pages making utilization difficult
Solution Approach 1:
The system segments web pages into discrete table units with associated hidden attributes, allowing fragmented tables to be identified and reassembled. Each table segment is extracted with its contextual metadata, enabling systematic reconstruction of complete tables across multiple pages.
Solution Approach 2:
Hidden attributes serve as intermediary elements that bridge fragmented tables across different web pages. These attributes contain contextual information that enables the system to match and join table segments from different sources into coherent unified tables.
2Quantity of substance
If related tables are joined together, then comprehensive data is created, but the data may become confusing and inconsistent without context
Solution Approach 1:
The system performs preliminary extraction of hidden attributes from web pages before joining tables. This advance preparation ensures that contextual information is available prior to the joining process, enabling the system to make informed decisions about table compatibility and maintain data consistency.
Solution Approach 2:
The system uses hidden attributes as feedback mechanisms to verify table compatibility before joining. By checking contextual information stored in hidden attributes, the system can determine whether tables from different sources are consistent and should be merged, preventing confusion and maintaining reliability.
3Loss of information
If hidden attributes are extracted and added to union tables, then data disambiguation is achieved, but the processing complexity increases
Solution Approach 1:
The system extracts hidden attributes separately from the main table data, isolating contextual information into distinct metadata fields. This extraction approach allows the system to manage and process contextual information independently, reducing the complexity of integrating it with the main data processing pipeline.
Data Source
AI summary
Systems and techniques are provided for generating a union table with from stitchable tables. Tables may be extracted from web pages to obtain extracted tables. Stitchable tables may be determined from the extracted tables. Hidden attributes for the stitchable tables may be extracted from the web pages from which the stitchable tables were extracted using segmentation of text for contextual data from the web pages into segment sequences, and alignment of the segment sequences. Iterative pairwise alignment may be used to align the segment sequences and obtain aligned segments. The stitchable tables may be joined into a union table. Hidden attributes from the aligned segments may be added to the union table. Headers for the hidden attributes in the union table may be labeled using a database of entities and class labels.