Synthesizing Mapping Tables from Table Corpus
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods lack efficient repositories for synthesizing mapping relationships from large table corpora, facing challenges in completeness, synonymous mentions, and spurious mappings, which hinder applications like auto-correction and auto-join in data cleaning and integration.
Innovation Solution
A system and method for synthesizing mapping tables by extracting two-column tables from a corpus, pruning non-functional pairs, and resolving conflicts to determine coherent mapping relationships, using algorithms for statistical compatibility and functional dependencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If mapping relationships are synthesized from large table corpora using traditional methods, then completeness of mapping coverage is improved, but time consumption and processing difficulty increase significantly
Solution Approach 1:
The patent segments the large table corpus into smaller manageable units by extracting only two-column tables that contain potential mapping relationships. This segmentation allows the system to process data in discrete, focused units rather than attempting to analyze the entire corpus at once, thereby reducing processing time while maintaining mapping completeness.
Solution Approach 2:
The system extracts relevant two-column tables from the larger table corpus, isolating only those tables that contain potential mapping relationships. This extraction process removes irrelevant data and focuses computational resources on the specific subsets of data that contribute to mapping synthesis, efficiency improvement without sacrificing coverage.
Solution Approach 3:
The patent performs preliminary filtering and validation of column pairs to identify functional dependencies before full mapping synthesis. By pre-processing the data to identify candidate mapping pairs and validate their functional relationships in advance, the system reduces the complexity of subsequent processing steps and accelerates overall synthesis time.
2Quantity of substance
If traditional methods are used to synthesize mapping relationships, then comprehensive coverage is achieved, but difficulty in detecting and measuring functional dependencies increases
Solution Approach 1:
The patent replaces manual or traditional analytical methods for detecting functional dependencies with automated computational algorithms. The system uses programmatic approaches to systematically evaluate column pairs, test for functional relationships, and validate mapping candidates, thereby reducing the difficulty of detection while maintaining comprehensive coverage.
Solution Approach 2:
The system implements feedback mechanisms where extracted column pairs are validated against functional dependency criteria, and results are used to refine subsequent extraction and validation processes. This iterative feedback loop improves the accuracy of functional dependency detection while maintaining systematic coverage of the table corpus.
3Quantity of substance
If mapping tables are generated from corpus without pruning, then completeness is maintained, but reliability decreases due to non-functional and spurious mappings
Solution Approach 1:
The system extracts and identifies non-functional column pairs and spurious mappings, then removes them from the candidate set before final mapping table generation. This selective removal of unreliable mappings preserves the completeness of valid mappings while improving overall reliability by eliminating false positives.
Solution Approach 2:
The patent converts the presence of non-functional and spurious mappings into a benefit by using them as training data or validation cases. The system identifies these erroneous mappings to refine its detection algorithms and improve the accuracy of functional dependency recognition, thereby transforming potential harm into improved reliability.
4Productivity
If automated synthesis methods are implemented, then productivity is improved, but device complexity increases
Solution Approach 1:
The patent implements a universal framework that handles multiple types of mapping relationships (one-to-one, one-to-many, many-to-one) using the same core extraction and validation mechanisms. This multi-functional approach improves productivity across different mapping scenarios while avoiding the need for separate complex systems for each mapping type.
Solution Approach 2:
The system performs self-validation and self-correction during the mapping synthesis process, automatically identifying and resolving conflicts without requiring external intervention. This self-service capability reduces operational complexity while maintaining high productivity through automated conflict resolution and validation.
Data Source
AI summary
Methods and systems for synthesizing mapping tables using table corpus is provided. A functional dependency between at least two items of an input table is determined. A plurality of two-column tables are extracted from the table corpus. The extracted plurality of two-column tables are synthesized to determine at least one mapping table having a first column having the functional dependency with a second column. A next item of the input table is provided from the determined at least one mapping table.