Column Meaning Inference Using Cross-Column Similarity Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for inferring the meaning of columns in a table have low accuracy as they only utilize data from the column itself, leading to incorrect interpretations, such as inferring the meaning of a column as 'Heisei' when it actually represents 'age'.
Innovation Solution
A meaning inference system that selects candidates for column meanings, computes similarity scores with other columns and the table, and identifies the correct meaning based on these scores, using a concept dictionary to connect related meanings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If only data from the column itself is used for meaning inference, then the inference process is simple and fast, but the accuracy of meaning inference is low
Solution Approach 1:
The patent combines multiple data sources (column data, other columns, and table-level information) into a unified inference process. The scoring mechanism integrates statistics from the target column with similarity calculations against other columns and table meanings, creating a comprehensive evaluation that improves accuracy while managing complexity through structured computation.
Solution Approach 2:
The patent extends the inference process from one-dimensional (column-only) to multi-dimensional analysis by incorporating: (1) statistical features from the target column, (2) similarity scores with other columns, and (3) relationship scores with table meanings. This dimensional expansion enables more accurate meaning identification by considering multiple aspects simultaneously.
2Measurement precision
If statistical values and similarity scores are computed for multiple candidates, then the accuracy of meaning inference is improved, but the computational time and resources increase
Solution Approach 1:
The patent performs preliminary computations by pre-calculating statistical values for candidate meanings and pre-computing similarity scores between columns and table meanings. These pre-computed values are stored and reused during the actual inference process, significantly reducing the time required for meaning identification while maintaining high accuracy through comprehensive candidate evaluation.
Data Source
AI summary
A column meaning candidate selection means 303 selects a candidate for meaning of a column whose meaning is to be inferred. A column similarity computation means 304 computes, for each candidate for meaning selected by the column meaning candidate selection means 303, a score indicating a similarity between the selected candidate for meaning and meaning of each column other than the column whose meaning is to be inferred contained in a table. A column meaning identification means 305 identifies meaning of the column whose meaning is to be inferred from the candidates for meaning of the column with use of the score computed by the column similarity computation means 304.