Coupling Table Search Using Precomputed Similarity Indexes

Solution Overview

Problem

Existing methods for finding a couplable table from an external table group for a given base table are inefficient due to the high calculation cost, especially when dealing with a large number of records and varying notation rules or schemas across tables.

Innovation Solution

A coupling table specification system that constructs a coupling index for each similarity function, allowing for efficient search of couplable records by prioritizing indices with the highest estimated number of results, thereby reducing the search cost.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If similarity functions are used to calculate similarities of all records in columns of all tables in the external table group for each record of each column of the base table, then couplable tables can be specified, but the calculation amount becomes enormous when the number of records is large

Engineering Contradiction:
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The patent applies preliminary action by pre-calculating and storing similarity values between records in external tables before the actual coupling search. The system computes similarity matrices in advance and stores them in storage units, so that during the coupling search, pre-computed similarity values are retrieved rather than recalculated. This resolves the contradiction by preparing computational work beforehand, maintaining measurement precision while dramatically improving productivity during actual table coupling operations.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent segments the large-scale similarity calculation problem into smaller manageable units by processing tables and columns individually. The system divides the external table group into separate tables, and further divides each table into columns, processing similarity calculations column-by-column rather than all-at-once. This segmentation allows the system to handle large datasets efficiently by breaking down the enormous calculation task into smaller, parallelizable units, thus improving productivity while maintaining accuracy.

Inventive Principle:
Principle #1Segmentation

2Reliability

If multiple similarity functions are used to calculate similarities, then more accurate coupling determination can be made, but the calculation amount increases to O(NMF) where F is the number of similarity functions

Engineering Contradiction:
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The patent applies preliminary action by pre-calculating similarity values for multiple similarity functions and storing them in advance. Instead of computing multiple similarity functions during the actual coupling search, the system computes and stores similarity matrices for each similarity function beforehand. During search operations, these pre-computed values are retrieved and combined, maintaining reliable coupling determination while avoiding the O(NMF) calculation cost during actual use.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent uses copying by creating and storing similarity matrices as separate data structures that can be reused multiple times. The system generates similarity matrices for each similarity function and stores them in storage units, creating reusable copies of similarity data. This allows multiple similarity functions to be applied efficiently by referencing stored similarity copies rather than recalculating, thus maintaining reliability while reducing productivity loss.

Inventive Principle:
Principle #26Copying

3Adaptability or versatility

If all records in external tables are searched for coupling, then comprehensive coupling tables can be found, but the search cost becomes prohibitively high for large datasets

Engineering Contradiction:
Core Design Contradiction:
Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-computing and storing similarity matrices that identify potential couplable record pairs before the actual coupling search. The system calculates similarity values for all records in advance and stores them, creating a pre-processed index of potential couplings. During the actual search, the system queries these pre-computed results rather than performing full searches, ensuring comprehensive coupling detection while dramatically reducing search time.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent introduces similarity matrices as intermediary data structures that mediate between the raw external table data and the final coupling results. These similarity matrices serve as intermediate representations that capture coupling potential without requiring full re-search of the original data. The intermediary similarity matrices enable comprehensive coupling search by providing a pre-processed view of potential matches, reducing the time needed for actual coupling determination.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS12222916B2Coupling table specification system, coupling table search device, method, and program
Publication Date: 2025.02.11
  • US12222916B2 patent drawing
  • US12222916B2 patent drawing
  • US12222916B2 patent drawing

AI summary

A coupling table search device 530 includes: input means 531 that receives an input of a coupling index from a coupling index storage unit 520; search plan generation means 532 that calculates, for each coupling index, the number of results that is the number of couplable record pairs estimated in a case where a record in a target column is found using the coupling index for each similarity function, and generates a search plan that defines a use order of the coupling index in descending order of the calculated number of results; and record search means 533 that searches for a record in the target column by using the coupling index in the order defined in the search plan, and specifies the external table in which records whose number is equal to or larger than a predetermined threshold have been found as a coupling table.