Grouping Identity Records for Entity Resolution Candidate Lists

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current identity resolution systems face performance issues due to the need for constant database querying for each input record, leading to inefficiencies in generating candidate lists and resolving entity relationships.

Innovation Solution

Grouping identity records allows for a composite query to be generated and applied to the database, followed by individual queries on cached results to determine candidate lists, improving performance by reducing the number of database queries and enabling batch processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If the database is queried for each input record to generate candidate lists, then the candidate list can be generated for each record, but the system performance deteriorates due to constant querying

Engineering Contradiction:

Improvecandidate list generation efficiencyVSAvoiddatabase query time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by grouping identity records and pre-computing candidate lists before entity resolution. By generating candidate lists in advance for grouped records rather than querying the database for each individual record during resolution, the system reduces real-time database query overhead and improves overall processing efficiency.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system merges multiple database queries into a single composite query by grouping identity records with similar attributes. Instead of querying the database separately for each input record, the system combines multiple records into groups and executes one composite query that returns candidate lists for all records in the group, significantly reducing the number of database interactions.

Inventive Principle:

Principle #5Merging (Combining)

2Measurement precision

If individual queries are performed for each identity record, then accurate candidate lists can be generated, but the processing time increases

Engineering Contradiction:

Improvecandidate list accuracyVSAvoidprocessing time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary grouping of identity records and pre-computation of candidate lists before the actual entity resolution process. By preparing candidate lists in advance for grouped records, the system maintains accurate matching while reducing the time spent during resolution operations.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system segments the entity resolution process into distinct phases: grouping identity records, generating candidate lists through composite queries, and then performing resolution. This segmentation allows the system to batch-process records efficiently while maintaining the precision of individual record resolution through subsequent individual queries against pre-fetched data.

Inventive Principle:

Principle #1Segmentation

3Adaptability or versatility

If the database is queried N times for N input records, then each record can be processed individually, but the system complexity and query overhead increase

Engineering Contradiction:

Improveindividual record processing capabilityVSAvoidquery processing complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system merges multiple individual database queries into a single composite query by grouping identity records. This consolidation reduces query processing complexity from N separate queries to one unified query that handles multiple records simultaneously, while the system maintains the ability to process individual records through subsequent individual queries against the pre-fetched composite results.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system performs preliminary grouping and candidate list generation before individual record processing. By preparing the data structure in advance, the system reduces the complexity of subsequent processing steps and enables efficient individual record resolution without requiring complex real-time querying for each record.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS8510323B2Grouping identity records to generate candidate lists to use in an entity and relationship resolution process

Publication Date: 2013.08.13 AIRBNB INC

AI summary

Provided is a method for grouping identity records to generate candidate lists to use in an entity and relationship resolution process. A plurality of identity records provide attributes of entities. The received identity records are grouped into a group of identity records. A composite query on values for selected attributes of the identity records in the group is generated and applied to an entity database to obtain composite results of entity records in the entity database matching the attribute values of the composite query. For the identity records in the group, an individual query on attributes of one of the identity records is performed against the composite results of the entity records to determine a candidate list of entity records from the entity database for the identity record.