Entity Resolution via Token Normalization and ML Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing business systems often have disparate data representations of entities across different applications, leading to challenges in understanding comprehensive relationships, such as those between customers and organizations, due to varying information needs in financial, contractual, and transactional contexts.
Innovation Solution
A system that tokenizes records to create a normalized representation, correlates them with a machine learning system to determine existing resolutions, and uses web search results to match entities, updating records to provide a comprehensive view of entities across systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If entity resolution is performed using multiple disparate data sources, then completeness of entity information is improved, but system complexity increases
Solution Approach 1:
The system segments entity resolution into distinct components: normalization module that standardizes data formats, matching module that compares entities, and learning module that improves over time. This segmentation allows comprehensive information gathering while managing complexity through modular design.
Solution Approach 2:
The patent introduces an intermediary normalization layer that translates disparate data from multiple sources into a common format before comparison. This intermediary structure enables complete information integration without directly coupling all data sources, thereby reducing system complexity.
2Loss of information
If comprehensive entity data is collected from multiple systems, then understanding of entity relationships is improved, but data processing time increases
Solution Approach 1:
The system performs preliminary normalization of entity data from multiple sources before matching and comparison. By pre-processing and standardizing data formats in advance, the system reduces the time required for actual entity resolution while maintaining comprehensive relationship understanding.
Solution Approach 2:
The learning module enables the system to self-improve by automatically learning from resolved entity matches and refining its normalization and matching strategies over time, reducing processing time for future queries without requiring additional manual intervention.
3Measurement precision
If entity records are normalized and compared, then matching accuracy is improved, but computational overhead increases
Solution Approach 1:
The normalization module transforms entity records into standardized parameters and formats, enabling accurate comparison while reducing computational complexity through consistent data representation. This parameter standardization improves matching accuracy without proportionally increasing computational overhead.
Solution Approach 2:
The learning module receives feedback from matching results and continuously refines normalization strategies and matching thresholds, improving accuracy over time while optimizing computational resource usage based on learned patterns from previous resolutions.
Data Source
AI summary
A pair of records is tokenized to form a normalized representation of an entity represented by each record. The tokens are correlated to a machine learning system by determining whether a learned resolution already exists for the two entities. If not, the normalized records are compared to generate a comparison measure to determine whether the records match. The normalized records can also be used to perform a web search and web search results can be normalized and used as additional records for matching. When a match is found, the records are updated to indicate that they match, and the match is provided to the machine learning system to update the learned resolutions.