Semantic Hashing for Entity Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional entity resolution methods are computationally demanding and time-consuming, especially at large scales, due to the need to process thousands of mentions and entities across multiple documents, making them inefficient for determining coreference in natural language text.
Innovation Solution
The method generates semantic hashes for entities with assigned IDs, changes IDs at entity milestones such as merges, splits, or removals, and maintains history information, allowing for efficient similarity searches through a search structure that enables nearest neighbor searches across these hashes, incorporating high confidence information like biographical, distribution, co-location, and temporal data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional entity resolution methods are used to process thousands of mentions and entities across multiple documents, then comprehensive entity resolution can be achieved, but computational complexity and processing time increase significantly
Solution Approach 1:
The patent creates a simplified copy of entity information in the form of semantic hashes. Instead of storing and processing complete entity records with all their attributes and relationships, the system generates compact hash representations that capture essential semantic characteristics. These hashes enable fast similarity comparisons without requiring full entity record processing, thus maintaining resolution accuracy while dramatically reducing processing time.
Solution Approach 2:
The patent transforms entity representations from detailed structured data to compact hash values. By changing the representation parameter from full entity records to semantic hashes, the system enables efficient similarity search through hash comparison rather than comprehensive attribute matching. This parameter transformation reduces computational complexity while preserving the ability to identify coreferent entities.
2Reliability
If comprehensive history information for all entities is maintained to track changes over time, then complete entity tracking is achieved, but storage requirements increase significantly
Solution Approach 1:
The patent maintains only essential entity information as semantic hashes rather than complete historical records. The hash representation captures the core semantic identity of entities at different time points, allowing the system to track entity changes and merges without storing full historical context. This selective retention of critical information maintains tracking accuracy while minimizing storage requirements.
Solution Approach 2:
The patent extracts only the most important entity attributes needed for identification and tracking, discarding redundant historical details. By taking out only the essential semantic characteristics and storing them as hashes, the system achieves complete entity tracking for coreferent resolution while eliminating unnecessary storage overhead from full historical data retention.
3Loss of information
If detailed entity attributes and relationships are stored for all entities, then complete entity information is available, but system complexity increases
Solution Approach 1:
The patent creates simplified hash copies of entity information that preserve essential semantic meaning without retaining complex structural details. These hash representations maintain the core information needed for entity resolution and coreferent detection while eliminating the complexity of managing full entity attribute sets and relationship graphs.
Solution Approach 2:
The patent extracts only the essential semantic characteristics from detailed entity attributes, separating the critical identification information from redundant details. This extraction process maintains information completeness for coreference resolution while significantly reducing system complexity by working with simplified hash representations rather than comprehensive entity data structures.
Data Source
AI summary
According to one aspect, a computer-implemented method for entity resolution is disclosed. In one embodiment, the method includes generating a semantic hash for an entity having an assigned entity identifier (ID) and, upon the occurrence of an entity milestone, changing the entity ID. The method further includes generating a semantic hash for the entity having the changed entity ID, and maintaining history information associated with the entity and corresponding entity IDs and semantic hashes over a period of time that includes a plurality of entity milestones. The method also includes periodically removing at least one set of older entities and retaining entity IDs and semantic hashes associated with the removed entities.