Entity Matching With Topic-Aware Tokenization for Heterogeneous Records
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing natural language processing systems are limited to identical matches and require custom frameworks for heterogeneous record matching, which are not adaptable to changes in data format or sources, necessitating time-consuming and costly adjustments.
Innovation Solution
A generalized entity matching system that preprocesses data pairs into parsable structures, serializes them into token sequences with topic attributes, pools contextualized representations, and predicts matching labels using machine learning models like BERT, allowing for adaptable and efficient matching across different data sources and formats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If custom matching frameworks are used for heterogeneous record matching, then matching capability is improved, but adaptability to changes in data format or sources deteriorates
Solution Approach 1:
The patent applies universality by designing a matching framework that can handle both structured and unstructured data formats through a unified processing pipeline. The system uses generic components including a tokenizer that handles multiple data types, an encoder that processes diverse input formats, and a pooling mechanism that aggregates results consistently across different data sources, eliminating the need for custom frameworks for each data type.
Solution Approach 2:
The patent implements parameter changes by dynamically adjusting processing parameters based on the input data characteristics. The system automatically detects whether input data is structured or unstructured and modifies its processing approach accordingly, using learnable parameters in the neural network model that adapt to different data formats and sources without requiring manual reconfiguration.
2Adaptability or versatility
If custom matching frameworks are adjusted regularly for changes in data format or sources, then adaptability is improved, but time and cost increase
Solution Approach 1:
The patent applies self-service through automated model training and adaptation mechanisms. When new data sources or formats are introduced, the system automatically retrains its neural network model using the new data, adjusting its parameters and learning patterns without requiring manual intervention. This self-adaptation capability eliminates the need for regular adjustments by data engineers or specialists.
Solution Approach 2:
The patent implements preliminary action by pre-processing data into standardized representations before matching. The system converts both structured and unstructured data into a common format using the tokenizer and encoder, performing necessary transformations in advance. This preliminary standardization reduces the effort required for future adaptations when new data sources are added.
3Device complexity
If identical matches or equality checks are used, then simplicity is improved, but matching accuracy for heterogeneous records deteriorates
Solution Approach 1:
The patent introduces intermediaries in the form of learned embeddings and contextual representations that mediate between different data formats and the matching decision. Instead of directly comparing raw structured and unstructured data, the system uses neural network encoders to transform both into a common vector space where semantic similarity can be effectively measured, serving as an intermediary representation that preserves information from both data types.
Data Source
AI summary
Disclosed embodiments relate to generalized entity matching. Techniques can include receiving a data pair of two entities that may be pre-processed to have parsable data structures, and serializing the data pair into a sequence of tokens based on data structure of each entity in the data pair. Techniques can further include encoding the serialized data pair to include topic attributes that may be mapped to data in the data pair and the topic of the mapped data matches the topic represented by topic attribute and the data in the data pair is concatenated. Techniques can further include pooling attributes in the data pair based on contextualized attributed representations of each encoded entity in the data pair and schema of each entity of the data pairs, where the contextual attribute representations are based on a first token of each encoded attribute in the sequence of tokens, and predicting matching labels between the data pairs based on pooled attributes.