Entity Matching via Joint Blocking and Learning

The entity matching process is divided into two sequential stages: a blocking stage that filters out obviously non-matching pairs using simple heuristics, and a matching stage that applies complex learning models only to remaining candidate pairs. This segmentation reduces computational complexity from O(N1×N2) to approximately O(N1×N2×ε) where ε is the blocking effectiveness, making the system scalable to large datasets while maintaining accuracy.

The blocking module performs preliminary filtering before the matching module processes entity pairs. By pre-filtering out clearly non-matching pairs using simple blocking functions, the system reduces the workload for the more computationally intensive matching stage, thereby improving overall computational efficiency without sacrificing matching accuracy.

The system uses unlabeled data to train the blocking module, which automatically generates pseudo-labels for entity pairs. These pseudo-labels are then used to train the matching module without requiring manual annotation. The blocking module essentially serves itself by learning from unlabeled data and providing labels that enable downstream matching tasks, eliminating the need for extensive labeled datasets.

The blocking module acts as an intermediary that transforms unlabeled data into useful pseudo-labels, which then serve as training data for the matching module. This intermediary layer enables the system to leverage large amounts of unlabeled data while achieving performance comparable to approaches that require labeled data, effectively bridging the gap between data availability and model training requirements.

The system employs dynamic model selection where the blocking and matching models are trained on domain-specific unlabeled data from the target dataset itself. This allows the models to adapt to the specific characteristics of each new dataset without requiring retraining on comprehensive labeled datasets. The blocking module can be quickly retrained on new unlabeled data from emerging domains, enabling the system to adapt to new datasets while maintaining accuracy.

The system changes its training parameters and model architecture based on the characteristics of the input data. By training blocking and matching models on the actual unlabeled data from each domain, the system adjusts its parameters to fit the specific data distribution of that domain. This parameter adaptation enables the system to achieve high accuracy on new datasets without requiring transfer from pre-trained models trained on different data distributions.

The system segments the matching process into a simple blocking stage using heuristics and a more sophisticated matching stage using learning models. The blocking module maintains implementation simplicity by using straightforward blocking functions, while the matching module achieves high accuracy through learning-based approaches. This segmentation allows the system to combine the advantages of both simple heuristics and complex learning models, achieving high accuracy without excessive implementation complexity.

The blocking module performs preliminary filtering using simple heuristics before the learning-based matching module processes entity pairs. This preliminary action reduces the number of pairs that need to be processed by the complex matching module, making the overall system more manageable and easier to implement while maintaining high accuracy through the subsequent learning-based matching stage.