Entity Matching Model With Human-in-the-Loop Confidence Thresholding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The proliferation of diverse sources of metadata commentary across various platforms makes timely manual identification and review impractical, leading to a need for an entity matching solution that balances automation efficiency with human accuracy.
Innovation Solution
A trained entity matching predictive model is used to match metadata from different sources, with a 'human-in-the-loop' process confirming or rejecting matches below a certain confidence threshold, combining automated efficiency with human accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated systems are used for entity matching, then efficiency is improved, but accuracy deteriorates
Solution Approach 1:
The system segments the entity matching process into two distinct phases: an automated phase that handles initial matching and high-confidence cases, and a human review phase that handles ambiguous or low-confidence cases. This segmentation allows the system to leverage automation for efficiency while preserving human accuracy for challenging cases.
Solution Approach 2:
The system introduces an intermediary human-in-the-loop mechanism that acts as a mediator between automated matching and final decision-making. Humans review and confirm or correct matches that fall below a confidence threshold, thereby improving accuracy without completely sacrificing automated efficiency.
2Measurement precision
If manual review is used for entity matching, then accuracy is improved, but time consumption increases
Solution Approach 1:
Instead of requiring full manual review of all matches, the system applies partial human action only to cases that fall below a confidence threshold. High-confidence matches are accepted automatically, while only ambiguous cases require human review, thereby reducing overall time consumption while maintaining accuracy where needed.
Solution Approach 2:
The system dynamically adjusts the confidence threshold parameter to balance between automation and human review. By changing this parameter, the system can optimize the division of labor between automated and manual processes based on specific operational requirements, thereby controlling time consumption while preserving accuracy.
Data Source
AI summary
A metadata aggregation system includes a computing platform having a hardware processor and a memory storing a software code including a trained entity matching predictive model trained using training data obtained from a reference database. The hardware processor executes the software code to obtain metadata inputs from multiple sources, conform the metadata inputs to a common format, match, using the trained entity matching predictive model, at least some of the conformed metadata inputs to the same entity, and determine, using the trained entity matching predictive model, a confidence score for each match. The software code further sends a request to one or more human editor(s) for confirmation of each match having a confidence score greater than a first threshold and less than a second threshold, and updates the reference database, in response to receiving a confirmation that at least one match is a confirmed match, to include the confirmed match.