Entity Source Scoring for Trustworthy Resolution Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing entity resolution systems face challenges in determining the trustworthiness and believability of new data sources and assessing the impact of existing data sources on the effectiveness and defensibility of entity resolution decisions, leading to the purchase of duplicate and unreliable data.
Innovation Solution
A computational method for scoring the trustworthiness and believability of data sources by analyzing localized and random samples, using profiling and entity resolution processes to generate a profiling scorecard that evaluates data quality and impact on the entity resolution system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data vendors provide highly curated samples for evaluation, then the sample quality appears high, but the sample is not representative of the entire data set making extrapolation questionable
Solution Approach 1:
The system performs preliminary actions by requesting vendors to provide both highly curated samples and random samples before full data integration. This advance preparation allows the entity resolution system to assess both quality and representativeness beforehand, preventing unreliable data from being added to the data graph.
Solution Approach 2:
The system requests more data samples than traditionally provided by vendors - specifically requiring both highly curated samples and random samples. This excessive sampling approach ensures sufficient data for comprehensive evaluation of both quality metrics and population representativeness.
2Reliability
If manual evaluation of data sources is performed, then subjective human judgment can assess data quality, but the process is time-consuming and inconsistent
Solution Approach 1:
The system replaces manual human evaluation with an automated entity resolution system that computationally processes data samples. The system uses algorithms to perform entity resolution, generate metrics, and produce scoring reports automatically, eliminating time-consuming manual review while maintaining consistent, objective assessment criteria.
Solution Approach 2:
The evaluation system is self-service in that it automatically processes incoming data samples, performs entity resolution analysis, generates quality metrics, and produces scoring reports without requiring manual intervention. The system serves itself by autonomously evaluating data source quality and providing actionable insights.
3Measurement precision
If vendors provide full data sets for evaluation, then complete data quality can be assessed, but vendors rarely provide full data sets before purchase
Solution Approach 1:
The system accepts partial data provision from vendors (samples rather than full datasets) but processes these samples excessively thoroughly using comprehensive entity resolution analysis. This allows near-complete evaluation quality to be achieved from limited sample data, making the process feasible for vendors while maintaining assessment rigor.
Solution Approach 2:
The system creates a representative copy of the vendor's data population through random sampling, then performs comprehensive evaluation on this copy. The entity resolution system analyzes the sample copy to infer qualities of the full dataset, enabling thorough assessment without requiring vendors to provide complete data sets.
4Adaptability or versatility
If existing data sources are added to expand data graph reach, then more entities can be resolved, but duplicate and unreliable data may be introduced
Solution Approach 1:
The system performs preliminary evaluation of candidate data sources before integration into the data graph. By assessing trustworthiness, representativeness, and expected impact in advance, the system prevents unreliable or duplicate data from being added, thereby expanding reach while maintaining data quality and defensibility.
Solution Approach 2:
The system provides feedback to vendors about their data quality metrics and representativeness scores. This feedback loop enables vendors to improve their data quality and allows the system to make informed decisions about which data sources to integrate, balancing reach expansion with reliability maintenance.
Data Source
AI summary
A method for computationally scoring the trustworthiness of source data begins with a subset of the raw source data. Fields and related fields are identified, profiled, and results aggregated. In addition, the subset of raw source data is input to an entity resolution system, with results summarized and aggregated. The output of these two streams of processes are used to compute a scorecard. In addition, a sandbox of the entity resolution system's data graph is constructed, the sandbox is modified with the source data, and the difference between the baseline sandbox and modified sandbox is computed. Changes to entities are computed for all entities as well as for most sought-after entities, and the results are summarized and aggregated into the overall scorecard.