Approximate Linkage of Datasets Over Quasi-Identifiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for linking datasets over encrypted quasi-identifiers face challenges in achieving accurate linkage while protecting privacy, as hashing quasi-identifiers leads to poor linkage accuracy due to standardization, schema, typographical errors, and noisy data issues, and introduces vulnerabilities like rainbow attacks.
Innovation Solution
An approximate linkage system that uses a semi-trusted third party to perform linkage on hashed and encrypted quasi-identifiers, employing generalization logic data structures and similarity scores to enable high-quality linkage without revealing original values, allowing data owners to post-process results for final linkage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If quasi-identifiers are hashed for privacy protection, then privacy is protected, but linkage accuracy deteriorates due to standardization issues, schema differences, typographical errors, and noisy data
Solution Approach 1:
The patent segments the quasi-identifier matching process into multiple hierarchical levels (e.g., city → state → country). Instead of requiring exact hash matches at a single level, the system allows progressive generalization from specific to general categories, enabling matches even when data represents the same entity at different levels of detail or contains minor variations.
Solution Approach 2:
The system dynamically changes the parameter of generalization level during the linkage process. By adjusting the granularity of quasi-identifier representation (e.g., switching between specific city names and broader geographic regions), the system adapts to handle data quality issues while maintaining privacy protection through hierarchical abstraction.
2Measurement precision
If exact matching of hashed quasi-identifiers is used, then linkage accuracy is improved, but privacy protection deteriorates due to vulnerabilities like rainbow attacks
Solution Approach 1:
The patent applies preliminary hashing and encryption to quasi-identifiers before any matching operations. By pre-processing the data with cryptographic transformations and organizing it into hierarchical structures, the system establishes privacy protection in advance, allowing subsequent approximate matching without exposing raw data to attacks like rainbow tables.
Solution Approach 2:
The system introduces an intermediary hierarchical structure between the raw quasi-identifiers and the matching process. This intermediate representation uses generalized categories and similarity scores rather than exact hashes, serving as a mediator that enables accurate matching while preventing direct exposure of original data values to potential attackers.
3Reliability
If generalization and approximate matching is used, then privacy protection is maintained, but linkage accuracy deteriorates due to loss of specific information
Solution Approach 1:
The patent implements a dynamic matching approach where the system first attempts exact or near-exact matches at specific hierarchical levels, then progressively generalizes to broader categories only when necessary. This dynamic adjustment of matching strictness based on data characteristics maintains privacy while maximizing linkage accuracy by preserving specific information when possible.
Solution Approach 2:
The system applies partial generalization only to the extent necessary for privacy protection and matching success. Rather than uniformly generalizing all quasi-identifiers to broad categories, the system selectively applies generalization at appropriate hierarchical levels, maintaining as much specific information as possible while still achieving privacy-protected approximate matching.
Data Source
AI summary
A mechanism is provided in a first client for approximate linkage of datasets over quasi-identifiers. The mechanism receives a generalization logic data structure representing sets of values for each quasi-identifier in a first dataset of the first client. For each record in the first dataset, the mechanism generates at least one generalization of a value of a given quasi-identifier in the first dataset based on a selected generalization logic data structure corresponding to the given quasi-identifier and generates a generalized record for each of the at least one generalization to form a first generalized dataset. The mechanism sends the first generalized dataset to a semi-trusted third party for approximate linkage of the first dataset with a second dataset of a second client, receives an approximate join result from the semi-trusted third party, performs post-processing on the approximate join result, and determines a final linkage result based on the post-processing.