Private Set Intersection Using Locality Sensitive Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for privacy-preserving record linkage in entity resolution, particularly in large datasets, face challenges in securely identifying matching records without revealing private information, especially when records differ due to typos, omissions, or different formatting, and often require trusted third parties that introduce additional security issues.
Innovation Solution
The method employs locality-sensitive hashing (LSH) to hash similar inputs into similar output values, allowing for private set intersection (PSI) to be performed on these hash values rather than the raw records, ensuring that only matching records with a high probability are identified without exposing private data, using a computer-implemented system to compute and compare hash values across datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If private set intersection is performed on raw records to identify matching records, then matching accuracy is improved, but private information is exposed
Solution Approach 1:
The patent applies preliminary hashing to transform raw records into hash values before performing private set intersection. This preliminary action (hashing) preserves the ability to identify matching records while preventing exposure of the original private information, as the hash values cannot be reversed to obtain the original data
Solution Approach 2:
The patent introduces hash values as an intermediary between the raw private records and the comparison process. Instead of directly comparing sensitive records, the system compares hash values which serve as a mediator, enabling matching identification without direct exposure of the underlying private information
2Productivity
If LSH is applied to hash similar records, then processing efficiency is improved, but false matches may increase
Solution Approach 1:
The patent uses LSH to perform partial matching by comparing only hash values rather than complete records. This partial action improves efficiency by avoiding full record comparisons, while the system accepts that some false matches may occur and handles them through additional verification steps
Solution Approach 2:
The patent transforms the comparison parameter from raw record data to hash values, changing the state of the data being compared. This parameter change enables efficient comparison of similar records through LSH while managing the trade-off between efficiency and accuracy through the properties of the hash function
3Reliability
If trusted third parties are used for privacy-preserving record linkage, then security is improved, but system complexity increases
Solution Approach 1:
The patent extracts the trust requirement from the system by using cryptographic hash functions that provide privacy preservation without requiring a trusted third party. The security is built into the mathematical properties of the hash function rather than relying on external trusted entities, thereby reducing system complexity
Solution Approach 2:
The system achieves privacy-preserving record linkage through self-service mechanisms using one-way hash functions. The parties can independently compute hash values and perform comparisons without needing a trusted intermediary to manage or verify the process, making the system autonomous and simpler
Data Source
AI summary
A computer-implemented method for privately determining data intersection is disclosed. The computer-implemented method includes performing private set intersection between two record sets to determine identical intersecting records corresponding to a particular record field. The computer-implemented method includes removing any identical intersecting records from each record set to form two record subsets. The computer-implemented method includes separately computing locality sensitive hash values for each of the two record subsets, wherein the locality sensitive hash values are computed for records corresponding to the particular record field. The computer-implemented method includes jointly performing private set intersection between the locality sensitive hash values separately computed for each of the two record subsets. The computer-implemented method further includes determining that an intersecting pair of records between the two record subsets are a match based, at least in part, on a similarity score associated with the intersecting pair of records being above a predetermined threshold.