Differential Privacy Sketches for Cross-Provider Frequency Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for deduplicating data sets across multiple providers are limited in per-frequency analysis due to privacy concerns, leading to increased size, complexity, and error in data exchange protocols, and violate differential privacy standards.
Innovation Solution
A protocol for generating and combining private sketches that encode user identities without frequency data, allowing providers to share privatized interaction data securely, reducing size and complexity while maintaining differential privacy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If providers share detailed interaction data to enable accurate frequency analysis, then measurement precision is improved, but privacy security deteriorates
Solution Approach 1:
The patent extracts frequency information from detailed interaction data by generating private sketches that encode only frequency counts without exposing user identities or specific interaction details. This allows frequency analysis to be performed on aggregated data while keeping individual-level information private.
Solution Approach 2:
The patent introduces private sketches as an intermediary data structure that mediates between raw interaction data and frequency analysis requirements. These sketches encode frequency information in a privatized form that can be shared and combined across providers without revealing sensitive user information.
2Object-affected harmful factors
If providers share privatized data to maintain differential privacy, then privacy security is improved, but measurement precision deteriorates
Solution Approach 1:
The patent changes the parameter representation by encoding frequency information in a transformed space using private sketches. Instead of sharing raw frequency counts that might reveal sensitive patterns, the system uses sketch-based representations that preserve statistical properties while protecting privacy, allowing accurate frequency analysis to be recovered through proper decoding.
3Measurement precision
If detailed frequency data is shared across providers, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent uses sketch-based representations that are compact copies of the essential frequency information. Instead of exchanging complete detailed frequency datasets, providers exchange condensed sketch representations that capture the necessary information for deduplication and frequency analysis with significantly reduced data volume and protocol complexity.
4Measurement precision
If complete interaction data is exchanged for accurate deduplication, then measurement precision is improved, but loss of information increases due to privacy protection requirements
Solution Approach 1:
The patent extracts and retains only the essential frequency information needed for deduplication while discarding or protecting sensitive user identity information. The private sketches encode frequency counts and patterns necessary for accurate deduplication without containing or exposing identifiable user data, thus maintaining measurement precision while protecting privacy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods are disclosed herein for improved per-frequency counting systems that record interactions between individuals and a group of providers while maintaining differential privacy. A protocol may be defined that specifies frequency bins to categorize identifiers corresponding to individuals. A provider may generate a plurality of private sketches, each corresponding to a plurality of frequencies defined in the protocol. Frequency data is determined for each identifier. Identifiers are encoded into the private sketches corresponding to the identifiers' associated frequency. The plurality of private sketches from each provider in the group of providers are combined to generate a deduplicated distribution across the group. In one implementation, the private sketches of each provider are sequentially merged until all sketches have been combined, from which the total distribution can be estimated.