Bloom Filter Union Cardinality Estimation Under Non-Uniform Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audience measurement systems face challenges in accurately estimating the cardinality of users represented in arbitrarily distributed Bloom filter arrays across multiple database proprietors while preserving user privacy and avoiding the use of third-party cookies, leading to double counting and inaccurate reach metrics.
Innovation Solution
The use of Bloom filter arrays with agreed-upon hash functions and noise addition to estimate cardinality across multiple database proprietors, allowing for deduplication of user representations without revealing personally identifiable information, thereby generating sketch data that provides accurate unique audience size estimates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If database proprietors share user data to enable accurate audience measurement, then measurement precision improves, but user privacy protection deteriorates
Solution Approach 1:
The patent creates a simplified representation (copy) of user data in the form of Bloom filter arrays. These arrays contain only the necessary information for cardinality estimation while omitting personally identifiable information. The copies are then shared between database proprietors and the AME for audience measurement without exposing actual user data, thus resolving the contradiction between measurement accuracy and privacy protection.
Solution Approach 2:
The patent introduces Bloom filter arrays as an intermediary data structure between database proprietors and the AME. This intermediary enables the transmission of aggregate statistical information (cardinality estimates) without transmitting sensitive user information. The Bloom filters act as a mediator that preserves measurement precision while eliminating direct exposure of user privacy data.
2Adaptability or versatility
If third-party cookies are used to track users across platforms, then audience measurement coverage improves, but user privacy protection and platform independence deteriorate
Solution Approach 1:
The patent extracts the essential functionality of cross-platform user tracking from third-party cookies. Instead of relying on cookies that are embedded in and controlled by third-party domains, the system extracts user identification and tracking to locally maintained Bloom filter arrays at each platform. This eliminates dependency on third-party cookies while preserving the ability to measure cross-platform audience reach.
Solution Approach 2:
The patent enables each database proprietor to maintain and manage their own Bloom filter arrays independently, without requiring third-party cookie infrastructure. Each platform serves itself by locally tracking users through their own Bloom filters, then sharing only the aggregated cardinality information with the AME. This self-service approach replaces the centralized third-party cookie model with distributed platform-owned tracking.
3Measurement precision
If Bloom filter arrays with noise addition are used to estimate cardinality, then measurement precision and privacy protection improve, but computational complexity increases
Solution Approach 1:
The patent applies parameter changes by introducing noise with controlled probability (e.g., 10% probability of flipping bits) into the Bloom filter arrays. This parameter modification enables differential privacy protection while maintaining the ability to estimate cardinality. The noise parameter is tuned to balance privacy protection with measurement accuracy, resolving the contradiction between precision and complexity.
Data Source
AI summary
Methods, apparatus, systems, and articles of manufacture to estimate cardinality of users represented in arbitrarily distributed bloom filter arrays are disclosed. A system includes a communication interface to: access a first Bloom filter array representative of first entries in a first database, the first entries allocated to ones of first elements in the first Bloom filter array based on a non-uniform distribution of outputs of a hash function applied to the first entries, and access a second Bloom filter array representative of second entries in a second database. The system also includes machine readable instructions to cause one or more processors to estimate a cardinality of a union of the first and second entries based on the non-uniform distribution of the outputs of the hash function.