Bloom Filter Meta-Estimation for Secure VoC Sketch Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data analysis systems face challenges in efficiently representing and processing large datasets of client device identifiers while preserving privacy and security, as existing sketch representations like EBF and BF sketches are vulnerable to privacy breaches, while VoC sketches offer better security but are computationally intensive.
Innovation Solution
A method to convert EBF or BF sketches to VoC sketches, allowing for secure transmission and processing of large datasets by generating a VoC sketch that represents the information in EBF or BF sketches, reducing computational overhead through a meta-estimation process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If EBF or BF sketches are used for data representation, then processing efficiency is improved, but privacy and security are compromised
Solution Approach 1:
The patent introduces a conversion process that transforms EBF or BF sketches into VoC sketches through a meta-estimation intermediary system. This mediator converts the insecure sketch format into a secure one without requiring access to the original data, thereby resolving the security vulnerability while maintaining processing efficiency.
Solution Approach 2:
The patent changes the fundamental parameters of the sketch representation by converting from EBF/BF structure to VoC structure. This parameter transformation alters the security properties of the data structure, making it resistant to privacy breaches while preserving the ability to perform efficient set operations.
2Reliability
If VoC sketches are used for secure data representation, then privacy and security are improved, but computational overhead increases
Solution Approach 1:
The patent performs the computationally intensive conversion from EBF/BF to VoC sketches in advance, before the actual set operations are needed. This preliminary action creates a secure representation that can then be used for multiple subsequent operations without repeating the heavy computational work, thereby reducing overall computational overhead.
Solution Approach 2:
The patent creates a converted copy of the EBF or BF sketch in VoC format through meta-estimation. This copied representation maintains the essential information needed for set operations while having the security properties of VoC sketches, avoiding the need to work directly with the computationally intensive original format.
3Measurement precision
If traditional de-duplication processes are applied to convert sketches, then accurate VoC sketch generation is achieved, but processing time increases significantly
Solution Approach 1:
The patent replaces the mechanical de-duplication process with a meta-estimation algorithm that directly computes VoC sketch values from EBF or BF sketches. This substitution eliminates the need for actual de-duplication operations, achieving accurate conversion without the significant time penalty that would result from processing and removing duplicates.
Solution Approach 2:
The meta-estimation process serves as an intermediary that bridges EBF/BF sketches and VoC sketches without requiring the intermediate step of actual de-duplication. This intermediary computation achieves the same accuracy goal as de-duplication would provide, but much more efficiently by working directly with the sketch data structures.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for the meta-estimation of data structures representing identifiers are disclosed. The system maintain one or more data records comprising one or more identifiers and one or more attributes. Using the data records, the system can generate a first data structure, such as a probabilistic data structure, that represents the plurality of data records. The first data structure can have a plurality of registers. The system can identify a subset of the plurality of registers that are equal to a predetermined value, and generate a second data structure that represents the subset of the plurality of registers. The system can then store the second data structure as a meta-estimation of the first, and can utilize the second data structure in further processing operations.