Encrypted Hash Map Filtering for Secure Raw Data Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in securely transmitting raw data over networks, particularly in ensuring data privacy and security while minimizing computational resources and costs, especially when dealing with unknown domains and potential data poisoning attacks.
Innovation Solution
A system comprising a client device, a shuffler system, and an auxiliary system processes data using hash functions and encryption, creating a hash map to select and transmit encrypted data subsets based on frequency thresholds, ensuring only encrypted data is processed by the recipient system, thus maintaining privacy and reducing computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all raw data is transmitted and processed by the recipient system, then data accuracy is improved, but computational resource usage and network bandwidth increase
Solution Approach 1:
The patent extracts and processes only the essential features of the data (hash values, frequency information, domain characteristics) before transmission to the recipient system. The auxiliary system performs preprocessing operations including hashing, frequency counting, and domain determination, then transmits only the processed results rather than all raw data, thereby maintaining data accuracy while reducing computational resource usage.
Solution Approach 2:
The auxiliary system performs preliminary data processing operations before the recipient system receives the data. This includes computing hash values, determining frequencies, identifying domains, and filtering data subsets in advance. These preliminary actions reduce the computational burden on the recipient system while preserving the accuracy needed for subsequent processing.
2Reliability
If encryption is applied to all transmitted data, then data security is improved, but computational costs and processing time increase
Solution Approach 1:
The system extracts and transmits only the encrypted essential components (hash values, frequency data, domain information) rather than encrypting all raw data. This selective extraction approach maintains data security for the critical elements while avoiding the computational overhead of encrypting entire datasets, thereby improving processing speed without compromising security.
Solution Approach 2:
The patent applies encryption selectively to only the portions of data that require security protection (hash values, frequency information, domain characteristics) rather than applying it uniformly to all data. This partial application of encryption maintains security where needed while reducing overall computational costs and processing time.
3Measurement precision
If the auxiliary system processes all received data, then data analysis accuracy is improved, but network bandwidth and computational resources are wasted
Solution Approach 1:
The auxiliary system extracts and transmits only the essential analytical components (hash values, frequency distributions, domain identifiers, and selected data subsets) rather than transmitting all raw data. This extraction approach maintains data analysis accuracy by preserving the critical features needed for processing while significantly reducing the volume of data transmitted over the network.
Solution Approach 2:
The patent segments the data processing pipeline into distinct stages performed by different systems. The auxiliary system handles initial processing (hashing, frequency counting, domain determination) and transmits only the processed results to the recipient system. This segmentation allows each system to perform its specialized function efficiently, improving data analysis accuracy while reducing unnecessary data transmission.
4Productivity
If frequency threshold filtering is applied, then computational efficiency is improved, but data completeness may be reduced
Solution Approach 1:
The system applies frequency threshold filtering as a partial selection criterion rather than an absolute filter. Data items are selected for transmission based on whether their frequency meets the threshold, but the system also preserves other important characteristics (hash values, domain information) for all items. This partial application maintains computational efficiency while minimizing information loss by retaining essential data about all items regardless of frequency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing encrypted and hashed versions of raw data. One of the methods includes receiving, by a system, a message that comprises an encrypted version of raw data and a hash of the raw data; determining, using the hash of the raw data, a first location in a hash map; inserting the encrypted version of the raw data to the first location in the hash map; determining to transmit, to another system, data for a second location in the hash map; selecting, using the second location, at least a subset of the data for the second location in the hash map; and transmitting, to the other system, the subset of the data for the second location in the hash map.