Data Distillation Using Anonymized Hashes for Privacy-Safe Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The sharing of user data by service providers raises privacy concerns and results in the generation and storage of redundant data, wasting computational resources.
Innovation Solution
A system and method for data distillation that uses a distributed database to merge, combine, and prune user data, maintaining privacy by using anonymized hashes and pseudonymized identifiers, and utilizing a distributed ledger for secure data sharing and management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If service providers share user data to maximize information discernment, then information completeness is improved, but user privacy is compromised and redundant data is generated
Solution Approach 1:
The patent extracts only the necessary identifying elements (anonymized hashes) from user data for matching purposes, while separating out personal identifiers. This allows data sharing and deduplication without exposing actual user identities, thus maintaining privacy while achieving information completeness.
Solution Approach 2:
The patent introduces anonymized hashes as an intermediary representation of user identifiers. These hashes serve as mediators that enable data matching and deduplication across providers without revealing actual user identities, thus resolving the contradiction between information sharing and privacy protection.
2Object-affected harmful factors
If service providers remove identifying information to maintain user privacy, then privacy protection is improved, but redundant data is generated and computational resources are wasted
Solution Approach 1:
The patent extracts anonymized hashes from user data that retain matching capability without containing personal identifiers. This extraction enables efficient deduplication through hash comparison while maintaining privacy, avoiding the need to store and process multiple redundant datasets.
Solution Approach 2:
The patent transforms user identifiers into anonymized hashes, changing the parameter representation from identifiable data to non-identifiable but matchable data. This parameter transformation enables efficient deduplication while preserving privacy, reducing computational waste.
3Adaptability or versatility
If service providers store multiple sets of repetitive user data, then data availability is improved, but storage efficiency is reduced
Solution Approach 1:
The patent merges multiple instances of user data that reference the same user into a single consolidated record identified by an anonymized hash. This combining eliminates redundant storage while maintaining data availability through the unified record.
Solution Approach 2:
The patent uses anonymized hashes as lightweight copies or references to user identifiers, enabling efficient data linking without duplicating full user profiles. This reference-based approach reduces storage requirements while maintaining data accessibility.
Data Source
AI summary
Systems and methods are described for distilling data. First data associated with a user may be received. The first data associated with the user may comprise an anonymized hash of an identifier associated with the user. A database may be determined to comprise a first record indicating the anonymized hash. The first record may comprise second data associated with the user. Based on the determining that the database comprises the first record, a second record may be generated. The second record may comprise the first data associated with the user, the second data associated with the user, and the anonymized hash. Based on the determining that the database comprises the first record, the example method may be stored to the database. These and other user and/or data distillation methods and systems are described herein.