Differential Compression That Ignores Anonymized Data Fields
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data storage systems face inefficiencies in differential compression due to data anonymization, which alters data fields and prevents effective similarity hashing, leading to increased storage space occupation and reduced performance.
Innovation Solution
A data storage arrangement and method that incorporates anonymization-aware differential compression, where similarity hashes are generated by ignoring anonymized data portions, enabling efficient compression by masking and deleting these portions, thus improving the likelihood of matching hashes and reducing storage space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data anonymization is applied to protect privacy, then privacy protection is improved, but differential compression effectiveness deteriorates
Solution Approach 1:
The patent segments data into two categories: anonymized fields and non-anonymized fields. Similarity hashing is applied selectively only to non-anonymized fields, while anonymized fields are excluded from the hashing process. This segmentation allows the system to maintain privacy protection for sensitive data while preserving compression effectiveness for identifiable data portions.
Solution Approach 2:
The patent applies different quality treatments to different parts of the data structure. Non-anonymized fields undergo full similarity hashing for compression, while anonymized fields are masked out and excluded from hashing. This local differentiation ensures that compression efficiency is maintained in appropriate regions without compromising privacy in sensitive regions.
2Productivity
If conventional similarity hashing is used for differential compression, then compression efficiency is improved, but duplicate data identification fails with anonymized data
Solution Approach 1:
The patent extracts and removes anonymized fields from the data before applying similarity hashing. By taking out the anonymized portions that would interfere with hash matching, the system preserves the accuracy of similarity detection for the remaining non-anonymized fields, thereby maintaining both compression efficiency and matching precision.
Solution Approach 2:
Instead of hashing the entire data block including anonymized fields (which would prevent matching), the patent inverts the approach by hashing only the non-anonymized portions. This inversion of the conventional hashing approach allows duplicate identification to succeed despite the presence of anonymization in parts of the data.
3Reliability
If anonymized data is stored without differential compression, then privacy is maintained, but storage space consumption increases
Solution Approach 1:
The patent performs preliminary masking of anonymized fields before the differential compression process. By pre-identifying and masking the anonymized portions, the system enables subsequent similarity hashing to work effectively on the remaining data, achieving compression without compromising privacy. This preliminary action prevents storage space waste while maintaining privacy protection.
Data Source
AI summary
An example data storage device includes a memory and a controller. The controller is configured to store at least one of the one or more data elements utilizing differential compression. The controller is further configured to receive a data element to be stored, generate a copy of the data element to be stored, and mask data to be anonymized by deleting one or more portions to be anonymized. The controller is further configured to generate similarity hashes for one or more portions of the copy of the data element with masked data for finding one or more reference portions, and compress the data element to be stored utilizing differential compression with reference to the one or more reference portions.