Hash-Derived Dictionary Indexing for Consistent Data Deidentification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data deidentification techniques fail to generate outputs that are truly anonymized, consistent, and representative of real data, consuming significant computing resources and being error-prone, while often being limited to specific datastores and requiring extensive custom implementations.
Innovation Solution
A data deidentification system utilizing hash-derived indexing substitution models that select dictionaries based on original data, hash the data into codes, extract sequences from these codes, and retrieve substitution values from sorted dictionaries to generate deidentified data, ensuring the output is representative, consistent, and reflective of the original data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If current data deidentification techniques are used, then data can be deidentified, but the output is not truly anonymized, consistent, and representative of real data
Solution Approach 1:
The patent introduces hash-derived indexing as an intermediary mechanism between original data and substitution dictionaries. This intermediary layer ensures consistent mapping by using hash codes to index into sorted dictionaries, guaranteeing that the same input always produces the same substitution while maintaining representativeness of the original data distribution.
Solution Approach 2:
The patent creates sorted copies of substitution dictionaries that are indexed by hash codes. These copied and organized substitution values maintain the representative characteristics of real data while being decoupled from the original sensitive information, enabling consistent and reliable deidentification without exposing actual PII.
2Reliability
If current data deidentification techniques are used, then data can be deidentified, but significant computing resources are consumed
Solution Approach 1:
The patent performs preliminary sorting of substitution dictionaries and pre-computation of hash indexing structures before actual deidentification operations. This preliminary organization allows subsequent deidentification tasks to use simple hash-based lookups instead of complex real-time processing, significantly reducing computing resource consumption while maintaining high accuracy.
Solution Approach 2:
The patent replaces complex mechanical processing operations with hash-based indexing and dictionary lookup operations. Instead of performing heavy computational analysis during deidentification, the system uses efficient hash functions to map data to pre-sorted dictionary entries, dramatically reducing the computational energy required while preserving deidentification quality.
3Reliability
If current data deidentification techniques are used, then data can be deidentified, but the process is error-prone
Solution Approach 1:
The patent changes the fundamental parameter of data mapping from complex many-to-many relationships to deterministic hash-based one-to-one mappings. By transforming the deidentification process into a parameter-driven system where hash codes directly index into sorted dictionaries, the patent eliminates errors associated with manual mapping and ensures consistent, reproducible results across different implementations.
4Adaptability or versatility
If current data deidentification techniques are used, then data can be deidentified, but they are limited to specific datastores
Solution Approach 1:
The patent creates a universal deidentification framework based on hash-derived indexing that can operate with any datastore type. The system uses language-agnostic hash functions and standardized dictionary structures that work across different programming languages and database systems, making the solution broadly adaptable while remaining easy to implement through consistent lookup operations.
Data Source
AI summary
A device may receive original data to be deidentified and may select one or more dictionaries, from a plurality of dictionaries, based on the original data. The device may sort the one or more dictionaries based on an output control key to generate one or more sorted dictionaries, and may hash the original data into one or more hash codes. The device may extract a sequence of a quantity of digits or characters, from each of the one or more hash codes, to generate one or more sequences, and may retrieve, from the one or more sorted dictionaries, one or more substitution values corresponding to the one or more sequences. The device may generate deidentified data based on the one or more substitution values, and may perform one or more actions based on the deidentified data.