Data Tokenization Tables for Format-Preserving Anonymization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data anonymization techniques impose burdensome processes and lead to poor quality or lack of anonymized data, affecting timeliness and increasing development costs, as they often require preliminary access to production data and fail to preserve the format of the original data.
Innovation Solution
A tokenizer system that generates token tables with replacement mappings to maintain the format of sensitive data, allowing users without access to production data to create anonymized datasets efficiently, using a token table that can be accessed in a controlled manner and distributed across computing nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional anonymization processes are applied to generate anonymized data, then data security is improved, but data quality and timeliness deteriorate due to burdensome access processes
Solution Approach 1:
The patent introduces an intermediary system that sits between the production data environment and the anonymized data environment. This intermediary automatically performs anonymization transformations using pre-defined mapping rules, eliminating the need for users to directly access production data while maintaining both security and data quality. The intermediary handles the complex anonymization process transparently, allowing users to work with anonymized data as if it were the original data.
Solution Approach 2:
The patent implements preliminary action by pre-defining anonymization mapping rules and transformations before the actual data anonymization process. These rules are established in advance based on the structure and characteristics of production data, enabling rapid anonymization without requiring users to perform preliminary access procedures. The system proactively maintains the anonymization mappings, ensuring data quality is preserved from the outset.
2Reliability
If access control constraints are enforced on production data, then data security is improved, but user understanding and task completion deteriorate
Solution Approach 1:
The patent creates accurate copies of production data structures and formats in the anonymized data environment. These copies maintain the same schemas, data types, and organizational structures as the original production data, but with anonymized values. Users can work with these structural copies to understand and complete their tasks without needing access to the actual sensitive production data, thus maintaining both security and operational ease.
Solution Approach 2:
The intermediary system translates user requests and operations between the anonymized data environment and the production data environment. It allows users to interact with anonymized data as if it were production data, automatically handling the security constraints in the background. This mediator eliminates the need for users to navigate complex access control procedures while maintaining security boundaries.
3Reliability
If anonymization transformations are applied to protect sensitive data, then data security is improved, but data format preservation deteriorates
Solution Approach 1:
The patent applies parameter changes by transforming only the sensitive value parameters while preserving the structural parameters of the data. The anonymization process changes the content parameters (actual data values) through substitution and masking, while maintaining the structural parameters (data types, formats, schemas, and organizational structures) identical to the original production data. This selective parameter transformation ensures both security and format preservation.
Solution Approach 2:
The patent implements local quality by applying different treatment to different parts of the data. Sensitive fields undergo anonymization transformations while non-sensitive fields and the overall data structure remain unchanged. The anonymization is applied locally to specific columns and rows that contain sensitive information, while preserving the global data format and structure, ensuring that the anonymized data maintains its original format characteristics.
4Manufacturing precision
If users must obtain preliminary access to production data, then data quality control is improved, but development costs and time investment increase
Solution Approach 1:
The patent performs preliminary action by pre-establishing anonymization mappings and transformations based on the production data structure. The system proactively analyzes and understands the production data formats in advance, creating ready-to-use anonymization rules and mappings. This preliminary setup eliminates the need for users to spend time obtaining access permissions, as the anonymization infrastructure is already in place and configured to maintain data quality automatically.
Solution Approach 2:
The patent implements self-service by enabling the anonymization system to automatically maintain data quality without requiring user intervention in the access process. The system autonomously performs format validation, mapping maintenance, and quality assurance on the anonymized data, freeing users from the time-consuming process of obtaining production data access while ensuring data quality is maintained through automated mechanisms.
Data Source
AI summary
A device, method, and system for treating data from legacy infrastructure is disclosed. Illustratively, the device memory stores computer executable instructions that when executed by the processor cause the processor to provide a dataset comprising a plurality of characters and provide a token table for tokenizing datasets. The token table includes mappings that define replacement tokens for characters in datasets. The instructions cause the processor to generate a tokenized dataset based on the dataset by (1) for each contiguous sequence of letter characters of the plurality of characters, determining a respective letter token having the same length as the respective contiguous sequence, and (2) generate the tokenized dataset by replacing each contiguous sequence of letter characters of the plurality of characters with the determined respective letter token.