Semantic Column Expansion for Privacy-Aware Data Anonymization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data management systems lack effective methods for automatically classifying and anonymizing columnar data, especially in the context of increasing privacy regulations, which require understanding and protecting personal data while deriving insights.
Innovation Solution
A cloud computing platform that classifies columnar data by determining semantic categories using bloom filters, lookup tables, and machine learning models, and anonymizes data based on privacy categories to protect sensitive information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual data classification methods are used, then data understanding and governance are achieved, but the process is time-consuming and cannot scale to large datasets
Solution Approach 1:
The patent replaces manual mechanical classification processes with automated machine learning models and algorithms that can process large volumes of data rapidly. The system uses trained ML models to automatically classify data into sensitivity categories without human intervention, dramatically increasing productivity while reducing time loss.
Solution Approach 2:
The system enables data to be automatically classified and tagged through self-service mechanisms where the ML models autonomously analyze data characteristics, determine sensitivity levels, and apply appropriate classification tags without requiring manual review or intervention for each data element.
2Reliability
If data is anonymized to protect privacy, then compliance with privacy regulations is achieved, but data utility and ability to derive insights is reduced
Solution Approach 1:
The patent applies different levels of anonymization and protection to different data elements based on their classified sensitivity. Highly sensitive data receives stronger anonymization measures while less sensitive data maintains more utility. This local quality approach ensures privacy compliance for critical data while preserving data usefulness for analysis where appropriate.
Solution Approach 2:
The system applies partial anonymization where full anonymization would destroy data utility. Instead, it uses techniques like masking, generalization, or differential privacy that provide sufficient protection for the required compliance level while retaining enough information to derive valuable insights from the data.
3Reliability
If comprehensive data classification is performed to identify all personal data, then privacy regulation compliance is achieved, but system complexity and computational resources increase
Solution Approach 1:
The patent segments the data classification process into distinct stages: initial scanning, ML model analysis, sensitivity categorization, and tagging. Each segment handles a specific aspect of classification, making the overall complex system manageable and maintainable while achieving comprehensive coverage of personal data identification.
Solution Approach 2:
The ML classification system is designed to be universal and multi-functional, capable of handling various data types, formats, and sensitivity categories through a single integrated platform. This reduces overall system complexity by avoiding the need for separate specialized tools for different classification tasks.
Data Source
AI summary
An approach is disclosed that retrieves data from a data set organized in multiple columns, where a first column includes both a first and a second data type. The approach expands the first column into a second column for the first data type and a third column for the second data type; determines a semantic category for each data type; and assigns a privacy category to each semantic category. The approach then anonymizes the second column using a first anonymization technique based on the first privacy category, and anonymizes the third column using a second anonymization technique based on the second privacy category. In turn, the approach generates an anonymized view of the data set using the anonymized data.