Distributed Data Anonymization via Generalization Lattice
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for anonymizing large datasets are inefficient and unable to operate on big data due to memory constraints, as they require loading all data into a single computer's memory for processing, limiting their applicability to large data volumes.
Innovation Solution
A method that determines a combination of generalization levels for quasi-identifiers on a central node and transmits these levels to multiple sub-nodes for parallel anonymization, allowing for distributed processing and overcoming memory limitations by using a generalization lattice and search heuristics to achieve k-anonymity, l-diversity, and t-closeness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing anonymization methods are used, then k-anonymity and privacy conditions can be achieved, but the methods cannot operate on large datasets due to memory constraints requiring all data to be loaded into a single computer
Solution Approach 1:
The system divides the anonymization task across multiple computing nodes (master node and worker nodes), where each node processes a portion of the data independently. The master node coordinates the overall anonymization process while worker nodes perform local generalization and suppression operations, enabling the system to handle large datasets that cannot fit in single-machine memory.
Solution Approach 2:
The patent introduces a distributed computing framework with master and worker nodes that act as intermediaries between the data storage system and the anonymization algorithm. This intermediary layer enables efficient data processing by allowing nodes to communicate and coordinate without requiring all data to be loaded into a single memory space.
2Reliability
If higher generalization levels are applied to achieve k-anonymity, then privacy protection improves, but information loss increases
Solution Approach 1:
The system dynamically determines the optimal generalization level for each quasi-identifier attribute based on the data distribution and anonymity requirements. Rather than applying a fixed high generalization level, the algorithm adapts the generalization intensity to achieve the minimum necessary k-anonymity, thereby preserving as much information as possible while maintaining privacy protection.
Solution Approach 2:
The patent changes the parameter of generalization level dynamically during the anonymization process. The system evaluates different generalization levels and selects the optimal combination that achieves k-anonymity with minimal information loss, allowing the generalization parameter to vary across different attributes and data regions.
3Reliability
If the optimization problem for finding optimal generalization levels and suppression is solved using existing algorithms, then anonymization quality improves, but the algorithms are NP-hard and computationally complex
Solution Approach 1:
The complex optimization problem is segmented into smaller sub-problems that can be solved independently on different computing nodes. Each worker node handles a specific portion of the data and performs local optimization, while the master node coordinates the overall process. This segmentation reduces the computational complexity at each individual node while maintaining the global optimization objective.
Solution Approach 2:
The system implements an iterative anonymization process where generalization and suppression operations are applied continuously until the desired anonymity level is achieved. The algorithm repeatedly evaluates the data, applies transformations, and re-evaluates until k-anonymity is satisfied, ensuring progressive improvement rather than requiring a single complex optimization step.
Data Source
AI summary
Provided is a method for anonymising data stocks, including the steps of determining a combination of generalization stages for quasi-identifiers of a data stock at a central node; transmitting the combination of generalization stages to a plurality of sub-nodes; and a parallel performing of an anonymisation of the data stock on the basis of the combination of generalization stages by the sub-nodes.