Data Association System Anonymization Pipeline Orchestration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data anonymization systems face challenges in efficiently managing data storage and deletion processes, particularly in handling errors during anonymization processing, which can lead to data inconsistencies and security risks due to unresolved personal information.
Innovation Solution
A data association system that includes a data collection system, a data storage system, and an anonymization control system, where the anonymization control system instructs either the data collection or storage system to delete data before anonymization is completed, ensuring secure data handling and error management through pipeline orchestration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored before anonymization processing, then data availability for processing is improved, but data security deteriorates due to risk of personal information exposure
Solution Approach 1:
The system performs preliminary storage of data in the data storage system before anonymization processing, ensuring data availability. The anonymization control system then coordinates the anonymization process and subsequently instructs deletion of the pre-anonymized data, thus resolving the security risk while maintaining data availability during processing
Solution Approach 2:
The system stores data temporarily in the data storage system for processing purposes, then discards the pre-anonymized data after anonymization is complete by executing deletion instructions from the anonymization control system. This temporary storage and subsequent disposal approach maintains data availability when needed while eliminating security risks afterward
2Reliability
If anonymization processing is executed with error handling, then processing reliability is improved, but data consistency deteriorates due to potential unresolved personal information
Solution Approach 1:
The anonymization control system monitors the anonymization processing and implements feedback mechanisms to detect errors. When errors are detected, the system can re-execute the anonymization process or take corrective actions to ensure data consistency, thus maintaining both processing reliability and data stability
Solution Approach 2:
The system prepares error handling measures in advance by having the anonymization control system monitor processing status and have deletion capabilities ready. If anonymization fails, the pre-prepared error handling ensures that personal information is still deleted or re-processed, cushioning against data consistency issues while maintaining processing reliability
3Object-affected harmful factors
If pre-anonymized data is deleted after processing, then data security is improved, but storage efficiency deteriorates due to additional deletion operations
Solution Approach 1:
The data storage system automatically executes deletion of pre-anonymized data when instructed by the anonymization control system. This self-service approach allows the storage system to handle its own data management tasks, improving data security through automated deletion while minimizing the overhead of additional operations through efficient integration with the existing data management workflow
Data Source
AI summary
A data association system includes a POST connector which collects data maintained in an information system a pipeline which stores the data collected by the POST connector; and a pipeline orchestrator which controls anonymization of the data stored by the pipeline. the data association system is characterized in that the pipeline executes anonymization processing of anonymizing the data, the POST connector and the pipeline store data before being anonymized by the anonymization processing, and the pipeline orchestrator instructs, after execution of the anonymization processing, the POST connector and the pipeline to delete the data before being anonymized.