Log Data Anonymization via Template-Based Sensitive Value Replacement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data analysis techniques, such as natural language processing and dictionary lookup, are limited in effectively identifying and removing sensitive information like personal identification information from unstructured and dynamic log messages due to their unstructured and dynamic nature, posing challenges in compliance with regulations like GDPR.
Innovation Solution
The method involves creating templates from associated documentation to annotate and identify sensitive data types in log messages, replacing these identified values with new data values to create a modified dataset, thereby ensuring compliance and maintaining data usability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If natural language processing and dictionary lookup techniques are used to identify sensitive information, then the process is simple and automated, but the accuracy of identifying sensitive information in unstructured and dynamic log messages is insufficient
Solution Approach 1:
The patent applies preliminary action by creating templates from documentation before processing the data set. These templates are prepared in advance to match patterns of sensitive information, enabling accurate identification when the templates are applied to the unstructured log messages. This pre-preparation of matching patterns resolves the contradiction by establishing accurate identification criteria before the automated processing begins.
Solution Approach 2:
The patent introduces templates as an intermediary between the automated processing system and the unstructured log messages. These templates serve as a mediator that bridges the gap between simple automated lookup and accurate pattern recognition in dynamic messages. The templates contain patterns derived from documentation that enable the automated system to accurately identify sensitive information varying in format and structure.
2Measurement precision
If templates are created from documentation to annotate and identify sensitive data, then the accuracy of sensitive information removal is enhanced, but the complexity of the processing system increases
Solution Approach 1:
The patent applies segmentation by breaking down the complex task of sensitive information identification into manageable template patterns. Each template represents a specific pattern or format of sensitive information derived from documentation. This segmentation allows the system to handle complexity in a structured way, where each template can be independently created, stored, and applied, reducing the overall system complexity while maintaining high accuracy.
Solution Approach 2:
The patent changes the parameter of data representation by transforming unstructured log messages into a structured format that matches template patterns. By changing how the data is represented and compared (from direct text matching to pattern-based template matching), the system achieves high accuracy without requiring equally high complexity in the processing logic.
3Reliability
If sensitive information is removed from the data set, then compliance with GDPR and security regulations is achieved, but the usability of the data for continued analysis is reduced
Solution Approach 1:
The patent applies the taking out principle by extracting and removing only the specific sensitive information portions from the log messages while preserving the rest of the data structure and content. This selective extraction ensures GDPR compliance by removing personally identifiable information while maintaining the usability of the remaining data for continued analysis, thus resolving the contradiction between compliance and data utility.
Data Source
AI summary
Embodiments of the invention are directed to annotating a log based on processing log documentation. Aspects include obtaining the data set and obtaining a set of templates, wherein one or more template of the set of templates includes a variable labeled as the data type. Aspects also include annotating the data set based on the set of templates, wherein the annotating includes labeling each value of the variable in the data set with the data type. Aspects further include creating an updated data set by replacing each value of the variable in the data set that is labeled with the data type with a new data value.