Rule-Based Document Scrubbing for Sensitive Telemetry Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Telemetric data often includes sensitive information that complicates analysis and needs to be protected while preserving valuable performance insights, as existing methods struggle to efficiently filter or anonymize such data across diverse document formats.
Innovation Solution
A rule-based attribution technique applies script-based rules to identify and tag sensitive data, allowing for its scrubbing through deletion, obfuscation, or conversion into non-sensitive formats, using techniques like simple hashing or rolling hashes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sensitive data is removed from telemetric data, then data security is improved, but data completeness and analysis value deteriorate
Solution Approach 1:
The patent segments telemetric data into distinct categories: identifiable sensitive data (requiring removal), pseudonymizable data (requiring anonymization), and actionable telemetry data (to be preserved). This segmentation allows selective processing that removes only necessary sensitive information while preserving analytical value through pseudonymization techniques that maintain data utility for analysis.
Solution Approach 2:
The patent introduces pseudonymization as an intermediary technique between complete data removal and full data retention. By replacing directly identifiable information with pseudonyms or hashed values, the system maintains a middle ground where data security is enhanced through removal of direct identifiers, yet data completeness is preserved through retained pseudonymized data that can still be used for analysis and correlation.
2Reliability
If comprehensive scrubbing of sensitive data is performed, then data protection is improved, but processing complexity and time increase
Solution Approach 1:
The patent performs preliminary classification and tagging of data fields before the actual scrubbing process. By pre-identifying which fields contain sensitive data and categorizing them by sensitivity level, the system prepares the data structure in advance, allowing the subsequent removal and pseudonymization operations to proceed efficiently without requiring complex real-time analysis during the scrubbing phase.
Solution Approach 2:
The patent applies different scrubbing intensities to different data fields based on their sensitivity classification. Rather than uniformly processing all data with the same level of scrutiny, the system applies targeted removal for highly sensitive fields and selective pseudonymization for moderately sensitive fields, optimizing processing time by avoiding unnecessary intensive scrubbing of low-risk data while maintaining strong protection for critical sensitive information.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A rule-based attribution mechanism analyzes documents having different types of data in different formats through the application of script-based rules that apply a tag to the document identifying the type of sensitive data that is contained in the document. Documents having similar tags are aggregated so that the sensitive data is scrubbed from the document leaving the telemetric data available for downstream processing. The scrubbing entails different actions, such as, eliminating the sensitive data, obfuscating the sensitive data, and converting the sensitive data into a non-sensitive value.