Uniqueness-Based PII Detection in Telemetry Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for detecting Personal Identifiable Information (PII) in telemetry data rely on pattern matching algorithms, which are limiting as most PII data does not follow patterns, making it difficult to identify and prevent accidental collection.

Innovation Solution

A system and method that determines the likelihood of PII data in telemetry collections by analyzing data values for uniqueness, organizing data into sets based on parameters, and using a risk estimation module to assess the presence of PII without relying on known patterns.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If pattern matching algorithms are used to detect PII data, then detection can be performed for known patterns like email addresses and phone numbers, but most PII data that does not follow patterns cannot be detected

Engineering Contradiction:

ImprovePII detection accuracyVSAvoiddetection coverage

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent changes the detection parameter from pattern-based matching to uniqueness-based analysis. Instead of checking if data matches known PII patterns (email, phone, SSN), the system analyzes whether data values are unique across the dataset. This parameter change enables detection of both pattern-based and non-pattern-based PII data, resolving the contradiction between detection accuracy for known patterns and coverage for unknown patterns.

Inventive Principle:

Principle #35Parameter changes

2Productivity

If traditional pattern-based detection methods are used, then the detection process is simple and fast, but it is very limiting and cannot detect most PII data

Engineering Contradiction:

Improvedetection efficiencyVSAvoiddetection completeness

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent replaces the mechanical pattern-matching system with a statistical uniqueness analysis system. Instead of mechanically comparing data against predefined patterns, the system uses statistical analysis to identify unique data values that may represent PII. This substitution maintains computational efficiency while dramatically improving detection completeness, as uniqueness analysis can identify both pattern-based and non-pattern-based PII data.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Ease of manufacture

If known patterns are required for PII detection, then the detection method is straightforward, but it is very limiting as most PII data does not follow patterns

Engineering Contradiction:

Improvedetection method simplicityVSAvoiddetection applicability

Core Design Contradiction:

Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The patent creates a universal detection method based on data uniqueness that can identify all types of PII data regardless of format or pattern. The uniqueness-based approach serves multiple functions: it detects traditional pattern-based PII (emails, phones), non-pattern PII (passwords, custom identifiers), and even contextual PII that would be missed by pattern matching. This universal method maintains simplicity while dramatically expanding detection applicability.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS10482279B2Pattern-less private data detection on data sets

Publication Date: 2019.11.19 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

In an embodiment, data that is associated with one or more parameters that describe information corresponding to the data is accessed. The data is organized into data sets having the same combination of parameters. Each data set includes a data value determined by the combination of the parameters. The data values that are unique in relation to the other data values are determined. The likelihood that the accessed data includes PII data for the combination of parameters is determined based on the data values that are determined to be unique.