Principal Components Analysis for Data Fingerprinting

Solution Overview

Problem

Existing data watermarking and fingerprinting methods are inefficient for large datasets, such as those with hundreds of millions of records, as they require extensive preprocessing and computational resources, making it impractical to compare leaked data with reference datasets in a timely manner.

Innovation Solution

The application of Principal Components Analysis (PCA) to reduce datasets to a smaller set of principal components, allowing for efficient comparison of leaked data with reference datasets by generating statistical correlations rather than comparing individual cell values, thereby reducing processing times and computational complexity.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional data watermarking and fingerprinting methods are used on large datasets, then data ownership can be authenticated, but computational complexity and processing time increase significantly

Engineering Contradiction:

Improvedata ownership authenticationVSAvoidcomputational complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent extracts only the most significant statistical features (mean, standard deviation, skewness, kurtosis) from each variable in the dataset, rather than processing all individual data points. This extraction of essential characteristics reduces the data dimensionality while preserving the unique fingerprint of the dataset, enabling efficient comparison without full dataset processing

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent transforms the original data parameters into statistical descriptors (mean, standard deviation, skewness, kurtosis) that capture the essential characteristics of each variable. This parameter transformation converts raw data into a compact statistical representation that maintains authentication reliability while reducing computational burden

Inventive Principle:

Principle #35Parameter changes

2Reliability

If traditional data watermarking and fingerprinting methods are used on large datasets, then data ownership can be authenticated, but processing time becomes impractical

Engineering Contradiction:

Improvedata ownership authenticationVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent extracts only the most significant statistical features (mean, standard deviation, skewness, kurtosis) from each variable in the dataset, rather than processing all individual data points. This extraction of essential characteristics reduces the data dimensionality while preserving the unique fingerprint of the dataset, enabling efficient comparison without full dataset processing

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies partial action by computing statistical descriptors for only a subset of variables or using fewer statistical moments than would be required for complete data analysis. This partial processing approach provides sufficient information for authentication while significantly reducing computation time

Inventive Principle:

Principle #16Partial or excessive action

3Measurement precision

If individual cell values are compared between datasets, then accurate matching can be achieved, but computational resources become excessive

Engineering Contradiction:

Improvedata matching accuracyVSAvoidcomputational resources

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent extracts only the most significant statistical features (mean, standard deviation, skewness, kurtosis) from each variable in the dataset, rather than processing all individual data points. This extraction of essential characteristics reduces the data dimensionality while preserving the unique fingerprint of the dataset, enabling efficient comparison without full dataset processing

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent segments the dataset into individual variables and computes statistical descriptors for each variable separately. This segmentation allows the system to process variables independently and compare their statistical profiles, reducing the overall computational burden while maintaining matching accuracy through the cumulative effect of multiple variable comparisons

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11157657B2Mixed data fingerprinting with principal components analysis

Publication Date: 2021.10.26 LIVERAMP

AI summary

Principal components analysis is applied to data sets to fingerprint the dataset or to compare the dataset to a “wild file” that may have been constructed from data found in the dataset. Principal components analysis allows for the reduction of data used for comparison down to a parsimonious compressed signature of a dataset. Datasets with different patterns among the variables will have different patterns of principal components. The principal components of variables (or a relevant subset thereof) in a wild file may be computed and statistically compared to the principal components of identical variables in a data provider's reference file to provide a score. This constitutes a unique and compressed signature of a file that can be used for identification and comparison with similarly defined patterns from other files.