Parallel Data Protection Scanning for PII Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data protection systems face challenges in rapidly identifying and securing large volumes of personal identifiable information (PII) from vast datasets, requiring significant processing power and time.

Innovation Solution

A data protection system employing parallel job-submission and job-execution techniques across multiple tiers of computing nodes, utilizing scanning-job-submission services to scan files in parallel, determine sample sizes, and flag target data, while also providing incremental processing and dynamic resource allocation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If sequential scanning of files is used to detect target data, then processing simplicity is maintained, but processing time and productivity are significantly increased

Engineering Contradiction:

Improvescanning speedVSAvoidprocessing architecture

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent divides the file scanning task into multiple independent segments by creating multiple scanning-job-submission services that each handle specific subsets of files. This segmentation enables parallel processing across multiple services, significantly improving scanning speed while maintaining manageable complexity through standardized service templates

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a hierarchical dimension to the processing architecture with multiple tiers: control loop generating scanning-job-submission services, services submitting scanning jobs to job scheduler, and worker nodes executing scans. This dimensional expansion enables parallelism at multiple levels, transforming a single-point processing bottleneck into a distributed processing system

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Measurement precision

If complete file scanning is performed to ensure detection accuracy, then measurement precision is improved, but processing time and energy consumption increase

Engineering Contradiction:

Improvetarget data detection accuracyVSAvoidscanning time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies partial action by having scanning jobs examine only specific portions or samples of files rather than always processing complete files. The sampling strategy determines sample size based on file characteristics and risk assessment, achieving sufficient detection accuracy for compliance while reducing unnecessary processing time and energy consumption on low-risk files

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If parallel scanning services are deployed to improve processing speed, then productivity increases, but system complexity and resource requirements increase

Engineering Contradiction:

Improvedata processing throughputVSAvoidsystem architecture

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent creates universal scanning-job-submission services that can be instantiated multiple times with the same standardized template. Each service performs identical functions (retrieving files, submitting scanning jobs, collecting results) but operates on different data subsets. This universality simplifies system management despite parallel deployment, as all services follow the same proven pattern

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent introduces a job scheduler as an intermediary component between scanning-job-submission services and worker nodes. This mediator handles job distribution, resource allocation, and coordination, abstracting the complexity of managing parallel services and enabling scalable deployment without requiring direct complex interactions between all components

Inventive Principle:

Principle #24Intermediary (Mediator)

4Use of energy by moving object

If dynamic resource allocation is implemented to optimize processing efficiency, then energy consumption is reduced, but control complexity increases

Engineering Contradiction:

Improveprocessing energy consumptionVSAvoidresource management

Core Design Contradiction:

Use of energy by moving objectVSDevice complexity

Solution Approach 1:

The patent implements feedback mechanisms where scanning-job-submission services monitor their execution status, success rates, and resource consumption. This feedback enables dynamic adjustment of service instantiation, job submission timing, and resource allocation based on actual performance data, optimizing energy consumption while keeping control logic manageable through automated decision-making

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS11741252B1Parallel and incremental processing techniques for data protection

Publication Date: 2023.08.29 SAS INSTITUTE INC

AI summary

A data protection system is provided to detect data and execute security actions on the detected data using multiple tiers of parallel processing and incremental processing. For example, the data protection system can employ parallel job-submission and parallel-job execution to cataloging, scanning, searching, and other processes. Only source data that has not already been processed or has modified may be loaded to a cataloging data queue and a scanning data queue to reduce processing time. Scan results can include different data groups and can be used to search for specific data sets.