Smart Sampling of Cloud Storage for Sensitive Data Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Scanning cloud resources for sensitive data is costly in terms of time and computing resources, and customers may decline full scans due to invasiveness and cost, leading to compromised security.

Innovation Solution

Implement smart sampling by clustering cloud storage resources based on metadata, selecting a subset for full scanning, and using probabilistic methods to ensure accurate detection of sensitive data while reducing resource consumption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If full scan of cloud resources is performed, then detection accuracy of sensitive data is improved, but computing resource consumption and time cost increase significantly

Engineering Contradiction:

Improvedetection accuracyVSAvoidcomputing resource consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent segments cloud storage resources into multiple clusters based on metadata characteristics (such as file type, size, location, access patterns). Instead of scanning all resources uniformly, the system selects representative samples from each cluster for scanning. This segmentation allows the system to maintain detection accuracy across diverse data types while significantly reducing overall scanning scope and resource consumption.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies different scanning strategies to different clusters based on their local characteristics.Clusters with higher sensitivity or higher probability of containing sensitive data are scanned more extensively, while clusters with lower risk are scanned less. This local quality approach optimizes resource allocation by focusing scanning efforts where they are most needed, rather than applying uniform scanning across all resources.

Inventive Principle:

Principle #3Local quality

2Measurement precision

If full scan of cloud resources is performed, then detection accuracy of sensitive data is improved, but scanning time increases significantly

Engineering Contradiction:

Improvedetection accuracyVSAvoidscanning time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent divides the entire cloud storage resource set into multiple clusters based on metadata characteristics. By scanning representative samples from each cluster rather than all resources sequentially, the system maintains comprehensive detection coverage while dramatically reducing total scanning time. The clustering enables parallel processing of multiple clusters simultaneously.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs partial scanning by selecting a subset of resources from each cluster for full scanning, while other resources in the cluster are inferred or sampled at lower intensity. This partial action approach provides sufficient detection accuracy for most purposes while reducing scanning time, accepting that some resources may be scanned less thoroughly but maintaining overall security posture.

Inventive Principle:

Principle #16Partial or excessive action

3Use of energy by moving object

If sampling method is used to reduce scanning scope, then computing resource consumption is reduced, but detection accuracy may deteriorate

Engineering Contradiction:

Improvecomputing resource consumptionVSAvoiddetection accuracy

Core Design Contradiction:

Use of energy by moving objectVSMeasurement precision

Solution Approach 1:

The patent segments resources into clusters and performs sampling within each cluster rather than random sampling across all resources. This ensures that sensitive data in any cluster has a non-zero probability of being detected, as each cluster is represented in the sample. The segmentation approach maintains detection accuracy while enabling resource-efficient sampling.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent employs feedback mechanisms where scanning results from sampled resources are used to refine future sampling strategies. By analyzing detection outcomes from sampled clusters, the system can adjust sampling probabilities and cluster representations to better identify sensitive data patterns, thereby maintaining or improving detection accuracy over time despite using sampling rather than full scanning.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS12423468B2Smart sampling for sensitive data detection

Publication Date: 2025.09.23 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

A computer implemented method includes obtaining a list of information storage resources that contain electronic files and forming multiple clusters of the information storage resources from the list of information storage resources. A subset of the information storage resources is selected from the clusters. Each subset the information storage resources is scanned for a selected type of content. The selected information storage resources are flagged as containing the selected type of content as a function of the scanning finding the selected type of content.