Cloud Data Classification via Sampling and Truth Tables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing solutions for classifying data in cloud computing environments are cumbersome and ineffective in identifying sensitive data stored in all data stores, especially in large environments with hundreds of thousands or millions of files.
Innovation Solution
A method and system for data classification that involves sampling a dataset into numerical and string samples, applying a truth table to numerical samples, and using machine learning classifiers on string samples to determine classifications, thereby identifying sensitive data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual permission granting is used for each data store, then access control is precise, but the process becomes cumbersome and scalable only to small environments
Solution Approach 1:
The system automatically classifies data by sampling and analyzing data stores without requiring manual permission granting for each one. The classification system independently performs sampling, feature extraction, and classification to identify sensitive data across all data stores in the cloud environment.
Solution Approach 2:
The system performs preliminary sampling of data stores before full classification. By sampling a subset of data and analyzing its characteristics, the system can infer properties of the entire data store and determine sensitivity without manually examining every data store in detail.
2Reliability
If comprehensive data store listing is required, then all data can be analyzed, but technical expertise and time consumption increase significantly
Solution Approach 1:
The system autonomously discovers and analyzes data stores without requiring comprehensive manual listing. It automatically connects to data stores, performs sampling, and classifies data regardless of whether the data store is explicitly listed or documented in the cloud environment.
Solution Approach 2:
The system creates a sampled representation of each data store's data characteristics rather than requiring full enumeration. This sampling copy allows the system to infer sensitivity and classify data without examining every individual record or requiring complete data store inventories.
3Productivity
If existing classification solutions are used, then some data can be classified, but they fail to analyze data in unlisted data stores
Solution Approach 1:
The classification system is designed to universally apply to all data stores in the cloud environment regardless of their listing status. It performs the same sampling and classification operations on both listed and unlisted data stores, ensuring comprehensive coverage without requiring separate handling procedures.
Solution Approach 2:
The system independently discovers and analyzes data stores without relying on pre-provided inventories or explicit permissions for each data store. It autonomously connects to data stores, samples their data, and performs classification, ensuring that both listed and unlisted data stores are analyzed equally.
Data Source
AI summary
Techniques for data classification. A method includes sampling a dataset into first and second samples. Each first sample is a numerical value, and each second sample is a string of characters. A truth table is applied to the first samples from a dataset. The truth table includes multiple first columns, each of which accepts an input value determined for each of the first samples, and a second column which outputs first scores representing likelihoods for respective classifications. Classifiers are applied to features extracted from the second samples, where each classifier is a machine learning model trained to output a second score representing a likelihood for a respective classification for each second sample. Classifications are determined based on the first and second scores. The classifications include a classification for each first sample determined based on the first scores and a classification for each second sample determined based on the second scores.