Sensitive Data Estimation Using NLP Features and Pretrained Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for identifying sensitive data in large datasets stored in cloud storage are time-consuming, costly, and resource-intensive, requiring full data scanning.
Innovation Solution
A data sensitivity estimation model, such as a heuristic or machine learning model, is trained using natural language processing to predict sensitive data based on features like naming conventions, access policies, and organizational properties, allowing for rapid and accurate identification of sensitive data without full data access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional full data scanning methods are used to identify sensitive data, then measurement precision is improved, but productivity deteriorates and loss of time increases
Solution Approach 1:
The patent segments the data identification process into two distinct phases: (1) training phase where the system learns from manually labeled sensitive data to build a machine learning model, and (2) inference phase where the trained model automatically identifies sensitive data. This segmentation allows the system to achieve high accuracy through learning while maintaining high productivity during automatic identification, resolving the contradiction between measurement precision and productivity.
2Measurement precision
If traditional full data scanning methods are used to identify sensitive data, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The patent applies preliminary action by performing manual labeling and model training in advance during the training phase. The machine learning model is pre-trained on a dataset with ground truth labels before deployment. This preliminary action creates a ready-to-use model that can rapidly identify sensitive data in the inference phase without requiring time-consuming manual scanning, thus reducing loss of time while maintaining measurement precision.
3Productivity
If machine learning models are used for data sensitivity estimation, then productivity is improved, but device complexity increases
Solution Approach 1:
The patent introduces a machine learning model as an intermediary between the raw data and the sensitivity identification process. The model acts as a mediator that has been pre-trained on labeled data and can now automatically perform sensitivity classification. This intermediary approach simplifies the operational complexity during inference while achieving high productivity, as the complex learning process is confined to the training phase rather than requiring complex real-time processing infrastructure.
Data Source
AI summary
The disclosed technology is generally directed to data classification. In one example of the technology, training data and a ground truth that indicates sensitive data within the training data is received. Based at least on the training data, natural language processing is used to learn features. The features include a naming feature that is associated with names of data resources in the training data. Based at least on the training data and the ground truth, using supervised learning, a model that is a heuristic model and/or a machine learning model is created. Input data information that is associated with input data is received. The model is used to determine a data resource sensitivity estimator (DRSE) value for each portion of the input data. The determination is based on the combination of features for the input data. Potentially sensitive data within the input data is flagged based on the DRSE values.


