PII Detection via Word Contribution Ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for determining personally identifiable information (PII) in data processing are limited, as they primarily focus on direct PII and struggle to identify indirect PII, which lacks specific data patterns, leading to incomplete recognition and potential privacy leaks.
Innovation Solution
A data processing method that trains a classification model on reference documents to associate individual words with their contributions to object identification, enabling the automatic generation of identification information datasets without manual labeling, thereby improving the detection of both direct and indirect PII.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional PII detection methods are used, then direct PII can be identified, but indirect PII cannot be effectively detected
Solution Approach 1:
The patent transforms the detection approach from pattern-matching (traditional parameter) to contribution-based ranking (new parameter). By calculating contribution values of word sequences to PII identification and sorting them in descending order, the system can identify both direct PII and indirect PII without relying on predefined patterns, thus resolving the contradiction between detection accuracy and adaptability to indirect PII
Solution Approach 2:
The patent replaces the mechanical pattern-matching system with a machine learning-based contribution calculation system. Instead of using fixed rules to detect PII, the system learns from training data to determine the contribution of each word sequence to PII identification, enabling detection of indirect PII that lacks specific data patterns while maintaining high accuracy
2Manufacturing precision
If manual labeling is used to create PII datasets, then high-quality training data can be obtained, but the process is time-consuming and labor-intensive
Solution Approach 1:
The patent implements self-service by enabling the system to automatically generate and label PII training datasets without human intervention. The contribution-based ranking mechanism automatically identifies and labels PII in training data, eliminating the need for manual labeling while maintaining high data quality, thus resolving the contradiction between dataset quality and time consumption
Solution Approach 2:
The patent performs preliminary action by pre-calculating and storing contribution values of word sequences during the training process. This pre-computed contribution information is then reused for both training and inference, eliminating the need for repeated manual labeling and significantly reducing time loss while maintaining high-quality dataset generation
Data Source
AI summary
A method, an electronic device, and a computer program product for processing data is disclosed. The method includes training a classification model based on a plurality of reference documents describing different objects, the trained classification model respectively associating the plurality of reference documents with the described objects. The method further includes determining from the individual words identification information that can identify the objects based on contributions of individual words in the reference documents to the association. Identification information that can identify objects in documents describing the objects may be determined, so that an identification information data set is automatically generated for training a machine learning model that is used to determine the identification information.