Confidential Information Detection via Modular Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing environments face challenges in detecting confidential information without human intervention, which can lead to security and privacy breaches and liabilities, as existing systems are inefficient in automating the detection of sensitive data across large datasets.
Innovation Solution
A computer-implemented system that automatically detects confidential information by analyzing electronically stored data using a runtime environment with modules for credit card, social security, telephone, name, street address, and zip code detection, assigning scores, and flagging files likely to contain such information, thereby reducing human review and enhancing security measures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated detection systems are implemented to search for confidential information, then productivity and efficiency are improved, but device complexity increases
Solution Approach 1:
The detection system is divided into multiple specialized modules, each responsible for detecting specific types of confidential information (credit card numbers, social security numbers, telephone numbers, names, addresses, zip codes). This segmentation allows the complex detection task to be distributed across independent, manageable components, improving productivity while controlling complexity through modular design.
Solution Approach 2:
The system employs a universal detection framework that handles multiple types of confidential information through a common architecture. The runtime environment and file system interface provide universal access mechanisms that work across different data types and storage locations, reducing overall system complexity while maintaining high detection productivity.
2Measurement precision
If comprehensive detection rules are applied to all file types, then measurement precision is improved, but processing time increases
Solution Approach 1:
The system applies different detection rules and strategies tailored to specific file types and confidential information categories. Each module uses locally optimized detection logic appropriate to its specific data type (e.g., pattern matching for credit card numbers, geographic validation for addresses), achieving high detection precision without requiring uniform complex processing across all files.
Solution Approach 2:
The system implements detection rules that go beyond basic pattern matching by incorporating validation logic specific to each confidential information type. For example, credit card detection includes Luhn algorithm validation, and address detection includes geographic region verification. This partial excessive action ensures high detection accuracy while maintaining reasonable processing times through targeted validation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Detecting confidential information includes reading stored data and identifying strings within the stored data (210), where each string includes a sequence of consecutive bytes which all have values that are in a predetermined subset of possible values. For each of at least some of the strings, determining if the string includes bytes representing one or more format matches (220 - 270), wherein a format match includes a set of values that match a predetermined format associated with confidential information. For each format match, testing the values that match the predetermined format with a set of rules associated with the confidential information to determine whether the format match is an invalid format match that includes one or more invalid values and calculating a score for the stored data (280, 300), based at least in part upon the ratio of a count of invalid format matches to a count of other format matches.