Si Data Scanning Process Using Reference Subcombinations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data scrubbing technologies rely on inefficient pattern matching algorithms, leading to high processor loads, false positives, and inefficiencies in processing large data stores, especially when dealing with unstructured and sensitive information.
Innovation Solution
A system and method utilizing a reference table with a grid of data cells to generate reference subcombinations, which are used to analyze subject files for sensitive information, reducing processor load and false positives by identifying matches within data entries containing all reference data elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pattern matching algorithms are used to detect sensitive information, then detection capability is provided, but processor load increases and processing speed decreases
Solution Approach 1:
The patent segments the reference data into multiple columns within a grid structure, where each column represents a specific data element (e.g., first name, last name, date of birth). This segmentation allows the system to process and compare individual elements separately, reducing the computational complexity of pattern matching while maintaining detection accuracy.
Solution Approach 2:
The patent performs preliminary organization of reference data into a structured grid format with predefined columns and rows before the actual scanning process. This preliminary structuring enables faster lookup and comparison during scanning, as the system doesn't need to parse unstructured data during the detection phase, thereby improving processing speed.
2Reliability
If pattern matching algorithms are used to detect sensitive information, then detection capability is provided, but false positive rates increase
Solution Approach 1:
By segmenting the matching process into multiple column-based comparisons (first name, last name, date of birth, etc.), the system can verify multiple data elements simultaneously. This reduces false positives because a match requires consistency across multiple segmented fields rather than relying on a single pattern match.
Solution Approach 2:
The patent applies different matching criteria and data types to specific columns (e.g., alphabetic data for names, numeric data for dates and IDs). This local quality approach ensures that each data element is validated according to its specific format requirements, improving the precision of detection and reducing false positives.
3Measurement precision
If comprehensive data scanning is performed on large data stores, then detection accuracy is improved, but processor load increases
Solution Approach 1:
The reference data is pre-organized into a grid structure with defined columns and rows before scanning begins. This preliminary organization allows the scanning process to efficiently retrieve and compare only relevant data elements without parsing entire records, reducing processor load while maintaining comprehensive detection coverage.
Solution Approach 2:
The scanning process is segmented into column-based operations where each column is processed independently. This segmentation allows the system to handle large data stores more efficiently by processing data elements in manageable units rather than analyzing entire records at once, thereby reducing overall processor load.
Data Source
AI summary
A computer-implemented method for scanning data including accessing a reference table including a grid of data cells arranged in columns and rows and containing reference data elements. Each of the rows may relate to one of a plurality of data subjects. The method may also include generating a list of reference subcombinations. The list of reference subcombinations may be generated by designating a primary column for uniquely identifying the data subjects and generating a plurality of first preliminary reference subcombinations. Each of the first preliminary reference subcombinations may include reference data elements gathered from the primary column and a second column of a single row of the reference table. The method may also include accessing a subject file having a plurality of data entries. Each of the data entries may include a plurality of logically-related and delimited subject data elements. The method may also include identifying and generating a match record for each instance in which one of the data entries contains all reference data elements of one of the reference subcombinations.