Sensitive Data Detection Using Aho-Corasick Algorithm
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current sensitive data detection systems, such as PRESIDIO, face inefficiencies due to their time complexity being proportional to the number of recognizers used, leading to slower performance when detecting multiple types of sensitive data, especially when hundreds of types are involved.
Innovation Solution
The implementation of an efficient text searching algorithm like the Aho-Corasick algorithm, combined with a detection pipeline that includes a keyword matcher and a pattern matcher, reduces detection time complexity to be independent of the number of sensitive data types, improving scalability and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple recognizers are used to detect different types of sensitive data, then the detection coverage is improved, but the detection time increases proportionally with the number of recognizers
Solution Approach 1:
The patent merges multiple recognizers into a single integrated system using the Aho-Corasick algorithm. Instead of invoking hundreds of individual recognizers sequentially, the system combines all recognition patterns into one automated machine that processes text in a single pass, reducing time complexity from O(n*m) to O(n+m) where n is text length and m is number of patterns.
Solution Approach 2:
The Aho-Corasick algorithm serves as a universal detector that can handle multiple types of sensitive data recognition simultaneously. The single algorithmic framework is designed to detect various patterns (credit card numbers, social security numbers, addresses, etc.) in one unified process, eliminating the need for separate specialized recognizers for each data type.
2Adaptability or versatility
If a large number of recognizers are deployed to detect hundreds of sensitive data types, then the system's versatility is improved, but the processing speed deteriorates
Solution Approach 1:
The system combines all recognition patterns into a single Aho-Corasick automaton that processes text in one pass. This merging of hundreds of individual recognizer functions into one unified algorithm maintains the ability to detect all sensitive data types while achieving linear time complexity relative to input size, thereby preserving processing speed despite increased versatility.
Solution Approach 2:
The patent changes the fundamental parameter of how recognition is performed - from sequential invocation of individual recognizers to simultaneous pattern matching using the Aho-Corasick algorithm. This parameter change transforms the time complexity from proportional to the number of recognizers to independent of the number of sensitive data types, enabling high versatility without sacrificing processing speed.
Data Source
AI summary
The disclosed systems and methods are directed to detecting sensitive data on a computing device. This includes matching predetermined keywords in input data, to determine data in vicinities of matched keywords in the input data in which sensitive data is likely to be found, and matching predefined patterns associated with sensitive data to the data in vicinities of matched keywords to detect sensitive data. Matching the predetermined keywords occurs prior to matching the predefined patterns, and the data in vicinities of matched keywords is substantially shorter than the input data.