DLP Pattern Matching via Fixed String Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data leak prevention systems face performance issues due to the computational expense of processing regular expressions for pattern matching, particularly with complex data types like Social Security Numbers and Credit Card Numbers, leading to slow transactions and difficulty in parallel processing.
Innovation Solution
A Data Leak Prevention (DLP) system converts input strings into fixed string patterns using class definitions such as digit, letter, and symbol classes, enabling efficient matching against stored target strings using algorithms like deterministic finite automaton algorithms, allowing for parallel processing and detection of potential sensitive data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If regular expressions are used for pattern matching to identify sensitive data, then the ability to detect various data types is improved, but the processing time increases and transaction speed decreases
Solution Approach 1:
The patent segments the pattern matching process into two distinct phases: a fast pre-match phase using simplified patterns to filter obvious matches, and a slower full-match phase using complete regular expressions only for potential candidates. This segmentation reduces the overall processing time by avoiding full regular expression compilation and matching for all input strings.
Solution Approach 2:
The patent performs preliminary pattern matching using simplified patterns before applying full regular expressions. The pre-match phase prepares a filtered list of potential matches, so that computationally expensive full regular expression processing is only applied to a small subset of candidates, significantly reducing total processing time.
2Measurement precision
If multiple regular expressions are used to handle complex data types like Social Security Numbers and Credit Card Numbers, then the detection accuracy is improved, but the computational complexity increases and parallel processing becomes difficult
Solution Approach 1:
The patent divides the matching system into two layers: a simple pre-match layer using basic patterns for quick filtering, and a sophisticated full-match layer using complete regular expressions for accurate detection. This allows the system to maintain high detection accuracy while reducing computational complexity for the majority of processing operations.
Solution Approach 2:
The patent introduces an intermediary pre-match filtering step between the input data and the full regular expression matching. This intermediary layer processes data with simple patterns first, creating a filtered candidate set that is then processed by the complex regular expressions, thereby managing computational complexity while preserving detection accuracy.
3Reliability
If full regular expression matching is applied to all input strings, then the detection completeness is improved, but the transaction speed decreases
Solution Approach 1:
The patent applies preliminary filtering using simplified patterns before performing full regular expression matching. This ensures that no potential matches are missed (maintaining detection completeness) while significantly improving transaction speed by restricting full matching to only those strings that passed the pre-match filter.
Solution Approach 2:
The patent maintains continuous detection coverage through a two-stage process where the pre-match phase continuously filters inputs and the full-match phase continuously processes candidates. This continuous action ensures no detections are missed while optimizing throughput by keeping the expensive full-matching operation running at full capacity on a reduced input stream.
Data Source
AI summary
Systems and methods for preprocessing data to facilitate DLP pattern matching are provided. An input string is received by a Data Leak Prevention (DLP) system. The input string is converted by the DLP system into a fixed string pattern. The conversion is performed based on one or more of multiple class definitions, including a digit class, a letter class and a symbol class. A determination is then made by the DLP system regarding whether the input string contains potential sensitive data to which a full-match is to be applied by matching the fixed string pattern against one or more stored target strings representative of sensitive data.