Deterministic Secret Leak Detection via Exact String Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing secret detection methods, such as pattern matching and tokenization, are prone to false positives and negatives, especially at scale, and are not suitable for identifying secrets in unstructured data or large computing platforms, leading to potential security breaches.
Innovation Solution
An end-to-end system that securely maintains a repository of known secrets to deterministically identify potential secret leaks in digital documents, using distributed string-matching and filtering techniques to accurately detect and respond to secret leaks across various formats and platforms, without relying on predictive models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pattern matching techniques (regular expressions, tokenization) are used to detect secrets, then detection capability is provided, but false positives and false negatives occur especially at scale
Solution Approach 1:
The patent creates a deterministic copy of known secrets in a repository and uses exact string matching against this copy. Instead of attempting to predict or infer secrets through pattern matching, the system copies actual known secret values and searches for exact matches in digital documents, eliminating false positives while maintaining detection accuracy at scale
Solution Approach 2:
The patent replaces the mechanical pattern-matching system (regular expressions, tokenization) with a deterministic string-comparison system. Rather than using complex pattern recognition algorithms that are prone to errors, the system uses straightforward exact string matching against a repository of known secrets, achieving both accuracy and scalability
2Reliability
If prediction-based detection tools using AI are used, then detection capability is provided, but false positives or false negatives occur
Solution Approach 1:
Instead of using complex AI models to predict whether data contains secrets and risking false positives, the patent inverts the approach by maintaining a deterministic repository of known secrets and searching for exact matches. This inversion eliminates the need for complex prediction algorithms while achieving higher reliability
Solution Approach 2:
The patent uses simple, deterministic string comparison operations instead of complex, expensive AI prediction models. The system compares extracted strings against a repository of known secrets using straightforward matching algorithms, avoiding the computational complexity and reliability issues of AI-based approaches
3Reliability
If additional secrets are added to increase detection coverage, then detection capability improves, but time needed for prediction increases
Solution Approach 1:
The patent segments the detection process into two independent phases: (1) extracting candidate strings from digital documents, and (2) comparing these strings against a repository of known secrets. This segmentation allows the system to maintain comprehensive coverage by adding more secrets to the repository without increasing processing time, as the comparison operation remains O(n) where n is the number of candidate strings
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Aspects of the disclosure provide for an end-to-end system for detecting and responding to known secrets leaked on a computing platform 101. By securely retrieving secret data from a repository 235 of known secrets, the platform can detect whether potential secrets are leaked in digital documents generated by applications or services of the platform, and compare the potential secrets with known secrets in the repository. The system deterministically and accurately identifies whether secrets were leaked in a digital document, and can also identify both the source of the leakage and the affected users, devices, or groups of users and devices on the platform.