Sensitive Data Signatures for Fast, Accurate Secret Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing security mechanisms struggle to efficiently and effectively identify, locate, and manage sensitive data within large data streams, determine authorized deployment scope, and facilitate testing without exposing actual sensitive data.
Innovation Solution
Implementing a contiguous sensitive data identification data structure (SDIDS) with predefined primary and secondary adherence signatures, embedded metadata, and correlation identifiers to enhance security functioning by reducing false positives and enabling efficient scanning and governance of sensitive data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional security mechanisms are used to scan large data streams for sensitive data, then the scanning process becomes slow and inefficient, but using more comprehensive detection methods increases false positive rates
Solution Approach 1:
The patent divides the detection task into multiple stages by segmenting the data stream into records and analyzing them in batches. The sensitive data identification data structure (SDIDS) is processed in discrete units, allowing efficient scanning while maintaining accuracy through structured analysis of each segment.
Solution Approach 2:
The patent introduces an intermediary data structure (the SDIDS with adherence signatures) that mediates between the raw data stream and the detection process. This intermediary structure enables efficient pattern matching while filtering out false positives through the hierarchy of adherence signatures.
2Reliability
If comprehensive security checks are implemented to detect all vulnerabilities, then detection coverage improves, but system complexity and cost increase
Solution Approach 1:
The patent segments security checks into distinct layers: primary adherence signature detection, secondary adherence signature verification, and contextual analysis. This segmentation allows comprehensive security coverage while managing complexity through structured, modular checks that can be implemented progressively.
Solution Approach 2:
The patent performs preliminary actions by pre-defining adherence signatures and creating the SDIDS structure before actual data processing. This preliminary setup enables comprehensive security coverage through pre-established detection criteria while reducing runtime complexity through optimized search patterns.
3Speed
If sensitive data is scanned in real-time to prevent exposure, then security responsiveness improves, but computational resources are consumed
Solution Approach 1:
The patent performs preliminary actions by pre-compiling adherence signatures and pre-structuring the SDIDS format before data processing. This allows real-time scanning with minimal computational overhead during actual data flow, as the detection logic is optimized and pre-prepared.
Solution Approach 2:
The patent uses adherence signatures as copies or representations of sensitive data patterns that can be efficiently matched without processing the actual sensitive data. This copying approach enables real-time detection while reducing computational resource consumption by working with simplified signature patterns rather than full data records.
Data Source
AI summary
Some embodiments form a sensitive data identification data structure (SDIDS) which includes an identifiable sensitive data (ISD) portion. Some embodiments scan for an SDIDS, and some do both. The SDIDS is distinguished by at least one of: specified rarity of an adherence signature, absence of a checksum, primary and secondary adherence signatures, non-prefix adherence signature position, non-suffix checksum position, or particular kinds of metadata. Some examples of suitable metadata include timestamp metadata, deployment metadata, origination metadata, ownership metadata, metadata for testing, correlation metadata, and combinations thereof. Some ISD examples include security keys, tokens, passwords, pass phrases, cryptologic artifacts, confidential data, private data, critical data, and data that is tagged or labeled as sensitive.