Compact Repository Fingerprinting for Cloud DLP
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data loss prevention systems face challenges in efficiently managing large datasets in distributed networks, particularly in cloud services, due to high false positive rates and the inability to provide metadata for matched events, leading to inefficiencies in memory and communication resources.
Innovation Solution
A compact repository mechanism using a compressed bitmap and Bloom filters with a Roaring bitmap implementation, allowing for efficient storage and distribution of fingerprint data, reducing false positives and enabling metadata-driven event identification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional fingerprinting methods are used to monitor large amounts of data in enterprise networks, then robust identification capability is achieved, but significant memory resources are required
Solution Approach 1:
The patent extracts only the essential fingerprint information from the original data by using Bloom filters to create compact bitmaps that represent file fingerprints. This extraction process retains the identification capability while dramatically reducing the storage requirements from storing complete fingerprint sets to storing only the Bloom filter bitmaps.
Solution Approach 2:
The patent creates a compressed copy of the fingerprint data using Bloom filters, which are probabilistic data structures that use bit arrays to represent sets of fingerprints. This copy maintains the ability to identify files while using significantly less memory than the original fingerprint data would require.
2Quantity of substance
If compact repository solutions are used to reduce memory resource allocation, then memory efficiency is improved, but false positive rates increase
Solution Approach 1:
The patent changes the parameters of the Bloom filter by using multiple hash functions and adjusting the bitmap size to optimize the false positive rate. By carefully selecting the number of hash functions and bitmap dimensions, the system achieves low false positive rates while maintaining memory efficiency.
3Productivity
If compact repository files are distributed to multiple nodes for faster analysis, then analysis efficiency is improved, but communication and update complexity increases
Solution Approach 1:
The patent segments the fingerprint repository into multiple distributed copies that can be shared across network nodes. Each node maintains its own copy of the Bloom filter bitmaps, allowing parallel analysis while sharing the same compact data structure that simplifies synchronization.
Data Source
AI summary
A data loss prevention mechanism for a cloud-based security analytics system is provided that utilizes a compact repository that improves the ratio of false positives over traditional methods, maintains a small data size, permits distribution of compact repository files to a large number of analyzing nodes, and provides metadata for matched events. A compressed bitmap of information found in a compact repository is used, thereby improving the utilization of storage space for a repository containing information associated with a significant number of data fingerprints. Compression further allows for a larger entry size in the compact repository, thereby providing a lower false positive rate. A mechanism for distributing updates to compact repositories residing on cloud servers is provided by updating a central server and propagating the updates to remote servers. Identification of secured data associated with unstructured data fingerprints is also handled using distributed reference to a centralized fingerprint repository.