Ultra-light Clustering Intrusion Detection for False Positive Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing intrusion detection systems face challenges in achieving high detection accuracy with limited memory and processing resources, often resulting in a high number of false positives due to signature-based detection methods.
Innovation Solution
The proposed ultra-light clustering-based generative intrusion detection device and method utilize a streaming algorithm that identifies frequent signature groups in data streams using a minhashed virtual-vector (MV2) module and Jaccard-index grouping (JIG) module, generating signatures with a small amount of fixed memory and specific hash operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If signature-based detection methods are used, then detection accuracy is improved, but false positives increase significantly
Solution Approach 1:
The patent segments the data stream into chunks and processes them in batches rather than analyzing individual records. This segmentation allows the system to identify patterns across multiple data points, improving detection accuracy while reducing false positives by considering contextual relationships among segmented data portions.
Solution Approach 2:
The patent introduces an intermediary processing layer that uses batching and pattern matching algorithms to mediate between raw data and detection results. This intermediary layer filters and aggregates data before final detection, reducing false positives while maintaining detection accuracy through intermediate analysis steps.
2Measurement precision
If traditional intrusion detection systems process all data in memory, then detection accuracy is improved, but memory requirements and processing time increase
Solution Approach 1:
The patent divides the data stream into manageable chunks that can be processed in batches with limited memory. Each chunk is processed independently but contributes to overall pattern recognition, enabling accurate detection without requiring the system to load entire datasets into memory simultaneously.
Solution Approach 2:
The patent implements periodic processing of data batches rather than continuous analysis of the entire data stream. This periodic action allows the system to maintain detection accuracy by analyzing representative samples at regular intervals while significantly reducing memory requirements and processing overhead.
3Measurement precision
If manual analysis of security alerts is performed, then detection accuracy is improved, but productivity decreases due to overwhelming volume
Solution Approach 1:
The patent implements self-service automation where the system automatically processes and analyzes security alerts using pattern matching algorithms. The batching mechanism enables the system to autonomously handle large volumes of alerts with consistent accuracy, eliminating the need for manual analysis while maintaining detection quality and improving productivity.
Solution Approach 2:
The patent changes the operational parameters of alert processing by transitioning from individual record analysis to batch-based pattern recognition. This parameter change enables the system to process vastly increased volumes of alerts while maintaining detection accuracy through aggregated pattern analysis rather than individual case examination.
Data Source
AI summary
An ultra-light clustering-based generative intrusion detection device, includes: a data receiver configured to receive a data stream containing a specific type of data; a big-group identification unit configured to identify at least one big-group related to similar data encoded as a virtual vector based on a chunk set for each piece of data of the data stream; and a signature generator configured to extract signatures for each of the at least one big-group and generate a signature group