Security Rule Matching on Structurally Deduplicated Network Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing security management systems face challenges in efficiently detecting cyber threats due to high processing requirements for rule coverage, leading to reduced throughput and increased storage and bandwidth needs when handling network data, especially when traditional compression or deduplication methods fail to account for data structure.
Innovation Solution
A security service platform that structurally deduplicates network data based on a data model, allowing for faster and more efficient rule matching by using structurally deduplicated event data, which reduces redundancy and enables larger rule sets without impacting network sensor performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional compression or deduplication methods are used on network data, then storage and bandwidth needs are reduced, but processing efficiency remains slow and rule matching performance is not improved
Solution Approach 1:
The patent changes the parameter of data representation from traditional flat compression to a hierarchical structure-based format. By organizing network data into structured hierarchies (e.g., packets → flows → connections → events) and applying compression at each level, the system achieves both reduced storage/bandwidth and improved processing efficiency. The hierarchical structure enables selective decompression and faster rule matching on compressed data without full decompression.
Solution Approach 2:
The patent segments network data into hierarchical components (packets, flows, connections, events) and applies deduplication and compression at each segment level. This segmentation allows the system to process only relevant portions of data for rule matching, improving productivity while reducing overall storage and bandwidth requirements through multi-level optimization.
2Reliability
If larger rule sets are used to improve threat detection coverage, then security analysis capability is enhanced, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary organization of network data into hierarchical structures and pre-computation of relevant features before rule matching. By preparing data in advance with proper structuring and indexing, the system can efficiently apply larger rule sets without proportionally increasing processing time. The hierarchical structure enables quick navigation and selective evaluation of rules.
Solution Approach 2:
The patent creates compact hierarchical representations (copies) of network data that preserve essential structures for rule matching while reducing size. These structured copies enable efficient rule evaluation by maintaining the necessary data organization without requiring full original data, thus supporting larger rule sets with minimal time penalty.
3Loss of information
If network data is fully decompressed for analysis, then complete data availability is achieved, but processing speed and computational load increase significantly
Solution Approach 1:
The patent applies local quality by maintaining different levels of data compression and detail for different portions of the network data based on their analytical needs. Critical fields and structures are kept in optimized formats for fast access, while less critical data remains compressed. This selective approach ensures data availability where needed while minimizing overall computational resources required.
Solution Approach 2:
The patent applies partial decompression or expansion only to the specific portions of hierarchical data structures that are relevant for current analysis, rather than fully decompressing all data. This partial action approach maintains data availability for necessary fields while avoiding the computational overhead of processing entire data sets at full expansion.
Data Source
AI summary
Various embodiments include systems and methods pertaining to a security service platform that detects security threats based on a security service that operates on structurally deduplicated network data. The security service performs a security analysis that includes rule matching to detect threats to a network, where the rule matching operates on the structurally deduplicated data. The security service may compile one or more rulesets into an executable binary that efficiently operates over the format of the structurally deduplicated data.