Superimposing Network and Security Events for Faster Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current security and network event management systems face inefficiencies in analyzing large datasets due to the need for joining datasets, which increases size and time complexity, and are limited to analyzing only alerts and system log messages, failing to merge different data sources effectively.
Innovation Solution
The method involves collecting and superimposing events sharing a derived key into a single record before storage, creating an aggregated summary structure, and applying tags independently, allowing for faster and more comprehensive analysis of security and network events by bypassing the need for relational joins and enabling the merger of diverse data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If datasets are joined using relational database operations, then data correlation and analysis can be performed, but the size and time complexity of the analysis increases significantly
Solution Approach 1:
The patent segments the monolithic relational dataset into multiple hierarchical levels: summary data (aggregated statistics), detail data (individual events), and intermediate data (partial aggregations). This segmentation allows queries to operate on smaller, more manageable data subsets rather than joining entire large datasets, reducing time complexity while maintaining correlation accuracy.
Solution Approach 2:
The patent performs preliminary aggregation and indexing of data into summary structures before actual analysis queries are executed. By pre-computing aggregated statistics and organizing data into hierarchical summaries at multiple levels, the system prepares data in advance to avoid expensive runtime joins, thereby reducing analysis time complexity while preserving data correlation capabilities.
2Reliability
If traditional SIEM systems store and join security events and network flow data separately, then data integrity is maintained, but the ability to merge diverse data sources is limited
Solution Approach 1:
The patent implements a universal hierarchical data model that can accommodate multiple data sources (security events, network flows, logs, etc.) through a common summary data structure. This universal model uses standardized fields and aggregation levels that can represent diverse data types uniformly, enabling the system to merge and analyze different data sources together while maintaining their individual integrity through the hierarchical structure.
Solution Approach 2:
The patent introduces summary data structures as intermediary representations between raw detailed events and final analysis results. These summary structures act as mediators that normalize diverse data sources into a common format, preserving data integrity from original sources while enabling versatile merging and correlation across different data types through the intermediate summary layer.
3Reliability
If all detailed event data is stored and analyzed, then complete analysis coverage is achieved, but the time and cost of searches increases
Solution Approach 1:
The patent segments the data into hierarchical levels where summary data contains aggregated statistics and detail data contains individual events. Search operations can be performed on summary data first to quickly identify relevant time periods and patterns, then drill down to detail data only for specific areas of interest, achieving complete analysis coverage while minimizing search time by avoiding full dataset scans.
Solution Approach 2:
The patent implements partial action by allowing queries to operate on summary data without requiring access to all underlying detailed events. By performing analysis on aggregated summary statistics first, the system achieves sufficient analysis coverage for many use cases without the excessive time cost of processing every individual event, and only accesses detailed data when necessary for deeper investigation.
Data Source
AI summary
An integrated network flow and security information management system and method is provided, more particularly, an integrated network flow and security information management system and method which leverages a process of superimposing and cross referencing common events and attributes in order to increase the speed of searches, completeness of searches and size of dataset (flow data). In particular, the process of superimposing may increase the amount of information that can be processed, while accelerating the search, thereby providing the user with more responsive acts of pivoting and scoping leading to a more complete response to network errors and threats.