Threat Timeline Visualization with Progressive Lineage Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems lack effective methods for visualizing threat development in enterprise networks, necessitating improved tools for detection, investigation, and disposal of potential threats.
Innovation Solution
A threat management system that creates a lineage of security events on compute instances, including process identifiers and timestamps, which is transmitted to a threat management facility for immediate visualization and augmented with additional data from a data lake.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a lineage is created immediately with basic process information, then the threat timeline visualization can be displayed quickly, but the visualization lacks comprehensive data from long-term storage
Solution Approach 1:
The computeinstance creates and transmits a lineage with basic process information immediately upon detecting a security event, enabling quick visualization. The time stamp in the lineage identifier allows the threat management facility to later retrieve additional data from the data lake without requiring complete data to be available upfront.
Solution Approach 2:
The lineage acts as an intermediary structure that bridges immediate visualization needs and comprehensive data retrieval. It contains sufficient information (process identifiers, time stamps) to display basic threat timeline information while also enabling subsequent enrichment from the data lake through the time stamp lookup mechanism.
2Loss of information
If additional data is retrieved from the data lake to augment the visualization, then data completeness improves, but the system complexity increases
Solution Approach 1:
The system segments data retrieval operations into two distinct phases: immediate retrieval of basic lineage information for quick visualization, and subsequent enrichment retrieval from the data lake using time stamp lookups. This segmentation allows the system to maintain simplicity in the immediate response while enabling comprehensive data gathering when needed.
Solution Approach 2:
The system uses parameter changes in the lineage identifier (time stamp) to enable efficient data retrieval from the data lake. By storing the time stamp in the lineage, the system can dynamically query the data lake for additional information without increasing overall system complexity, as the time stamp serves as an effective retrieval parameter.
3Loss of information
If the lineage includes comprehensive process information immediately, then data completeness improves, but the transmission and processing time increases
Solution Approach 1:
The system extracts only the essential information needed for immediate visualization (process identifiers, time stamps) and transmits this minimized lineage data quickly. Additional comprehensive data is extracted from the data lake later using the time stamp, separating the immediate transmission requirement from the comprehensive data gathering requirement.
Data Source
AI summary
Security events are reported to a threat management facility for an enterprise network as self-contained lineages that include data concerning related processes such as a parent or child process related to the source of the event. By transmitting these to a short term data store, threat timeline visualizations can be more quickly rendered for an analyst in a user interface, after which the visualization can be augmented with other data from other sources such as a data lake or other long term data repository for the enterprise network, third party reputation sources, and so forth.