IDPS Malicious Flow Capture and Annotation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed intrusion detection and prevention systems (IDPS) lack the capability to store and tag malicious packet payloads with additional metadata, making it difficult for users to investigate alerts, especially in prevent mode where traffic can be dropped due to signature matches, and end-users have no way to store and tag the malicious packet payload with additional metadata for further debugging.
Innovation Solution
An IDPS engine operating in a software-defined datacenter captures packet flows in PCAP files, annotates them with contextual data including metadata, and sends them to a network management server for further analysis, allowing for the identification of malicious packets and the signatures triggered, enabling better investigation and debugging.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the IDPS system drops packets in prevent mode upon signature match, then intrusion prevention capability is improved, but the ability to investigate and debug alerts deteriorates due to loss of malicious packet payload
Solution Approach 1:
The system performs preliminary action by capturing and storing the malicious packet payload in a PCAP file before the prevent mode drops the packet. The IDPS engine captures packets belonging to the packet flow and stores them in a PCAP file, ensuring the payload is preserved for later investigation even when the packet itself is dropped for security reasons.
Solution Approach 2:
The system creates a copy of the malicious packet payload and stores it in a PCAP file with contextual metadata. Instead of relying on the original packet which gets dropped, the IDPS engine generates and stores a copy containing the payload and relevant flow information, enabling subsequent analysis without compromising security.
2Measurement precision
If the IDPS system generates alerts with signature match information, then detection accuracy is improved, but the ease of investigation deteriorates due to insufficient alert data
Solution Approach 1:
The system merges multiple pieces of information into a single PCAP file annotation: the malicious packet payload, flow contextual data (source/destination IPs, ports, protocol), signature match details, and packet timing information. This consolidation provides investigators with comprehensive data in one location, improving ease of investigation while maintaining detection accuracy.
Solution Approach 2:
The PCAP file with annotations serves as an intermediary between the alert generation and investigation processes. It bridges the gap by containing both the precise signature match information needed for accurate detection and the contextual payload data needed for easy investigation, eliminating the need for separate data retrieval steps.
3Ease of operation
If the IDPS system stores PCAP files for all packet flows, then the ability to investigate alerts is improved, but the device complexity and storage requirements worsen
Solution Approach 1:
The system extracts only the relevant PCAP files for storage and transmission. Instead of storing PCAP files for all packet flows, the IDPS engine identifies and extracts only those PCAP files associated with alerts that have been sent to the management plane, reducing storage complexity while maintaining full investigation capability for alerted flows.
Solution Approach 2:
The system discards PCAP files for packet flows that do not generate alerts, and recovers (retains) only the PCAP files associated with alerted flows. This selective retention strategy reduces overall storage requirements and simplifies management while ensuring all necessary data for alert investigation is preserved.
Data Source
AI summary
Some embodiments of the invention provide, for an intrusion detection and prevention system (IDPS) engine operating on a host computer deployed in a software-defined datacenter (SDDC), a method for detecting and analyzing malicious packet flows. Upon detecting a new packet flow, the method captures packets belonging to the new packet flow in a file. When the new packet flow ends, the method determines that a particular packet belonging to the new packet flow has triggered an alert indicating the particular packet includes a potentially malicious payload. The method annotates the file for the new packet flow with a set of contextual data that (1) specifies the new packet flow as a potentially malicious packet flow and (2) identifies the particular packet and at least one signature associated with the alert triggered by the particular packet. The method sends the annotated file to a network management server to analyze the set of contextual data to extract further information regarding the potentially malicious payload.