Rolling Buffer Network Threat Trace Engine
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern networking equipment struggles to keep pace with high-speed network traffic exceeding 40-100 Gbps/sec, making it impractical for network security devices to capture and store all network traffic for tracing purposes, especially in large virtual environments.
Innovation Solution
Implementing a rolling buffer system that tags and stores network packets with expirable trace time values, allowing machine learning algorithms and malware detectors to analyze and extend or modify these values based on host identification, thereby efficiently managing memory by removing uninteresting packets and focusing on sensitive or anomalous traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all network traffic is captured and stored for tracing purposes, then complete forensic detail is achieved, but storage requirements become impractical and network performance degrades at speeds exceeding 40-100 Gbps/sec
Solution Approach 1:
The patent segments network traffic into two categories: suspicious traffic that requires detailed tracing and stored in long-term storage, and normal traffic that is processed more efficiently with minimal storage. This segmentation allows the system to maintain complete forensic detail for relevant packets while avoiding the impractical requirement of storing all network traffic at high speeds
Solution Approach 2:
The system applies different quality levels of packet capture and storage to different types of network traffic. Suspicious packets receive full forensic detail and long-term retention, while normal packets receive reduced processing and minimal storage. This local quality approach ensures complete forensic detail is preserved where needed without the excessive storage requirements of universal capture
2Reliability
If network security devices capture and store all network traffic, then comprehensive threat analysis is enabled, but the devices cannot keep pace with high-speed network traffic exceeding 40-100 Gbps/sec
Solution Approach 1:
The system performs preliminary analysis of network packets using heuristics and machine learning models to identify suspicious traffic before committing resources to full capture and storage. This preliminary action enables reliable threat analysis by focusing resources on potentially malicious packets while maintaining the ability to process high-speed network traffic without bottlenecking
Solution Approach 2:
Instead of applying full packet capture and analysis to all traffic, the system applies partial action by selectively capturing and analyzing only suspicious packets. This approach maintains reliable threat analysis for relevant traffic while allowing the system to keep pace with high-speed network traffic that would be impossible to process if all packets were captured
3Loss of information
If packet capture is implemented in large virtual environments with multitudes of virtualized computers, then complete network monitoring is achieved, but it becomes impractical if not infeasible to capture and store all traffic
Solution Approach 1:
The patent segments the large virtual environment into manageable components using distributed tracing agents deployed across virtualized computers. Each agent independently identifies and reports suspicious packets, enabling complete network monitoring coverage across the entire virtual environment without the impractical complexity of centralized capture of all traffic
Solution Approach 2:
Individual virtualized computers or network segments perform self-service packet analysis using local heuristics and machine learning models to identify suspicious traffic. This self-service approach enables complete monitoring coverage across large virtual environments by distributing the analysis workload, making the system feasible where centralized capture would be impractical
Data Source
AI summary
An approach for high-volume network threat tracing and detection may be implemented by storing network communications received from a plurality of hosts in an initial recording data structure, such as a rolling buffer. Identifiers may be generated for the plurality of hosts associated with the network communications by according to host identity or the behavior of a given host. Extended trace time values may be assigned to a portion of the plurality of hosts based at least in part on the identifiers, and storing the portion of the network communications that have extended trace time values may be recorded as packet capture files in long term memory.