Network Traffic Capture System with Concurrent Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network systems face challenges in efficiently processing and managing increasing volumes of network traffic data, particularly in capturing, storing, and analyzing traffic for security and performance, with existing solutions failing to effectively address the need for cost reduction, improved efficiency, and timely management amidst growing internet demands.
Innovation Solution
A network traffic system that gathers and filters network packets, compresses capture files concurrently, and uses search values to display relevant data, incorporating features like independent port capture and filtering, and integrated storage configurations to enhance data handling and analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If network traffic data is captured and stored in detail for security analysis, then measurement precision and reliability improve, but storage volume and device complexity increase significantly
Solution Approach 1:
The system segments network traffic into multiple capture files organized in rings, where each ring contains capture files from a specific time period. This segmentation allows selective storage and retrieval of traffic data without maintaining all data indefinitely, reducing overall storage volume while preserving analysis precision for relevant time periods.
Solution Approach 2:
The system changes the parameter of data retention by implementing an expiration mechanism where capture files are marked for deletion after a specified time period. This parameter change allows the system to maintain high measurement precision for recent traffic while automatically reducing storage volume as older data expires and is removed.
2Reliability
If all network packets are captured and stored for comprehensive analysis, then reliability improves, but loss of time in processing and searching increases
Solution Approach 1:
The system segments captured network traffic into multiple discrete capture files organized in rings, allowing the search function to target specific files or rings rather than scanning all stored traffic. This segmentation reduces the time loss in searching while maintaining reliability by preserving all necessary traffic data in an organized structure.
Solution Approach 2:
The system performs preliminary organization of capture files into rings with expiration markers before search operations occur. This preliminary structuring allows the search function to efficiently locate and retrieve relevant data without processing unnecessary older files, reducing time loss while maintaining comprehensive coverage for reliability.
3Productivity
If network traffic is captured at high speed for real-time analysis, then productivity improves, but device complexity and processing requirements increase
Solution Approach 1:
The system segments incoming network traffic into discrete capture files that are written to storage in parallel operations. This segmentation allows the capture process to maintain high productivity by distributing the writing load across multiple files while the complexity of managing file creation, naming, and organization is handled by the ring structure that automatically manages file lifecycle.
Data Source
AI summary
A system and method of operation of a network traffic system includes: a first portion of network traffic gathered from an external network; a first network filter for selecting a first network packet from the first portion of the network traffic; a first capture file for storing the first network packet; a second network packet selected from a second network filter; a second capture file for storing the second network packet concurrently with the compression of the first capture file; and a display device for displaying the first compressed capture file using a search value.