Network Traffic Data Collection via PCAP Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network traffic data collected by DNS name servers is geographically isolated and not easily accessible for analysis, leading to inefficiencies in processing and resource degradation when handling analysis requests, and requiring new software updates to accommodate changing analysis requirements.
Innovation Solution
A system and method for collecting and storing network traffic data by extracting data from PCAP files at edge sites and transmitting it as header/digest pairs to a central repository, allowing for centralized analysis and reducing the burden on DNS name servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If PCAP files are stored at edge sites for network traffic recording, then network traffic data can be collected and stored locally, but the data becomes geographically isolated and not easily accessible for analysis
Solution Approach 1:
The patent extracts data from PCAP files at edge sites and transfers only the relevant information to a central repository. This extraction process separates the bulk storage function (at edge sites) from the analysis function (at central repository), resolving the contradiction by making data accessible for analysis while maintaining local recording capabilities.
Solution Approach 2:
The patent introduces a central repository as an intermediary between edge sites and analysis systems. This intermediary receives extracted data from edge sites and provides centralized access for analysis, eliminating the geographical isolation barrier while preserving the distributed recording architecture.
2Productivity
If DNS name servers process both DNS requests and analysis requests, then resource utilization increases, but processing speed decreases due to resource degradation
Solution Approach 1:
The patent extracts analysis processing functions from DNS name servers by creating a separate central repository for analysis requests. This separation allows DNS servers to focus solely on DNS request processing while the central repository handles analysis tasks, eliminating resource degradation and maintaining processing speed.
Solution Approach 2:
The patent segments the network traffic processing system into distinct functional components: edge sites for data collection, central repository for analysis, and DNS servers for name resolution. This segmentation isolates different processing functions to prevent resource conflicts and maintain optimal performance for each function.
3Adaptability or versatility
If new software updates are implemented at edge sites to accommodate changing analysis requirements, then analysis capability improves, but system complexity and maintenance burden increase
Solution Approach 1:
The patent positions the central repository as an intermediary that receives data from edge sites and performs analysis according to changing requirements. This intermediary architecture allows analysis capabilities to be updated centrally without requiring software updates at distributed edge sites, reducing system complexity while maintaining adaptability.
Solution Approach 2:
The patent creates a universal central repository that can handle various analysis requirements through a single platform. This multi-functional system accommodates changing analysis needs without requiring specialized software updates at each edge site, as the central repository can adapt its processing capabilities universally.
4Duration of action of stationary object
If PCAP files are kept at edge sites for future analysis, then data retention is maintained, but data processing efficiency decreases due to large file sizes and lack of indexing
Solution Approach 1:
The patent extracts relevant data from PCAP files at edge sites and stores only the essential information in the central repository. This extraction maintains data retention capability while dramatically improving processing efficiency by working with indexed, extracted data rather than large unindexed PCAP files.
Solution Approach 2:
The patent performs preliminary extraction and indexing of data at edge sites before transfer to the central repository. This preliminary action prepares data for efficient future analysis by creating structured, indexed formats that can be quickly processed without requiring re-processing of original PCAP files.
Data Source
AI summary
Systems and methods are disclosed for collecting network traffic logs at a plurality of network sites, such as DNS name servers and network routers, and transmitting data extracted from the network traffic logs to a central repository. In one implementation, a system includes a processor and a memory. The memory stores instructions that cause the processor to retrieve PCAP files from a plurality of servers and extract data from the PCAP files. The data comprises header data and digest data. The processor stores the header data and the digest data in a header/digest pair. In another aspect, the processor retrieves a sample of the PCAP files from each of the plurality of servers rather than retrieving all PCAP files.