Distributed Network Trace Analysis with Smart Caching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network analysis tools are proprietary, single-server based, and lack scalability, forcing network operators to acquire multiple tools for different protocols and data types, leading to inefficient use of processing power and difficulty in consolidating storage for cross-tool data queries.
Innovation Solution
A distributed parallel processing system with a single point of entry, utilizing commodity hardware and open architecture, that transforms disparate network data into a common format for holistic analysis, enabling real-time processing, high throughput, and efficient storage across multiple servers, while allowing for elastic scaling and fault tolerance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If proprietary single-server network analysis tools are used, then specific protocol analysis can be performed, but scalability is limited and multiple tools are needed for different protocols
Solution Approach 1:
The patent implements a universal network analysis platform that can handle multiple protocols (HTTP, HTTPS, FTP, SMTP, DNS, SIP, VoIP) through a single server using a common format translation layer. The system translates diverse protocol data into a unified internal representation, eliminating the need for separate specialized tools for each protocol while maintaining protocol-specific analysis capabilities.
Solution Approach 2:
The patent introduces a format translation layer as an intermediary between various protocol data formats and the analysis engine. This mediator converts disparate protocol-specific data structures into a common internal format, enabling a single analysis tool to process multiple protocols without requiring direct protocol-specific processing logic in the analysis core.
2Adaptability or versatility
If multiple separate network analysis tools are deployed, then comprehensive protocol coverage is achieved, but processing power is wasted and storage consolidation becomes difficult
Solution Approach 1:
The patent merges the functionality of multiple separate network analysis tools into a single unified platform. By consolidating processing resources and storage into one server that handles all protocol types through the format translation layer, the system eliminates redundant processing capabilities and achieves economies of scale in both CPU utilization and storage management.
3Reliability
If legacy applications are allocated to separate virtual machines, then application isolation is maintained, but the genericity of virtual machine functionality is lost
Solution Approach 1:
The patent implements a universal virtual machine template that can run multiple different applications (web server, mail server, database, file server) on the same hardware platform. This universal VM design maintains application isolation through virtualization while enabling the same VM image to serve multiple functions, thereby preserving functionality genericity without sacrificing reliability.
Data Source
AI summary
A method and system of distributed parallel processing. There are a plurality of distributed parallel processing units (DPPUs). Each DPPU is configured to receive data related to a condition of the network. The type of data received by each DPPU is disparate for each DPPU. Each DPPU analyzes its data. Upon determining that a predetermined condition is met or a predetermined threshold is exceeded, the disparate data is transformed into a common format using an appropriate driver of the configuration module. The common format data is sent to a storage device of a first DPPU of the plurality of DPPUs.