Remote Capture Agents for Cloud Network Data Event Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional network data capture technologies are inflexible and difficult to deploy in cloud computing environments, as they require physical hardware appliances and are often customized for specific purposes, making it challenging to adapt to changing business needs or deploy in distributed and remote locations.
Innovation Solution
A system that uses remote capture agents to process network data by obtaining protocol classifications, building event streams from packet flows, and transmitting them over a network for storage and processing, with configuration information dynamically updated to enable flexible data capture and analysis, eliminating the need for physical hardware and allowing on-the-fly configuration changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If physical hardware appliances are used for network data capture, then capture reliability is improved, but device complexity and ease of deployment deteriorate
Solution Approach 1:
The patent creates virtual copies of network capture functionality through software agents that replicate the capture and processing capabilities of physical hardware appliances. These virtual agents can be deployed on standard servers and cloud instances, eliminating the need for specialized physical equipment while maintaining capture reliability through software-based packet interception and analysis.
Solution Approach 2:
The patent replaces physical hardware-based network capture mechanisms with software-based virtualization. Instead of using dedicated physical tap devices or hardware appliances, the system uses software agents that run on standard computing platforms to perform packet capture, protocol analysis, and data processing functions traditionally requiring specialized hardware.
2Reliability
If physical network capture devices are deployed, then data capture capability is improved, but ease of operation and adaptability deteriorate
Solution Approach 1:
The patent implements dynamic configuration capabilities where the network capture system can be reconfigured in real-time based on changing business requirements. The virtualized architecture allows capture agents to be dynamically deployed, scaled, and repositioned without physical reinstallation, enabling the system to adapt to varying network conditions and analytical needs.
Solution Approach 2:
The patent creates a universal network capture platform that can serve multiple functions and vertical markets through a single software-based architecture. The same virtualized capture infrastructure can be configured for security analysis, performance monitoring, QoS measurement, and other purposes by changing software parameters rather than requiring different physical devices.
3Loss of information
If ETL processes are used for data processing, then data analysis capability is improved, but loss of time and productivity deteriorate
Solution Approach 1:
The patent performs preliminary data processing and transformation actions at the point of capture within the virtualized environment itself. Rather than collecting raw data and later processing it through time-consuming ETL processes, the system pre-processes packets for relevant information extraction, filtering, and transformation closer to the network source, reducing the time required for subsequent analysis.
Solution Approach 2:
The patent introduces virtualized processing layers as intermediaries between network capture and final data analysis. These virtual processing agents perform intermediate transformation and aggregation operations that bridge the gap between raw packet capture and business intelligence, eliminating the need for heavy-duty ETL processes by processing data in a more integrated manner.
Data Source
AI summary
The disclosed embodiments provide a system that processes network data. During operation, the system obtains, at a remote capture agent, a first protocol classification for a first packet flow captured by the remote capture agent. Next, the system uses configuration information associated with the first protocol classification to build a first event stream from the first packet flow at the remote capture agent, wherein the first event stream comprises time-series event data generated from network packets in the first packet flow based on the first protocol classification. The system then transmits the first event stream over a network for subsequent storage and processing of the first event stream by one or more components on the network.