High-Speed Packet Capture Architecture with FPGA Buffering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data packet capture tools struggle to provide reliable, low-loss packet capture in high-speed networks due to processor speed and memory limitations, leading to packet drops at Ethernet links operating at speeds of 10 gigabits per second, 40 gigabits per second, or 100 gigabits per second.
Innovation Solution
A packet capture architecture that processes chunks of packets rather than individual packets, utilizing a customized network interface based on field-programmable gate array technology, with ample buffering and pipelined processing between network interface, memory units, and non-volatile storage, allowing sustained capture rates of 100 gigabits per second.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If software-based packet capture is used on general purpose computing devices, then ease of operation is improved, but packet loss increases at high network speeds
Solution Approach 1:
The system segments the packet capture functionality into dedicated hardware components (network interface unit with packet capture engine, separate storage controllers, and memory units) rather than relying on general-purpose software processing. This hardware segmentation enables reliable high-speed packet capture while maintaining ease of operation through automated capture processes.
Solution Approach 2:
The patent introduces a dedicated packet capture engine and hardware buffer memory as intermediary components between the network interface and storage systems. These intermediaries handle the high-speed packet flow, preventing packet loss while allowing general-purpose computing devices to operate without specialized modifications.
2Reliability
If hardware-based packet capture is implemented, then reliability is improved, but device complexity increases
Solution Approach 1:
The network interface unit is designed with multi-functionality, serving both as a standard network interface and as a dedicated packet capture engine. This universal design provides reliable hardware-based packet capture without requiring separate specialized hardware devices, thereby limiting the increase in device complexity.
3Productivity
If packet capture rate is increased to match network speed, then productivity is improved, but loss of information increases due to buffer overflow
Solution Approach 1:
The system performs preliminary action by allocating充足 buffer memory capacity before packet capture begins. The buffer memory is sized to accommodate high-speed packet flows, and the capture engine is pre-configured to write packets to buffer memory at line rate, preventing buffer overflow and information loss while maintaining high productivity.
Solution Approach 2:
The packet capture engine creates copies of packets from the network interface and writes them to buffer memory and storage systems. This copying mechanism enables the system to capture packets at high speeds without losing information, as the buffer memory serves as an intermediate copy storage that prevents overflow.
4Measurement precision
If processing is performed on individual packets, then measurement precision is improved, but productivity decreases
Solution Approach 1:
The patent merges multiple packet processing operations into unified hardware-based capture processes. The packet capture engine simultaneously performs packet reception, buffering, and storage operations in parallel, achieving both high productivity and precise packet capture without the need for sequential individual packet processing.
Data Source
AI summary
An embodiment may involve receiving a chunk and a chunk index, where the chunk contains packets captured by a network interface unit and the chunk index contains timestamps of first and last packets within the chunk. The chunk may be stored in a first ring buffer of a first memory and the chunk index may be stored in an index buffer of the first memory. A processor may allocate an entry in an I/O queue of a second memory and an entry in a chunk processing queue of the first memory. The processor may read the chunk processing queue to identify and copy the chunk from the first ring buffer to a location in a second ring buffer of the second memory, the location associated with the entry in the I/O queue. The same or a different processor may instruct a controller to write the chunk to a non-volatile memory unit.