Network Interface Controller Packet Deduplication for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-speed packet streaming schemes for real-time video transmission across networks often require redundant streams to ensure data integrity, leading to increased memory footprint and processing burden on the host processor, which results in data latency and limited throughput.
Innovation Solution
A communication apparatus and method that utilizes a network interface controller (NIC) to receive and process redundant packet streams, mapping packet sequence numbers to buffer addresses to eliminate redundant data, thereby ensuring only one copy of each data slice is stored in order, offloading reordering and reconstruction tasks from the host processor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If redundant packet streams are transmitted to ensure data integrity, then reliability is improved, but memory footprint and processing burden increase
Solution Approach 1:
The patent extracts the packet reordering and deduplication function from the host processor and implements it in the network interface controller (NIC) hardware. The NIC maintains a reordering buffer and uses packet sequence numbers to automatically reorder packets and eliminate duplicates, removing this processing burden from the host while maintaining reliability through redundant stream handling.
Solution Approach 2:
The patent introduces a reordering buffer as an intermediary component between the network interface and the host processor. This buffer temporarily stores incoming packets from multiple redundant streams and allows the host to receive data in correct sequence without processing the complexity of reordering and deduplication, thus improving reliability while reducing host processing burden.
2Reliability
If packets are reordered and interleaved to reconstruct data, then reliability is improved, but data latency increases
Solution Approach 1:
The patent performs preliminary reordering and deduplication actions in the NIC's reordering buffer before data is transferred to the host. By pre-processing the packet streams and arranging packets in correct sequence in advance, the system ensures reliable data reconstruction without adding latency during the actual data transfer to the host processor.
3Reliability
If multiple redundant streams are processed in parallel, then reliability is improved, but processing load on host processor increases
Solution Approach 1:
The patent extracts the complex task of processing multiple redundant streams from the host processor and implements it in the NIC hardware. The NIC receives multiple packet streams, performs deduplication using packet sequence numbers, and delivers a single consolidated stream to the host, thereby improving reliability through redundant transmission while maintaining high data throughput by offloading processing.
Data Source
AI summary
Communication apparatus includes a host interface and a network interface, which receives at least first and second redundant packet streams, each including a sequence of data packets, which include headers containing respective packet sequence numbers and data payloads of a predefined, fixed size containing respective slices of the data segment. Redundant first and second copies of each slice are transmitted in respective packets in the first and second packet streams. Packet processing circuitry receives the data packets from the network interface, maps the data packets in both the first and second packet streams, using the packet sequence numbers, to respective addresses in a buffer, and writes the data payloads to the respective addresses via the host interface while eliminating redundant data so that the buffer contains exactly one copy of each slice of the data segment, ordered in accordance with the packet sequence numbers.