Packet Replication and Deduplication Across Multiple Access Paths

The deduplication mechanism is segmented into three distinct data structures: a hash table for rapid packet identification, a buffer map for storing packet segments, and a retransmission map for tracking retransmitted packets. This segmentation allows each component to specialize in a specific aspect of deduplication, improving overall reliability while managing complexity through modular design.

The patent introduces intermediary data structures that act as mediators between received packets and the final deduplication decision. The hash table serves as an intermediary that quickly identifies potential duplicates, the buffer map intermediates between packet segments and reassembly, and the retransmission map intermediates between retransmitted packets and deduplication logic. These intermediaries simplify the overall system by breaking down the complex deduplication task into manageable stages.

The system performs preliminary actions by pre-computing and storing packet hashes in the hash table before full packet processing. When a packet arrives, its hash is quickly computed and checked against the hash table to identify potential duplicates before committing resources to full packet storage and processing. This preliminary hash-based filtering minimizes redundant data processing while keeping processing time low through efficient hash computation.

The patent replaces traditional mechanical packet-by-packet comparison with a hash-based computational system. Instead of comparing entire packets sequentially (a mechanical process), the system substitutes hash computation and table lookup operations that are computationally efficient and can be performed rapidly, significantly reducing processing time while effectively minimizing redundant data.

Each data structure is optimized for its specific local function: the hash table uses compact key-value pairs for rapid identification with minimal memory footprint, the buffer map stores only necessary packet segments with pointers to avoid full duplication, and the retransmission map maintains selective tracking information. This local optimization ensures high identification accuracy while minimizing overall memory usage by allocating resources according to each component's specific needs.

The data structures are nested in a hierarchical manner where the hash table provides the outer layer of quick filtering, the buffer map provides the middle layer for segment management, and the retransmission map provides the inner layer for specific retransmission tracking. This nesting allows the system to achieve high identification accuracy through multiple layers of checking while minimizing memory usage by only storing detailed information in inner layers when necessary.