Dynamic Queue Network Monitoring for Duplicate Packet Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional network monitoring devices face scalability issues and fail to effectively identify and remove duplicate packets, which are created at multiple monitoring points and contribute to errors in network performance analysis due to fixed queue lengths that do not adapt to varying packet rates.
Innovation Solution
A network monitoring device that dynamically adjusts the queue size based on packet arrival rate and includes a threshold value to compare new packets with a variable-sized list of received packets, using a two-level comparison algorithm involving hashing and byte-by-byte verification to identify and remove duplicates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a fixed queue length is used for packet comparison, then the device complexity is reduced, but the reliability of duplicate packet identification deteriorates due to varying packet rates
Solution Approach 1:
The patent implements a dynamic queue size adjustment mechanism where the queue length is no longer fixed but adapts based on the observed packet arrival rate. The system continuously monitors the number of packets arriving within specific time windows and adjusts the queue size accordingly, allowing it to expand during high-traffic periods and contract during low-traffic periods. This dynamic behavior ensures that the queue always contains an appropriate number of recent packets for comparison, maintaining high duplicate detection accuracy without requiring excessive memory resources.
Solution Approach 2:
The system changes the parameter of queue size from a static fixed value to a dynamic value that varies with network conditions. By introducing a time-based observation window and counting packets within that window, the system transforms the queue size parameter into one that automatically adjusts to the current packet arrival rate. This parameter change enables the system to maintain optimal performance across varying network loads without manual intervention or complex configuration.
2Reliability
If the queue size is increased to capture more packets for comparison, then the reliability of duplicate identification improves, but the loss of time for processing increases
Solution Approach 1:
The system dynamically adjusts the comparison window size based on the packet arrival rate observed in recent time periods. When packet rates are high, the queue naturally contains more packets, but the system only compares against packets within a reasonable time window, preventing excessive processing. When packet rates are low, the queue size reduces accordingly. This dynamic adjustment ensures that the system processes only the necessary number of packets for accurate duplicate detection, avoiding unnecessary processing time while maintaining detection reliability.
Solution Approach 2:
The system implements periodic observation of packet arrival rates by measuring the number of packets within specific time windows (e.g., counting packets in the last 100ms, 500ms, or 1 second). This periodic measurement allows the system to adaptively adjust its comparison scope based on current network conditions rather than using a fixed, always-maximal comparison set. The periodic nature of this adjustment ensures that processing time remains proportional to actual network activity.
3Measurement precision
If multiple monitoring points are deployed to collect packets, then the measurement precision of network monitoring improves, but duplicate packets are generated causing errors in analysis
Solution Approach 1:
The system segments the duplicate detection process into two levels: first, a hash-based filtering stage that quickly identifies potential duplicates by comparing hash values of packet headers, and second, a detailed byte-by-byte verification stage that confirms actual duplicates. This segmented approach allows the system to efficiently handle packets from multiple monitoring points by first filtering out obvious duplicates through hashing, then performing thorough comparison only on candidates, thereby maintaining measurement precision while eliminating duplicate-related errors.
Solution Approach 2:
The patent introduces a hash value as an intermediary element in the duplicate detection process. Instead of directly comparing all packets from multiple monitoring points, the system first computes hash values and compares these intermediaries. Only when hash values match does the system proceed to full packet comparison. This intermediary mechanism efficiently handles multi-point monitoring data by filtering out non-duplicates early, reducing the computational burden while maintaining accurate duplicate identification.
Data Source
AI summary
A network monitoring device configured to collect a new packet from one or more observation points of a network and to compare the new packet with a list of a number of received packets based on a packet arrival rate and to identify a duplicate packet. In particular, the number of received packets in the list is equivalent to a number of packets received within a time period, i.e. the packet arrival rate. Stated differently, the network monitoring device is to compare the new packets with received packets stored in a queue of a buffer and wherein the queue has a size based on a packet arrival rate collected at one or more observation points. In addition, the time period is further adjusted according to a threshold value. The threshold value is a variable parameter that can be adjusted to compensate for different network deployment. In one embodiment, the threshold value is a time value that is not more than a transmission time of a TCP retransmitted packet.