A method and system for detecting high round-trip delay streams in high-speed network flows.

Abstract

This invention discloses a method and system for detecting high round-trip time (RTT) flows in high-speed networks, belonging to the field of traffic detection technology. The method includes: acquiring network data flows and generating RTT samples of the network data flows; judging the validity of the RTT samples and filtering out valid samples; statistically analyzing the valid samples within a statistical period using limited storage resources and obtaining statistical results; and based on the statistical results, identifying and outputting data flows corresponding to records whose cumulative RTT values ​​exceed a preset abnormal threshold as abnormal high RTT data flows. This invention employs a two-stage processing approach. The first stage generates reliable RTT samples and removes abnormally large RTTs caused by retransmissions, while the second stage continuously tracks abnormal RTT flows based on the cleaned RTT data. This achieves efficient and accurate detection of abnormal RTT flows while meeting the processing capabilities of high-speed networks.

Description

Technical Field

[0001] This invention relates to a method and system for detecting high round-trip delay flows in high-speed network flows, belonging to the field of flow detection technology. Background Technology

[0002] With the rapid development of cloud computing and the continuous expansion of data center scale, overall internet traffic is growing exponentially. Against this backdrop, achieving efficient and refined network management while matching network traffic of hundreds of Gbps or even higher has become a core challenge for modern network operations and maintenance. Network congestion control is a fundamental technology for key functions such as traffic load balancing, intelligent routing optimization, and anomaly protection. This technology typically infers the congestion status and potential bottleneck locations of network links by continuously monitoring changes in the round-trip time (RTT) of TCP connections (or sessions) within a certain time window, thereby assisting in accurate fault location, anomaly diagnosis, and route adjustment.

[0003] Traditional RTT measurement schemes (such as TCPtrace) are typically deployed on the user's host side, actively sending probe packets to obtain RTT information. However, with the increasing number of network users, the probe packets themselves consume a large amount of bandwidth resources and exacerbate network congestion, thus affecting measurement accuracy. Furthermore, in multi-tenant environments, they may compromise tenant isolation, making them unsuitable for cloud computing scenarios. In recent years, in-band network telemetry (INT) has provided a new approach to RTT measurement by embedding metadata such as timestamps in the packet header to achieve packet-by-packet latency collection. However, INT consumes additional link bandwidth, and its deployment relies on dedicated equipment, resulting in high costs and difficulties in implementation. To avoid additional bandwidth overhead, some research has attempted to use programmable switches to construct compact data structures to collect RTT for each flow. However, such methods typically require frequent interaction with the control plane, making communication costs unacceptable. Simultaneously, because the cumulative RTT value for each flow exhibits a significant tail-skewed distribution, only a small number of abnormal or congested flows show a significant increase in RTT, while most flows remain within the normal range. Performing equally granular monitoring on all flows would result in wasted storage resources. Therefore, there is an urgent need for a measurement mechanism that can operate entirely in the data plane and quickly and accurately identify abnormally high RTT streams without increasing bandwidth and communication overhead.

[0004] The available storage resources in modern programmable data planes are extremely limited, typically only tens of MB, and need to be shared with multiple critical functional modules such as access control, network security, and traffic engineering. The actual storage space allocated to measurement tasks is often less than 1 MB. This exacerbates the contradiction between massive amounts of data and limited processing resources in high-speed networks. To address this challenge, researchers have proposed several types of compact data structures (Sketch), which can achieve fast and high-precision traffic measurement within sublinear storage space through multiple independent hashes and minimal memory accesses, providing an effective solution for resource-constrained scenarios. However, existing Sketches are mainly geared towards coarse-grained traffic measurement tasks, such as per-flow size estimation or large-flow detection, focusing on statistical analysis of unidirectional traffic characteristics without considering the packet type matching characteristics required for RTT measurement—that is, only when data packets and their corresponding ACK packets participate together can an RTT sample be generated. Therefore, existing Sketches cannot be directly applied to fine-grained measurement of per-flow RTT, let alone meet the needs of detecting abnormally high RTT flows.

[0005] Therefore, in order to solve the above-mentioned technical problems, there is an urgent need for a method and system for detecting high round-trip delay flows in high-speed network flows. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for detecting high round-trip time (RTT) flows in high-speed network flows, which achieves efficient and accurate detection of abnormal RTT flows while meeting the processing capabilities of high-speed networks.

[0007] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0008] In a first aspect, the present invention provides a method for detecting high round-trip latency flows in high-speed network flows, comprising:

[0009] Acquire network data streams and generate round-trip delay samples of the network data streams;

[0010] The validity of the round-trip delay samples is assessed, and valid samples are selected.

[0011] Within limited storage resources, statistical analysis is performed on the valid samples in a statistical period to obtain statistical results;

[0012] Based on the statistical results, the data streams corresponding to records whose cumulative round-trip latency exceeds a preset abnormal threshold are identified as abnormally high round-trip latency data streams and output.

[0013] Furthermore, the acquisition of network data streams, by matching request data units and acknowledgment data units of the same data stream to generate round-trip delay samples for each data stream, includes:

[0014] TCP packets are filtered from the network data stream, and a flow label is generated for each TCP packet based on the source and destination network addresses and port information;

[0015] For each TCP packet, a matching request data unit is searched in a pre-built one-dimensional storage structure based on its flow label:

[0016] If no match is found, the current TCP packet is determined to be a request data unit; a free storage bit is found in the one-dimensional storage structure to store the flow tag and timestamp of the current TCP packet; when the request data unit is matched by a new TCP packet, the round-trip delay sample is calculated based on the difference between the timestamp of the new TCP packet and the timestamp of the request data unit.

[0017] If a match is found, the current TCP packet is determined to be an acknowledgment data unit, and the round-trip delay sample is calculated based on the difference between the timestamp of the current TCP packet and the timestamp of the matched request data unit.

[0018] Furthermore, during the search for free storage space, if the recording time of a certain storage space exceeds a preset threshold, the storage space is cleared and designated as a free storage space. If there is no free storage space in the one-dimensional storage structure, the processing of the current TCP data packet is abandoned.

[0019] Furthermore, the one-dimensional storage structure includes a bucket array, which includes multiple buckets, each of which includes multiple slots, and each slot is used to store the stream tag and timestamp information of a request data unit;

[0020] For each TCP packet, the corresponding bucket is located using a hash function based on its flow label, and all slots in the located bucket are traversed to search and match the requested data unit.

[0021] Furthermore, the statistics are implemented using a two-dimensional storage array. Each storage unit of the two-dimensional storage array records the stream label of the data stream and its cumulative round-trip latency value. Each data stream is associated with candidate storage units in different rows of the two-dimensional storage array through multiple independent mapping functions.

[0022] Furthermore, the step of statistically analyzing the valid samples within a statistical period within limited storage resources includes:

[0023] For each valid sample, examine all candidate storage units determined by multiple mapping functions;

[0024] If any candidate storage unit records the valid sample, then update its cumulative round-trip latency value;

[0025] If none of the candidate storage units have recorded the valid sample, then select one of the empty candidate storage units to record the valid sample.

[0026] If all candidate storage units are occupied, the valid sample replaces the existing record in one of the candidate storage units according to a predetermined replacement strategy.

[0027] Furthermore, if all candidate storage units are already occupied, then according to a predetermined replacement strategy, the valid sample replaces the existing record in one of the candidate storage units, including:

[0028] When all candidate storage units are occupied, the replacement probability is determined based on the round-trip latency sample value of the current data stream and the minimum cumulative round-trip latency value recorded in the candidate storage units. When the replacement probability is greater than the generated pseudo-random number, the replacement operation is performed.

[0029] The formula for calculating the replacement probability includes:

[0030] ;

[0031] In the formula, For the probability of replacement, This represents the sample value of the round-trip delay for the current data stream. It is the minimum cumulative round-trip latency value recorded in the candidate storage unit.

[0032] Secondly, the present invention provides a high round-trip latency flow detection system for high-speed network flows, comprising:

[0033] Generation module: used to acquire network data streams and generate round-trip delay samples of the network data streams;

[0034] Filtering module: used to determine the validity of the round-trip delay samples and filter out valid samples;

[0035] The statistics module is used to perform statistical analysis on the valid samples within a statistical period within limited storage resources and obtain statistical results.

[0036] Output module: Based on the statistical results, the data streams corresponding to records whose cumulative round-trip delay values ​​exceed a preset abnormal threshold are identified as abnormally high round-trip delay data streams and output.

[0037] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0038] Fourthly, the present invention provides a computer device, comprising:

[0039] Memory, used to store computer programs / instructions;

[0040] A processor for executing the computer program / instructions to implement the steps of any of the methods described above.

[0041] Fifthly, the present invention provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of any of the methods described above.

[0042] Beneficial effects

[0043] The high round-trip time (RTT) flow detection method for high-speed network flows provided by this invention adopts a two-stage processing. The first stage generates reliable RTT samples and removes abnormally large RTTs caused by retransmission and other behaviors. The second stage continuously tracks abnormal RTT flows based on the cleaned RTT data.

[0044] The high round-trip time (RTT) flow detection method for high-speed network flows provided by this invention takes into account the tail-biased characteristic of the cumulative RTT value of each flow and designs a probabilistic replacement strategy. This strategy prioritizes the retention of abnormal RTT flows in the storage space and gradually evicts normal flows, thereby significantly reducing storage overhead. While meeting the processing capabilities of high-speed networks, it achieves efficient and accurate detection of abnormal RTT flows. Attached Figure Description

[0045] Figure 1 This is a flowchart of a high round-trip delay flow detection method for high-speed network flows provided by an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram illustrating the specific implementation of the RTT sample generation part in a high round-trip delay flow detection method for high-speed network flows provided in an embodiment of the present invention.

[0047] Figure 3 This is a schematic diagram illustrating the specific implementation of the abnormal RTT sample filtering part in a high round-trip delay flow detection method for high-speed network flows provided in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram illustrating the specific implementation of the high RTT flow tracing part in a high round-trip delay flow detection method for high-speed network flows provided in an embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram illustrating the specific implementation of the abnormal RTT flow detection part in a high round-trip delay flow detection method for high-speed network flows provided in an embodiment of the present invention;

[0050] Figure 6 This is a schematic diagram showing the accuracy of the present invention on the MAWI-24 dataset as a function of storage space.

[0051] Figure 7 This is a schematic diagram of the recall rate versus storage space curve on the MAWI-24 dataset for this invention;

[0052] Figure 8 This is a schematic diagram of the F1 score versus storage space curve on the MAWI-24 dataset according to the present invention;

[0053] Figure 9 This is a schematic diagram of the throughput versus storage space curve of the present invention on the MAWI-24 dataset;

[0054] Figure 10 This is a schematic diagram showing the accuracy of the present invention on the MAWI-25 dataset as a function of storage space.

[0055] Figure 11 This is a schematic diagram of the recall rate versus storage space curve on the MAWI-25 dataset for this invention;

[0056] Figure 12 This is a schematic diagram of the F1 score versus storage space curve on the MAWI-25 dataset according to the present invention;

[0057] Figure 13 This is a schematic diagram of the throughput versus storage space curve of the present invention on the MAWI-25 dataset;

[0058] Figure 14 This is a flowchart illustrating a high round-trip delay flow detection method for high-speed network flows provided in an embodiment of the present invention. Detailed Implementation

[0059] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.

[0060] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0061] Example 1

[0062] Figure 14This is a flowchart of a high round-trip latency (RTL) flow detection method for high-speed network flows according to Embodiment 1 of the present invention. The high RTL flow detection method for high-speed network flows provided in this embodiment can be applied to a terminal and can be executed by a high RTL flow detection system for high-speed network flows. This system can be implemented in software and / or hardware and can be integrated into the terminal, such as any smartphone, tablet, or computer device with communication capabilities. See also... Figure 14 The method in this embodiment specifically includes the following steps:

[0063] Acquire network data streams and generate round-trip delay samples of the network data streams;

[0064] The validity of the round-trip delay samples is assessed, and valid samples are selected.

[0065] Within limited storage resources, statistical analysis is performed on the valid samples in a statistical period to obtain statistical results;

[0066] Based on the statistical results, the data streams corresponding to records whose cumulative round-trip latency exceeds a preset abnormal threshold are identified as abnormally high round-trip latency data streams and output.

[0067] In the above technical solution, the method mainly includes four parts: RTT sample generation, abnormal RTT sample filtering, high RTT stream tracing, and abnormal RTT stream detection.

[0068] The RTT sample generation section records the flow tag, ACK information, and timestamp information carried by the most recently arrived data packets of each TCP flow in each measurement period. For data packets that have not received a corresponding ACK packet for a long time, they are actively cleared to save storage space. Subsequently, this section matches the sequence number of newly arrived packets with the recorded ACK values ​​to generate RTT sample data for the corresponding flow.

[0069] The abnormal RTT sample filtering section is used to determine the validity of the generated RTT samples. Specifically, based on a pre-set valid RTT sample threshold T, samples with RTT values ​​higher than T are identified as invalid samples and prevented from entering the subsequent processing flow; samples with RTT values ​​not exceeding the threshold T are considered valid RTT data and are sent to subsequent modules for processing along with the corresponding stream tag.

[0070] The high RTT stream tracking section is used to record and store the cumulative RTT data of a small number of streams with abnormally high RTT within the current measurement cycle with high precision; at the same time, it adopts an active eviction strategy for a large number of low RTT streams to reduce storage overhead and improve overall measurement efficiency.

[0071] The abnormal RTT flow detection section is used to analyze the recorded flow information after the measurement period ends, identify and output all TCP flows with abnormally high RTT characteristics; then, these suspicious flow record sets are reported to the control plane to assist the administrator in performing more efficient network management and anomaly diagnosis.

[0072] In the RTT sample generation section, the acquisition of network data streams, by matching request data units and acknowledgment data units of the same data stream to generate round-trip time (RTT) samples for each data stream, includes:

[0073] TCP packets are filtered from the network data stream, and a flow label is generated for each TCP packet based on the source and destination network addresses and port information;

[0074] For each TCP packet, a matching request data unit is searched in a pre-built one-dimensional storage structure based on its flow label:

[0075] If no match is found, the current TCP packet is determined to be a request data unit; a free storage bit is found in the one-dimensional storage structure to store the flow tag and timestamp of the current TCP packet; when the request data unit is matched by a new TCP packet, the round-trip delay sample is calculated based on the difference between the timestamp of the new TCP packet and the timestamp of the request data unit.

[0076] If a match is found, the current TCP packet is determined to be an acknowledgment data unit, and the round-trip delay sample is calculated based on the difference between the timestamp of the current TCP packet and the timestamp of the matched request data unit.

[0077] During the search for free storage space, if the recording time of a certain storage space exceeds a preset threshold, the storage space is cleared and designated as a free storage space. If there is no free storage space in the one-dimensional storage structure, the processing of the current TCP data packet is abandoned.

[0078] The specific implementation is as follows:

[0079] TCP packets are filtered from the network data stream. This includes: for each packet arriving at the switch's on-chip buffer within the current period, firstly, its transport layer protocol number is checked against measurement requirements to ensure it is 6. If the protocol number is not 6, the packet is not a TCP segment and is directly forwarded to the switch without further processing. If the protocol number is 6, it is identified as a TCP segment, and its source IP address, destination IP address, source port, and destination port are extracted to form a TCP flow label f. A flow label is generated for each TCP packet based on the source and destination network addresses and port information. This includes: if the source IP address is less than the destination IP address, the flow label f ​​is defined as "source IP address - source port number - destination IP address - destination port number"; otherwise, the flow label f ​​is defined as "destination IP address - destination port number - source IP address - source port number". Subsequently, the sequence number, acknowledgment number (ACK), and timestamp information are further extracted from the TCP segment as flow elements. Through the above processing, each eligible data packet is modeled as a tuple consisting of a "flow label – flow element". This is used for the operation and processing of subsequent measurement modules.

[0080] For each pair obtained from the modeling First, it enters the RTT sample generation section for processing. If there is a stream tag in the system that matches this stream tag... If a matching record is found and the arriving data packet contains its corresponding ACK message, then based on the element... The timestamp carried in the stream is used to calculate the RTT sample for the stream using the timestamps already recorded. Furthermore, by comparing the timestamp mapped to the record unit with the current timestamp, a triggered clearing operation can be implemented to remove records that have not received an ACK for a long time from this stage, thereby further improving storage space utilization.

[0081] The generated RTT samples then enter the abnormal RTT sample filtering section for validity assessment. Specifically, the sample value is compared with a preset threshold T; if the sample value exceeds the threshold T, it is determined as an invalid sample and subsequent processing is terminated; if the sample value does not exceed the threshold T, it is considered valid RTT data, and the newly formed tuple is... Send to the high RTT stream tracing section for further processing;

[0082] After the measurement period ends, the algorithm enters the abnormal RTT flow detection section, which analyzes and judges the RTT data of each TCP flow recorded in the structure. If the RTT value of a flow exceeds the preset normal range threshold, the flow is identified as a suspicious flow and added to the abnormal RTT flow candidate set. Finally, the candidate set is reported to the control plane to support administrators in performing network anomaly location and management operations.

[0083] The one-dimensional storage structure includes a bucket array, which includes multiple buckets, each bucket including multiple slots, and each slot is used to store the stream tag and timestamp information of a request data unit;

[0084] For each TCP packet, the corresponding bucket is located using a hash function based on its flow label, and all slots in the located bucket are traversed to search and match the requested data unit.

[0085] The specific implementation is as follows:

[0086] Before the measurement begins, the RTT sample generation section needs to be initialized. This section includes a length of... bucket array This refers to the aforementioned one-dimensional bucket array, where each bucket contains c slots and is configured with a mapping function to map stream tags to bucket indices. Each slot contains three fields: an ID field (B[][].ID), an ACK field (B[][].ACK), and a Time field (B[][].Time), used to store the flow tag carried in the message, the acknowledgment number in the flow element, and the timestamp information, respectively. During initialization, the ID field in each slot is set to empty, and the ACK and Time fields are set to 0. The mapping function is defined as follows: ,in An independent and uniform hash function is used to assign stream tags. Mapping to array The corresponding position. In addition, a normal RTT sample threshold T needs to be preset according to the measurement requirements;

[0087] For the arriving data packets Record the stream tag it carries as The element ID is e. The element pe contains the sequence number peSEQ, the acknowledgment number peACK, and the timestamp pet. First, its index i in the bucket array B is calculated using the stream label-index mapping function. The specific implementation is as follows:

[0088] ;

[0089] Read the storage buckets sequentially The ID field of each slot is used to determine whether it matches the "source IP-source port-destination IP-destination port" carried in the data packet. Specifically, a matching function is defined. Used to determine the bucket Does the ID field of the j-th slot match the data packet? . Specifically, The range of values ​​is The calculation method is as follows:

[0090] (Formula 2);

[0091] In the formula, These correspond to the source IP, source port, destination IP, and destination port, respectively, according to The output of the RTT sample generation part can be further subdivided into two cases:

[0092] if Then judge Is it greater than T? If This means The recorded information has expired and needs to be cleared to allow for the storage of newly arriving data packets. At this point, the RTT sample generation section will... , and Set them to empty, 0, and 0 respectively. Record the index j in a variable during program execution. Medium; otherwise, it means The recorded information has not expired and will not be processed further.

[0093] if If 1 indicates that the arriving data packet p is an ACK packet for the recorded message, then RTT sample data is generated based on the timestamp difference between the two packets, and the RTT sample generation operation ends. Given a data packet p with a flow label f, its RTT sample calculation formula is as follows:

[0094] (Formula 3);

[0095] If the arriving data packet does not generate an RTT sample, then check the runtime variables of the program. Whether it is empty. Specifically, it can be divided into the following two cases:

[0096] if If it is not empty, it means the storage bucket is empty. No. Each slot can be allocated to an incoming data packet to store the three attributes it carries: "source IP-source port-destination IP-destination port", "acknowledgment number", and "timestamp".

[0097] if If it is empty, it means the storage bucket is empty. All slots in the packet are still active and waiting for the ACK message to arrive. Therefore, the packet is forwarded directly, ending all subsequent operations.

[0098] The validity of the round-trip delay samples is determined, and valid samples are selected. The specific implementation is as follows:

[0099] The data packets carrying the flow label f ​​generated in the RTT sample generation section Compare with T. Specifically, there are two cases:

[0100] if This indicates that the sample may be a retransmission packet sent due to the timeout retransmission mechanism in the data stream. It is necessary to prevent this invalid sample from being included in subsequent high-RTT stream tracing to avoid serious false positives caused by a large number of invalid samples. At this point, the abnormal RTT sample filtering section will... Discard and end subsequent operations;

[0101] if This indicates that the sample represents the round-trip latency generated by ordinary data packets in the data stream, and can be used to accurately characterize the overall latency of the data stream. Therefore, this section connects the stream label f ​​with... To form a new pair It is then sent to the high RTT stream tracing section for processing.

[0102] The statistical processing is implemented through a two-dimensional storage array. Each storage cell of the two-dimensional storage array records the stream label of the data stream and its cumulative round-trip latency value. Through multiple independent mapping functions, each data stream is associated with candidate storage cells in different rows of the two-dimensional storage array.

[0103] The statistical analysis of the valid samples within a statistical period within limited storage resources includes:

[0104] For each valid sample, examine all candidate storage units determined by multiple mapping functions;

[0105] If any candidate storage unit records the valid sample, then update its cumulative round-trip latency value;

[0106] If none of the candidate storage units have recorded the valid sample, then select one of the empty candidate storage units to record the valid sample.

[0107] If all candidate storage units are occupied, the valid sample replaces the existing record in one of the candidate storage units according to a predetermined replacement strategy.

[0108] If all candidate storage units are occupied, then according to a predetermined replacement strategy, the valid sample replaces the existing record in one of the candidate storage units, including:

[0109] When all candidate storage units are occupied, the replacement probability is determined based on the round-trip latency sample value of the current data stream and the minimum cumulative round-trip latency value recorded in the candidate storage units. When the replacement probability is greater than the generated pseudo-random number, the replacement operation is performed.

[0110] The formula for calculating the replacement probability includes:

[0111] ;

[0112] In the formula, For the probability of replacement, This represents the sample value of the round-trip delay for the current data stream. It is the minimum cumulative round-trip latency value recorded in the candidate storage unit.

[0113] The specific implementation is as follows:

[0114] Before measurement begins, the high RTT stream tracing section needs to be initialized. This section contains a two-dimensional array of buckets with height d and length w. This refers to the aforementioned two-dimensional storage array. Each bucket in the array has two fields: a stream tag field (C[][].ID) and a total RTT count field (C[][].S), used to record the tag information of the corresponding stream and its cumulative RTT value, respectively. In addition, this part also configures d independent mapping functions from stream tags to bucket indices. During initialization, the stream label field for each bucket is set to empty, and the total RTT count field is initialized to 0. The mapping function is defined as follows: ,in A set of independent and uniform hash functions is used to map the stream label f ​​to the corresponding position in the i-th row of the two-dimensional bucket array C;

[0115] For each valid sample, check all candidate storage units determined by multiple mapping functions; if any candidate storage unit records the valid sample, update its cumulative round-trip latency value; if none of the candidate storage units record the valid sample, select one empty candidate storage unit to record the valid sample; if all candidate storage units are occupied, replace the existing record of one candidate storage unit with the valid sample according to a predetermined replacement strategy, including: for the tuples formed by the valid samples obtained from the aforementioned partial filtering. First, calculate its position in the first row of the bucket array based on the first stream label-bucket index mapping function. subscript The specific implementation method is as follows:

[0116] (Formula 4);

[0117] If storage bucket [ If this data stream is recorded, its total RTT count field is incremented. Then the high RTT stream tracing section is terminated. The specific implementation is as follows:

[0118] (Formula 5);

[0119] If storage bucket [ If the record is of another stream, then further judgment is needed. [ Is the ID empty? This can be further divided into the following two cases:

[0120] if [ If ].ID is empty, then the variable will be used during program runtime. and Change them to 0 respectively. ;

[0121] if [ If ].ID is not empty, then the variable will be used during program execution. and Modify them respectively to and [ ].S.

[0122] Then, the high RTT stream tracing section will check the buckets mapped to by rows 2 to d in the two-dimensional bucket array C using the same steps, and update the runtime variables. , and Among them, temporary variables This represents the row number of the bucket with an empty ID field among the d candidate buckets; it is a temporary variable. This represents the column number of the bucket where the ID field is empty among the d candidate buckets; it is a temporary variable. This represents the row number of the bucket with the smallest cumulative RTT value among the d candidate buckets; it is a temporary variable. This represents the column number of the bucket with the smallest cumulative RTT value among d candidate buckets; it is a temporary variable. This represents the minimum cumulative RTT value among the d candidate buckets.

[0123] if If the value is not empty, it means that although the data stream is not recorded in C, there are empty buckets among the d candidate buckets in C. In this case, the high RTT stream tracing section will store the buckets. The ID and total RTT count fields in the data are updated to f and f, respectively. ;

[0124] if If the value is empty, it means that all d candidate buckets in the two-dimensional bucket array C for this data stream are occupied by other data streams. In this case, this part is based on the minimum total RTT count recorded in the d candidate buckets and the values ​​generated by the above part. Calculate the replacement probability And based on this probability, the candidate bucket carrying the minimum total RTT count is replaced. Wherein, the replacement probability... The specific calculation formula is as follows:

[0125] (Formula 6);

[0126] Before performing the replacement operation, a pseudo-random number r is first generated using the rand(·) function provided by the existing programming language, and it is then determined whether r satisfies the following inequality:

[0127] ;

[0128] If the conditions are not met, the operation of the high RTT stream tracing section will be terminated;

[0129] If satisfied, then store the bucket. The ID and S fields are updated sequentially with the flow label f ​​carried by the arriving data packet. Then end the operation for that part.

[0130] In the above technical solution, the generated pseudo-random numbers are uniformly distributed between 0 and 1. If the expulsion probability is higher, then r is more likely to fall within 0 to Pr; otherwise, it is more likely to fall outside this range (in which case the large flow is stored). Therefore, this not only increases the likelihood that the large flow will continue to be stored (tracked), but also expels as many small flows as possible.

[0131] After the measurement cycle ends, this part checks the two-dimensional storage bucket array in sequence. Each bucket in C. If the total RTT count in the bucket being checked exceeds a preset normal threshold, the ID field recorded by that bucket is added to the candidate high RTT stream set; otherwise, it means that the stream recorded by that bucket is not a stream with abnormal RTT, so the next bucket is checked, until all buckets in C have been traversed, and the generated abnormal stream set is transmitted to the control plane.

[0132] like Figure 1 As shown in the figure, the high round-trip latency flow detection method for high-speed network flows provided in this embodiment involves the following steps in its application process:

[0133] S11: Parse the data packets arriving at the switch's on-chip cache. Obtain the source IP address, destination IP address, source port number, destination port number, and transport layer protocol number from the packet header;

[0134] S12: Determine if the transport layer protocol number is 6. If yes, proceed to step S13; otherwise, proceed to step S14.

[0135] S13: Determine the data packet For TCP packets, and further define the stream labels. Specifically, if the source IP address is less than the destination IP address, the flow label f ​​is defined as "source IP address - source port number - destination IP address - destination port number"; otherwise, the flow label f ​​is defined as "destination IP address - destination port number - source IP address - source port number". Subsequently, the sequence number, acknowledgment number (ACK), and timestamp information are further extracted from this TCP segment as flow elements. The arriving data packets are then further modeled as a "stream label – stream element" tuple. Proceed to step S15;

[0136] S14: Forward the data packet from the switch's output port;

[0137] S15: For each pair obtained from the modeling If there is a record in the system that matches the flow label, and the arriving data packet is the matching ACK packet, then the RTT sample of the flow is calculated based on the timestamp carried in the element and the recorded timestamp, and step S16 is executed; otherwise, the subsequent operation is terminated and the process jumps to step S14.

[0138] S16: The generated RTT samples then enter the abnormal RTT sample filtering section for validity assessment. Specifically, the sample value is compared with a preset threshold T; if the sample value exceeds the threshold T, it is determined as an invalid sample and subsequent processing is terminated; if the sample value does not exceed the threshold T, it is considered valid RTT data, and the newly formed tuple is... Send to the high RTT stream tracing section for further processing;

[0139] S17: After the measurement period ends, the algorithm enters the abnormal RTT flow detection section, analyzes and judges the RTT data of each flow recorded in the structure. If the RTT value of a certain flow exceeds the preset normal range threshold, mark this flow as a suspicious flow and add it to the abnormal RTT flow candidate set, and report this candidate set to the control plane to support the administrator to perform network anomaly location and management operations;

[0140] As Figure 2 shown, the specific implementation of the RTT sample generation part includes the following steps:

[0141] S21: Initialize the data structure, including an array of buckets with a length of , where each bucket contains c slots, and a mapping function is configured to map the flow label to the bucket index . Each slot contains three fields: an ID field, an ACK field, and a Time field, which are used to store the flow label carried by the packet, the acknowledgment number in the flow element, and the timestamp information respectively. The ID field in each slot is set to be empty, and the ACK field and the Time field are set to 0. The mapping function is defined as , where is an independent and uniform hash function, which is used to map the flow label f to the corresponding position in the array B, and according to the measurement requirements, a normal RTT sample threshold T is set in advance;

[0142] S22: Obtain the flow label f of the packet modeled as a binary tuple p;

[0143] S23: Calculate its index i in the bucket array B according to formula (1), and set the program temporary variable j to 0;

[0144] S24: Read the ID field of each slot in the bucket in turn to judge whether it can match the "source IP address - source port number - destination IP address - destination port number" carried by the packet, and define the matching function to judge whether the ID field of the jth slot in the bucket matches the packet, where 0 j < c, calculate according to formula (2). If outputs 1, execute step S25, otherwise, jump to execute step S26;

[0145] S25: Generate RTT sample data according to formula (3), and then end the operation of the RTT sample generation part;

[0146] S26: Judge whether is greater than T. If If the above steps are not executed, proceed to step S27; otherwise, proceed to step S28.

[0147] S27: Will , and Set them to empty, 0, and 0 respectively, and record the index j in the variable at runtime. Then increment j by 1;

[0148] S28: Increment j by 1, without any other processing;

[0149] S29: Determine if j is less than c. If yes, proceed to step S24; otherwise, proceed to step S210.

[0150] S210: Judgment Is it empty? If If empty, proceed to step S211; otherwise, proceed to step S212.

[0151] S211: Storage Bucket All slots in the packet have not expired and are waiting for the ACK message to arrive. Therefore, the packet is forwarded directly, ending all subsequent operations.

[0152] S212: Storage Bucket No. Each slot can be allocated to an incoming data packet to store the three attributes it carries: "source IP address-source port-destination IP address-destination port", "acknowledgment number", and "timestamp".

[0153] like Figure 3 As shown, the specific implementation of the abnormal RTT sample filtering part includes the following steps:

[0154] S31: Receive RTT samples;

[0155] S32: The data packets carrying flow labels generated in the RTT sample generation section Compared with T, if If the above steps are not executed, proceed to step S33; otherwise, proceed to step S34.

[0156] S33: Will Discard and end subsequent operations;

[0157] S34: Connect the stream label f ​​with To form a new pair And send it to the high RTT stream tracing section for processing;

[0158] like Figure 4 As shown, the specific implementation of the high RTT stream tracing part includes the following steps:

[0159] S41: Initialize the RTT stream tracing section, which contains a two-dimensional bucket array of height d and length w. Each bucket in the array has two fields: a stream label field (F) and a total RTT count field (S), which are used to record the label information of the corresponding stream and its cumulative RTT value, respectively. In addition, this section also configures d independent mapping functions from stream labels to bucket indices. Set the stream label field of each bucket to empty and initialize the total RTT count field to 0. The mapping function is defined as follows: ,in These are mutually independent and uniform hash functions used to map stream labels to the corresponding positions in the i-th row of a two-dimensional bucket array C;

[0160] S42: Receive the binary tuples formed by the valid samples obtained from the aforementioned partial filtering. ;

[0161] S43: Calculate its value in the first row of the bucket array according to formula (4). subscript ;

[0162] S44: Determine the storage bucket [ [Is the data stream recorded? If yes, proceed to step S45; otherwise, proceed to step S46.]

[0163] S45: Increase its total RTT count field according to formula (5) Then the operation of the high RTT stream tracing section ends;

[0164] S46: Judgment [ ].Is ID empty? If it is empty, proceed to step S47; otherwise, proceed to step S48.

[0165] S47: Transfer variables during program runtime and Change them to 0 respectively. ;

[0166] S48: Transfer variables during program runtime and Modify them respectively to and [ ].S;

[0167] S49: Using the same steps, check the buckets mapped to rows 2 through d in the two-dimensional bucket array C for this data stream, and update the runtime variables. , , and ;

[0168] S410: Judgment Is it empty? If it is empty, proceed to step S411; otherwise, proceed to step S412.

[0169] S411: Calculate the replacement probability according to formula (6) and using existing programming languages The function generates a pseudo-random number r;

[0170] S412: Move the storage bucket The ID and total RTT count fields in the data are updated to f and f, respectively. ;

[0171] S413: Determine whether r satisfies formula (7). If it does, execute step S414; otherwise, proceed to step S415.

[0172] S414: Move the storage bucket The ID and S fields are updated sequentially with the flow label f ​​carried by the arriving data packet. Then end the operation for that part;

[0173] S415: End the operation of the high RTT stream tracing section;

[0174] like Figure 5 As shown, the specific implementation of the abnormal RTT stream detection part includes the following steps:

[0175] S51: After the measurement cycle is completed, the inspection begins;

[0176] S52: Initialize the exception stream set;

[0177] S53: Check each bucket in the two-dimensional bucket array in turn;

[0178] S54: Determine if the total RTT count in the storage bucket exceeds the preset normal threshold. If yes, proceed to step S55; otherwise, proceed to step S56.

[0179] S55: Add the ID field recorded in this bucket to the candidate high RTT stream set;

[0180] S56: No further processing is performed; the data packet is forwarded out of the switch's output port.

[0181] S57: Transmit the generated set of abnormal streams to the control plane.

[0182] This invention deploys the entire abnormal RTT flow detection process on the programmable data plane, eliminating the need for the control plane to participate during measurement. This completely avoids the additional communication overhead caused by control plane interaction, compared to traditional schemes that only collect RTTs on the data plane. Unlike existing Sketch structures that only measure flow size, this invention's RTT sample generation module fully distinguishes packet types, generating high-quality RTT samples with only minimal hashing and memory access. Simultaneously, the abnormal RTT sample filtering module designed in this invention incorporates the congestion characteristics of modern data center networks. By setting reasonable normal thresholds, it filters out invalid RTT samples caused by retransmission mechanisms, significantly improving the reliability of each flow's RTT data. Finally, unlike existing Sketch-based large flow detection methods, this invention's abnormal RTT flow tracking module utilizes the tail-skewed distribution of each flow's accumulated RTT value to prioritize recording abnormally high RTT flows and actively discard normal flows, effectively saving storage space in the tracking module and improving abnormal flow detection efficiency.

[0183] To further demonstrate the beneficial effects of this invention, this embodiment was simulated using an Intel® Core™ i7-8750HQ @ 2.20GHz CPU with 8GB of RAM. Since this invention is the first solution to this problem, only the effects of the proposed mechanism on the MAWI-24 and MAWI-25 datasets are shown.

[0184] The experiments were conducted using 15-minute real-world traffic datasets from both MAWI-24 and MAWI-25. The storage space for measurements was varied from 64KB to 1024KB, and the normal threshold for RTT was set to the 90th percentile of all RTTs sorted from smallest to largest. The experiments were evaluated using four metrics: precision, recall, F1 score, and throughput. The formula for calculating precision P is shown below:

[0185] ;

[0186] in, This refers to the set of abnormal RTT streams reported in this invention. It is the set of all abnormal RTT streams in a real environment.

[0187] The formula for calculating recall R is as follows:

[0188] ;

[0189] The F1 score is the harmonic mean of precision (P) and recall (R), which comprehensively measures the overall performance of a detection algorithm. Throughput is the number of data packets an algorithm can process per unit of time (in seconds); a higher throughput indicates a faster processing speed.

[0190] Specific experimental results are shown in Figures 6 to 13. This invention demonstrates stable and excellent abnormal RTT stream detection capabilities under different storage space configurations, with accuracy exceeding 0.95 and recall exceeding 0.90. Furthermore, the detection performance significantly improves with increasing available storage space. Specifically, even under extremely limited storage resources (e.g., only 64 KB), such as... Figure 6 , Figure 7 and Figure 8 As shown, the present invention achieves precision, recall, and F1 score of 0.962, 0.911, and 0.935 on the MAWI-24 dataset, respectively; Figure 10 , Figure 11 and Figure 12 As shown, the present invention achieves precision, recall, and F1 scores of 0.971, 0.905, and 0.936 on the MAWI-25 dataset, respectively, fully verifying that the present invention still possesses reliable detection performance under limited space conditions. Furthermore, when storage space is relatively sufficient (e.g., 1024 KB), as... Figure 6 , Figure 7 and Figure 8 As shown, the present invention improves the precision, recall, and F1 score on the MAWI-24 dataset to 1.000, 0.994, and 0.997, respectively; Figure 10 , Figure 11 and Figure 12 As shown, the accuracy, recall, and F1 score of this invention on the MAWI-25 dataset are 0.999, 0.992, and 0.996, respectively, indicating that this invention can achieve nearly 100% abnormal RTT stream detection performance under high resource configuration.

[0191] On the other hand, combining the throughput curves of the present invention on the MAWI-24 dataset shown in Figure 9 and the throughput curves of the present invention on the MAWI-25 dataset shown in Figure 13, it can be seen that the present invention also has extremely high processing throughput, and the processing speed can still be stably maintained at about 25 Mpps even in the worst case. This is mainly due to the simple yet efficient hashing and memory access process adopted by the present invention on the data plane side: in the RTT sample generation stage, only 1 hash and c memory accesses are required in the worst case; in the abnormal RTT stream tracing stage, only d hashes and d memory accesses are required in the worst case. Since c and d are usually set to extremely small positive integers such as 1, 2, and 4, the overall processing latency is low and the hardware execution efficiency is high, thus achieving high throughput. In addition, the throughput of the present invention will decrease slightly as the available storage space increases. This is because a larger storage space can generate more effective RTT samples, which increases the amount of data processed in the subsequent tracing stage, resulting in more frequent replacement and update operations, which slightly affects the overall processing speed.

[0192] Example 2: This example provides a high round-trip latency flow detection system for high-speed network flows, including:

[0193] Generation module: used to acquire network data streams and generate round-trip delay samples of the network data streams;

[0194] Filtering module: used to determine the validity of the round-trip delay samples and filter out valid samples;

[0195] The statistics module is used to perform statistical analysis on the valid samples within a statistical period within limited storage resources and obtain statistical results.

[0196] Output module: Based on the statistical results, the data streams corresponding to records whose cumulative round-trip delay values ​​exceed a preset abnormal threshold are identified as abnormally high round-trip delay data streams and output.

[0197] The specific functions of each module described above are explained in the relevant content of the method in Embodiment 1, and will not be repeated here.

[0198] Example 3: This example provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of Examples 1.

[0199] Example 4: This example provides a computer device, including:

[0200] Memory, used to store computer programs / instructions;

[0201] A processor for executing the computer program / instructions to implement the steps of the method described in any of Embodiment 1.

[0202] Example 5: This example provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the method described in any of Examples 1.

[0203] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

[0204] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0205] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0206] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0207] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0208] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure and not to limit its protection scope. Although this disclosure has been described in detail with reference to the above embodiments, those skilled in the art should understand that after reading this disclosure, they can still make various changes, modifications or equivalent substitutions to the specific implementation of the invention, but these changes, modifications or equivalent substitutions are all within the protection scope of the pending claims.

Claims

1. A method for detecting high round-trip delay flows in high-speed network flows, characterized in that, include: Acquire network data streams and generate round-trip delay samples of the network data streams; The validity of the round-trip delay samples is assessed, and valid samples are selected. Within limited storage resources, statistical analysis is performed on the valid samples in a statistical period to obtain statistical results; Based on the statistical results, the data streams corresponding to the records whose cumulative round-trip delay values ​​exceed the preset abnormal threshold are identified as abnormally high round-trip delay data streams and output. The step of acquiring the network data stream and generating a round-trip delay sample of the network data stream includes: TCP packets are filtered from the network data stream, and a flow label is generated for each TCP packet based on the source and destination network addresses and port information; For each TCP packet, a matching request data unit is searched in a pre-built one-dimensional storage structure based on its flow label: If no match is found, the current TCP packet is determined to be a request data unit; a free storage bit is found in the one-dimensional storage structure to store the flow tag and timestamp of the current TCP packet; when the request data unit is matched by a new TCP packet, the round-trip delay sample is calculated based on the difference between the timestamp of the new TCP packet and the timestamp of the request data unit. If a match is found, the current TCP packet is determined to be an acknowledgment data unit, and the round-trip delay sample is calculated based on the difference between the timestamp of the current TCP packet and the timestamp of the matched request data unit. The statistics are implemented using a two-dimensional storage array. Each storage unit of the two-dimensional storage array records the stream label of the data stream and its cumulative round-trip latency value. Each data stream is associated with candidate storage units in different rows of the two-dimensional storage array through multiple independent mapping functions. The statistical analysis of the valid samples within a statistical period within limited storage resources includes: For each valid sample, examine all candidate storage units determined by multiple mapping functions; If any candidate storage unit records the valid sample, then update its cumulative round-trip latency value; If none of the candidate storage units have recorded the valid sample, then select one of the empty candidate storage units to record the valid sample. If all candidate storage units are occupied, the valid sample replaces the existing record in one of the candidate storage units according to a predetermined replacement strategy.

2. The high round-trip delay flow detection method for high-speed network flows according to claim 1, characterized in that, During the search for free storage space, if the recording time of a certain storage space exceeds a preset threshold, the storage space is cleared and designated as a free storage space. If there is no free storage space in the one-dimensional storage structure, the processing of the current TCP data packet is abandoned.

3. The high round-trip delay flow detection method for high-speed network flows according to claim 2, characterized in that, The one-dimensional storage structure includes a bucket array, which includes multiple buckets, each bucket including multiple slots, and each slot is used to store the stream tag and timestamp information of a request data unit; For each TCP packet, the corresponding bucket is located using a hash function based on its flow label, and all slots in the located bucket are traversed to search and match the requested data unit.

4. The high round-trip delay flow detection method for high-speed network flows according to claim 1, characterized in that, If all candidate storage units are occupied, then according to a predetermined replacement strategy, the valid sample replaces the existing record in one of the candidate storage units, including: The replacement probability is determined based on the round-trip latency sample value of the current data stream and the minimum cumulative round-trip latency value recorded in the candidate storage unit. When the replacement probability is greater than the generated pseudo-random number, the replacement operation is performed. The formula for calculating the replacement probability includes: ； In the formula, For the probability of replacement, This represents the sample value of the round-trip delay for the current data stream. It is the minimum cumulative round-trip latency value recorded in the candidate storage unit.

5. A high round-trip delay flow detection system for high-speed network flows, characterized in that, include: Generation module: used to acquire network data streams and generate round-trip delay samples of the network data streams; Filtering module: used to determine the validity of the round-trip delay samples and filter out valid samples; The statistics module is used to perform statistical analysis on the valid samples within a statistical period within limited storage resources and obtain statistical results. Output module: Based on the statistical results, it identifies the data streams corresponding to records whose cumulative round-trip delay values ​​exceed a preset abnormal threshold as abnormally high round-trip delay data streams and outputs them. In the generation module, the step of acquiring the network data stream and generating a round-trip delay sample of the network data stream includes: TCP packets are filtered from the network data stream, and a flow label is generated for each TCP packet based on the source and destination network addresses and port information; For each TCP packet, a matching request data unit is searched in a pre-built one-dimensional storage structure based on its flow label: If no match is found, the current TCP packet is determined to be a request data unit; a free storage bit is found in the one-dimensional storage structure to store the flow tag and timestamp of the current TCP packet; when the request data unit is matched by a new TCP packet, the round-trip delay sample is calculated based on the difference between the timestamp of the new TCP packet and the timestamp of the request data unit. If a match is found, the current TCP packet is determined to be an acknowledgment data unit, and the round-trip delay sample is calculated based on the difference between the timestamp of the current TCP packet and the timestamp of the matched request data unit. In the statistics module, the statistics are implemented using a two-dimensional storage array. Each storage unit of the two-dimensional storage array records the stream label of the data stream and its cumulative round-trip latency value. Through multiple independent mapping functions, each data stream is associated with candidate storage units in different rows of the two-dimensional storage array. In the statistics module, the step of performing statistics on the valid samples within a statistical period within limited storage resources includes: For each valid sample, examine all candidate storage units determined by multiple mapping functions; If any candidate storage unit records the valid sample, then update its cumulative round-trip latency value; If none of the candidate storage units have recorded the valid sample, then select one of the empty candidate storage units to record the valid sample. If all candidate storage units are occupied, the valid sample replaces the existing record in one of the candidate storage units according to a predetermined replacement strategy.

6. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-4.

7. A computer device, characterized in that, include: Memory, used to store computer programs / instructions; A processor for executing the computer program / instructions to implement the steps of the method according to any one of claims 1-4.

Images